S3 usage Statistics.

21 Jul 2017
Submitted by Gangrif

So I've recently decided to give S3 a chance, for media storage.  Mainly for the Iron Sysadmin Podcast episodes.  I've gotten one or two comments, not necesarily complaints, that episode downloads seem to tax my server.  It came down to bandwidth, and while I have some things to tackle to help that scenario, it was a pretty easy decision to give S3 a try.  It's something I've been meaning to try it out anyway, and this seemed like a great use case.

So I setup a bucket for the podcast, and then enabled logging.  Assuming that these logs would be useful for statistics.  Then after a few days I went back to S3's dashboard more or less assuming that there would be some way of seeing stats on my file usage. 

Nothing.  Well, not nothing, I can see just enough to try to get some idea of what it's going to cost me, but even that seems very vague.  I thought maybe I was doing it wrong.  I am an S3 noob after all, right?  So I started googling, and instead of finding how-to's on "Here's how to get reasonable stats out of your S3 logs", I found services that did this as a service.  I'm usually a do-it-yourself sort of guy, but at this point I just wanted to see how my podcast episodes were doing.  So I read through one or two of them, and decided to give S3Stat a try. I signed up for their free trial, and pointed them at my Logs.  Almost immediately they had parsed the log data that was already there, and I was able to look at some numbers.

I'm still in their trial for the moment, but I've released two new podcast episodes since starting the trial, and I can say that their analysis has been useful.  One thing they do is estimate your S3 costs.  Which will be very helpful (though I havent been trhough a full billing cycle with S3 yet either) as the podcast is a labor of love, and not really making me any substantial money (A big thanks to you patreon folks!).  I think I'll be bringing in enough from Patreon to cover S3 for now, we'll see how that goes over time.  S3Stat is extimating under $5 this month.  AWS is estimating $6, but not showing me a breakdown of what's S3 and what's not.  We'll see how accurate it turns out to be.

Here are some real usage stats that I just took a screenshot of.

The breakdown is nice, you can really drill into a file, or overall usage in a number of ways.  I'm going to say for now that S3Stat has been a great tool, and worth looking at if you're trying to make sense of your amazon S3 usage.