xyzio

This is the first of several articles I’m going to write about Amazon S3.  I’ve done a lot of research and cheap, secure, reliable, and publicly hostable disk space is hard to come by.  Sure, there are some companies such as dreamhost and mediatemple that offer enormous amounts of disk space at a low price but often they put restrictions on the file size, locations, and types of files you can save.  Other companies even give it away for free, but I wonder how they plan to survive by giving their product away for free.  Ads may pay for some of their costs initially, but I doubt they will pay enough as they scale up their operation.  I have a feeling that many of them will simply disappear along with the data entrusted to them.  I invite you to come back to this post in a year and see how many of these hosts are still around.

My search for a cheap, reliable, and secure place to backup my data ended when I found out about Amazon S3 (Simple Storage Service).  S3 is touted as a data storage service and that is all that it is.  There is no limitation on the types or quantities of files one can store.  They charge you on a pay as you go plan and there are no upload and download limits. You simply pay for what you use.  Amazon’s own website is a testament to their experience in building massively scalable services.  These are their rates at this time:

Storage
$0.15 per GB-Month of storage used

Data Transfer
$0.100 per GB – all data transfer in
$0.170 per GB – first 10 TB / month data transfer out

Requests
$0.01 per 1,000 PUT, POST, or LIST requests
$0.01 per 10,000 GET and all other requests*

Considering how much other hosts charge, their rates are very reasonable.  Their storage and transfer rates have been coming down and I think this trend will continue.

Amazon has built a very simple API with a high level of abstraction for its S3 service.  Data is stored in ‘buckets’ and each bucket can contain an unlimited number of objects up to 5GB in size.  Objects can be made public or private which makes it ideal for hosting large files as well as using it for backing up your data.  In addition, you can choose to store your data in either Europe or the United States which gives your data an additional level of geo-redundancy and allows you to deliver files to users around the globe with minimal latency.

As a results of this simple API, Amazon S3 is very versatile and has a diverse eco-system built up around it.  A simple search for amazon s3 tools turns up several tools that abstract away the S3 API and simplify the user experience.  Of these, s3sync and JetS3t are my favorite.  I use jetS3t to back-up important files on my computer to S3 and s3cmd from the s3sync package to upload files I want to share.

In future columns, I’ll write more about these tools and how I use them to simplify my life.