Is AWS the right host for us?

Short answer: probably not for 80% of sites out there. But, it is right for us. To find out why, read on. For a quick guide, skip to the conclusions ;-)

Question of Cost

AWS: generally more expensive for hardware

For the hardware, AWS is expensive compared to traditional hosting (shared or dedicated). However, AWS gives you greater flexibility than a traditional host:

  • If you find yourself with too much capacity you can turn off a server or two and stop paying
  • If you find yourself with too little... you can turn them back on

If you have spiky traffic (e.g. slow periods on weekends) then with traditional hosting you will have to pay for spare capacity all the time. With AWS this isn't the case and it can result it lower costs for some sites.

AWS: Redundancy is cheaper

On a traditional host you might need two web servers to cope with your levels of traffic. Your business can't afford to have these go down so you have two backup web servers, always on standby and always costing money.

On AWS you can fire up a new server in minutes. So, instead of having two standby servers you might have one, or even none and just fire them up when something goes down (AWS can even do this for you). This isn't a luxury you get with most traditional hosts as it can take them some time to set up new servers.

Cost of Staff

Hardware costs aren't the only cost. You also have the cost of staff to consider. With dedicated hardware you might need to employ someone to do your sysops far sooner than with AWS. With AWS you can find far more information out there on how to do things and have your existing team maintain the infrastructure more easily. You can also afford mistakes as you can simply start a new server if you break one ;-)

Testing is Cheap

Testing new code can be really cheap with AWS. If we have a big feature to roll-out and want to give it a thorough working over before releasing then we can start up an entire farm of servers with the new code. When we're done, we kill the farm. No more paying for test servers to sit around all day, every day, not being used.

S3 is Cheap

It's worth pointing out that S3 can be cheaper compared to most CDNs. You can use S3 without using any other part of AWS if you wish. S3 is not a CDN itself but does do a good job of serving static content. CloudFront is AWS's CDN that is built on top of S3 and performs much better. CloudFront is more expensive than S3 but is still worth exploring.

Cost not a decider

For us, cost was not the decider. It's more expensive to be on AWS but the cost isn't so high that it's not worth the other benefits we get.

Scaling

80% of people think that their site needs to scale to epic proportions. Sadly, nearly all of them are wrong. Most sites do not need to cope with a million unique visitors a day. Many sites have fairly slow growth patterns and do not need to make use of AWS to get servers up quickly as they grow.

Some sites are not going to grow wildly but still need scaling.

Some sites (such as ours) are rapidly changing what they do and adding new features. In this case it can be incredibly hard to determine what your hardware needs are going to be ahead of time. Having to worry about resource requirements every time you add a substantial new feature can really slow your development. As an example, at Conversocial we added some new features for searching Twitter. This new feature had the potential to double the amount of content we handle each day. Did we worry about the performance impact? Yes, for about 2 minutes. Our answer was that we would roll it out and monitor the performance, if Conversocial slowed down then we'd fire up new servers.

RDS: Hosted MySQL

Amazon recently added RDS to their suite of features. RDS is hosted MySQL. It handles pretty much everything:

  • Failover
  • Backups
  • Replication setups
  • Upgrades
  • Point in time snapshots and recovery

It's actually pretty amazing. This was a big deal for us. Not having to worry about all of this freed up a large chunk of our time, it also meant we haven't had to employ someone to look after MySQL for us.

Growing Feature Set

AWS is constantly adding new features. Frankly, the pace is astonishing. There are some features currently lacking that we'd like (e.g. knowing we have SSDs for our disks) but we are confident that these features, or ones we haven't even thought of, will soon come out and make our lives better.

Community Support

There is a thriving community around AWS. The abstracted platform makes it really easy for people to share recipes. Want a Mongo server on AWS? There's a guide for it. We don't have a dedicated sysadmin (yet) and the community support was a big pull for us.

Availability Zones

All hosts have downtime for one reason or another (natural disaster, a bomb or someone destroying their wire to the outside world). If you're a business then you probably worry about this a lot.

AWS has solved this problem by having multiple Availability Zones (AZs) across the world. Each AZ is a data centre that is isolated from all the others. Certain things are shared across multiple AZs, such as machine images and EBS snapshots. EBS is the persistent disk-based storage that you can attach to a running machine.

If one AZ gets wiped off the face of the Earth then you can quickly restore your entire infrastructure on another AZ using your AMIs and snapshots. Alternatively, you can have hot standbys in different AZs that automatically take over when something goes wrong.

It's cool

I'll be honest, AWS is cool. Many technical folks get excited by it. This did play a small role in our decision making, if we want to hire great people then we want to make sure we're giving them opportunities to learn about interesting things. AWS helps us do this :)

Alternatives

Using AWS doesn't have to be an all-or-nothing undertaking. I know of people using AWS solely for their backend processing and web servers, their Mongo database is hosted on a dedicated server with an SSD raid somewhere with a good connection to AWS. Similarly, some companies use AWS solely for bulk processing of data with Hadoop.

Conclusions

AWS is not for everyone. In particular, it's not for you if:

  • You're running just a single server and aren't (honestly) going to need to scale
  • You're running a load of servers and have fairly stable hardware needs
  • Hardware is a high cost for you (as %age of expenditures)
  • Your system hits disk a lot (IO performance on AWS is known to be lacking)

That said, we went with AWS and we're loving it so far. It can be slow at times and we have had an instance die on us (but we started a new one and nobody noticed anything had happened). That said, at some point in the future when Conversocial is massive and we have a better picture of our hardware needs then we might look at a mixed hosting.

Here's what we have on AWS at the moment:

  • Solr for search
  • Web servers running Apache
  • MySQL (via RDS)
  • MongoDB
  • Memcached
  • Backend servers running Celery

Hope that helps. If anyone has any questions, please ask!

Discussion

blog comments powered by Disqus

Colin Howe

I'm Colin. I like coding, ultimate frisbee and startups. I am VP of engineering at Conversocial