A couple of days ago, I moved another application to Amazon Web Services. This is my third application I've setup on their infrastructure and I have to say, I think I'm addicted to the throwaway nature of Amazon Elastic Compute Cloud. I've got the whole long term, persistent storage thing nailed and can do quick recoveries so I am no longer worried about servers crashing... and boy that's a good feeling.
On our previous dedicated servers, there's always a worry about having a server meltdown because there is no quick way to get a new one up and running. It could take a day or more. Now I just launch new servers for fun. ;)
The big lessons learned:
- Automate absolutely everything. So when something goes wrong, you run a script or two to get a new instance running. And practice, practice, practice doing emergency recoveries (I've had to recover from crashed instances several times now so I learnt this lesson the hard way).
- Scale early. As soon as you notice some performance or memory limitations, start adding more instances to spread out the load. Need performance, put your database on a separate server from app. Need redundancy, replicate your database on two servers. Need memory, fire up some more instances just for caching (memcached, ehcache, jbosscache).
- Keep your database as small as possible. Store as much data as possible on Amazon Simple Storage System (S3), NOT in your database and just store the S3 key to the data in your database. Consider putting any blob or large text fields in S3. This will make it much easier and faster to manage database backups, plus your database will perform better.