Friday, April 27, 2007

Liveness Rule #2: Asynchronize Everything for Writes

And by everything, I mean at least as much as you possibly can. Writes are slow for all the same reasons brought up in the caching post, and then some. With writes you can add more potential performance speed bumps to "database hit list", such as table locking or row locking depending on how the database handles it and disk write speeds.

Caching works wonders on the Read side and because caching allows you to take the Read load off the database, it actually does help a lot to increase Write performance. What else can we do to increase liveness for operations that might take more than a fraction of a second? The answer: Go asynchronous.

Good examples of things that should be asynced:

  • Sending mail notifications. This usually takes a significant amount of time, so you should always async this
  • Database updates that the user doesn't need to see a response for. For instance, logging, counters (how many times has the user viewed item X), etc?

How Do I Implement Asynchronous Ops?

If you're using Java, the java.util.concurrent package is your new best friend. This has everything you need to asynchronize things in a simple and elegant way. The classes you will be interested in are Executor, ExecutorService, Executors (static factory methods). An Executor is essentially just a queue of tasks with a thread pool to dedicated to executing those tasks.

Here's a quick getting started guide to asynchronicity heaven:

  1. Create the ExecutorService:
    ExecutorService executor = Executors.newFixedThreadPool(THREAD_POOL_SIZE);
    Keep this available as a global variable, you can use this same Executor throughout your entire application if you want.
  2. Now pull out any code you want to execute asynchronously into small tasks that implement Runnable. For example, an emailer task might look like this:
    public class EmailTask implements Runnable {
    private Email email; // the email we want to send.
    public EmailTask (Email email) { = email;
    public void run() {
  3. Now when executing the event action, create a new instance of your Task class and pass it to the ExecutorService's execute command:
    EmailTask emailTask = new EmailTask(email);
  4. Done! That task will execute at some point in the future and you can respond to your user immediately without making him/her wait.

Another huge benefit of Asynchronizing things is that application automatically becomes more scalable and doesn't degrade as traffic goes up. Lets say you did the above example synchronously and it took 2 seconds on average to send an email. That may be fine when you have a small number of users (although 2 seconds is way too long to make someone wait), but what happens if you have have 1000 users? That's a total of 33.3 minutes (2000 seconds) of waiting time just for 1000 users. And lets say you have a maximum of 200 processing threads for your app server (Tomcat default). It won't take much to run out of processing threads if your traffic goes up, so now you have people waiting to get the application to respond at all. Pretty soon your app is on it's knees, dying a slow painful death. You can avoid all of these problems by taking a few extra seconds to implement it like I've shown above.

Learn it, live it, love it. Once you start getting into the habit of doing things like this, there's no turning back and you can rest easy knowing that you've future proofed your application.