Capacity planning and the cloud

One of the problems that cloud computing is trying to solve is the issue of dealing with capacity planning for companies and the services that they offer. Current datacenters for individual companies, and where relevant, for entire websites, are designed to cope with a particular peak load.

The problem with this model is that it means that a large number of machines may sit relatively idle while waiting for the traffic spike that causes them to be used. Meanwhile, these machines are sucking power, wasting management cycles, and ultimately iterating over their own lifespan waiting to be used. Altogether, it's a waste of time and resources on a whole number of levels.

Capacity planning is an attempt to resolve that by planning and optimizing your environment to handle what you need, and provide room for spikes and expansion without wasting resources.

The hope of cloud computing is that you have a generalized group of machines which you can use and expand into when you need to, while letting other projects, applications, and even companies, do the same on the same hardware. That's why Amazon's EC2 is called the elastic cloud, because it will allow usage to grow and expand and then retract back again.

O'Reilly have just announced The Art of Capacity Planning, a book written by a guy with a wealth of experience, John Allspaw, Operations Engineering Manager at Flickr.

Some of the techniques and solutions shown here in the book are common sense, others ned to be explained to some IT managers and businesses who approach the problem from the wrong direction. I like, for example, the advice at the beginning of the chapter on predicting trends:

Before you get too excited about charting massive growth and putting new servers in place to handle the deluge, let me remind you of one of the key economic factors you need to deal with: buying equipment too early is wasteful.

The chapter goes on to cover some basics on generating graphs, generating trend lines in your graphs and predicting when you will run out of a particular resource. And a later appendix goes on to look at the use of virtual computing and cloud computing as part of the capacity planning process, and how different companies are already using EC2 and S3 for deploying applications.

I'm preparing a more detailed review, but the book is definitely worth a read.