Agathon Group - "The Good Stuff"

The concept

The reason grid computing is so cool is that it decouples a service from the hardware on which it runs. Virtualization achieves some of this; one physical piece of hardware can run many virtual machines (which are, in turn, complete software implementations of a real computer). Indeed, many grid computing products rely on existing virtualization software to manage that piece of the puzzle. Grid computing goes one step further and allows those virtual machines to be spread across an entire farm of physical hardware with just one interface to manage it all.

Huh?

Maybe I’ve lost you a bit. (I tend to slip into “techniclese” when talking about stuff like this!) Let’s try this: even if you haven’t heard the term “grid computing” (or “cloud computing”, which is similar), you’ve probably heard of Amazon’s Elastic Cloud Cluster (EC2) product. Pay only for the CPU cycles you need! Pay only for the memory you need! Using S3, pay only for the storage you need! And get it all spread out over an enormous cluster of redundant physical servers. With EC2, you get all of the benefits of building out a large infrastructure without the biggest drawbacks: cost, hardware management, and the requirement for hardware and OS expertise. Maybe you’ve heard of SaaS? Well this is IaaS — Infrastructure as a Service — and it’s the way everyone is building their applications.

But…?

Amazon is the largest grid computing provider, and they have the benefit of being the first major player in the market. Still, EC2 has had a number of significant drawbacks. The lack of several important features (secure websites, multiple static IPs on an instance, persistent/shared storage, PCI compliance) and a mediocre stability record make EC2 a suspect choice for all but the simplest hobbyist site. In addition, at the end of the day, your data resides wholly on someone else’s system — a shared resource on a grid with, potentially, all sorts of adverse activities happening on it. Finally, EC2 can be archaic enough that a whole separate business (e.g., RightScale) has sprung up just to provide a simple, usable interface to the sometimes-very-complicated EC2.

So… what now?

So if you need those features (and chances are you do), what are your options? In the next blog post in this series, I’ll talk a little about our favorite grid computing solution, how they’ve elegantly solved the hardest requirements for solid grid computing, and what it might mean for your organization.