RAM in the cloud

The idea that RAM is cheap seems to go without saying these days. Linux/Windows data centers are going 64-bit and enterprise software seems to get more RAM hungry in each new version. When operating a data center, the costs of manpower, software, and disks seem to all add up to vastly more than the cost of RAM. Facebook's well-publicized memcached based architecture seems to add credence to the notion that it's cheaper to scale up with RAM than with any other component.

But in the cloud, how cheap is RAM really? Let's take a look at a few cloud providers and see how their pricing works out.

Amazon EC2

Amazon EC2 Pricing

With Amazon EC2, let's focus in Micro, Small, and Large instances. Micro instances have 632MB RAM, Small instances have 1.7GB, and Large instances have 7.5GB. As the instance type goes up, so too does IO priority and available CPU resources (burstable 2 units, 1 dedicated unit, and 4 units, respectively). In decimal math, that means 31.6GB/$, 20GB/$, and 22GB/$ as we go from Micro to Large. That means a significant savings if we can cluster our applications out into more smaller instances, which gives us access to more parallel CPU and IO usage as well.

Rackspace Cloud

Rackspace Cloud Pricing

In the Rackspace Cloud, things seem to scale out linearly, until we hit the largest server size, where we don't quite get all the RAM we did at each increase in each of the lower tiers. Since there isn't super-cheap add-on storage here like there is with Amazon EBS or S3, disk looks like it could be an equally limiting factor when scaling up. The cost of RAM is a consistent 17.1GB/$, although there are more options in the midrange than in EC2. Each server seems to get 4 virtual CPUs, but with larger sizes getting higher priority. Still, it's hard to imagine that you'd get more CPU capacity on a 4GB server than you would on 4 1GB servers, since you're accessing far more CPUs overall.

Terremark vCloud Express

Terremark vCloud Express Pricing

The pricing table Terremark vCloud Express seems to clearly indicate that RAM is far more expensive than CPUs, where double the cost gets you eight times the CPUs but only double the RAM (at about 17.1GB/$). And it seems like two 1GB/1VPU servers would access more overall resources than one 2GB/1VPU server, at the same cost.

GoGrid Cloud Hosting

GoGrid Cloud Hosting Pricing

With GoGrid Cloud Hosting, RAM appears to be the only metric, with the basic unit of cost being "Server RAM Hours," which is 1GB for an hour (which comes out to 5.39GB/$ for the most "cloud-like" pay-as-you-go plan and goes to 20.5GB/$ at the highest-end pre-paid plan). Being able to divide an application into more smaller (by RAM) instances seems to be a clear win for getting more of all the other resources (CPU, IO) at no additional cost.

What does this mean for cloud application design?

The fact that RAM seems to be the main pricing metric may mean less than it seems. After all, for many large deployments, things like manpower and development effort can mean a lot more than hardware infrastructure costs.

However, if we do focus on cloud hosting costs, it seems clear that RAM is our most expensive resource. If we can somehow divide our applications into bite-sized memory chunks and cluster them, we can get access to a lot of CPU and IO resources almost "for free".

The biggest ramification I see from this analysis is in application framework overhead. In theory, if we have only variable memory usage that increases linearly as our user base scales up, then at least in our web application layer, we can save costs and access more resources by using small instances and doing an elastic scale-up more often. However in practice, there is overhead from both the OS and our application framework (incl. application server memory) than can wipe out these gains. For instance, if our OS and framework use a svelte 64-128MB of fixed memory, then we can have a lot of potential gains by utilizing EC2 micro instances. With about 80% of our memory available for requests, the relatively cheaper memory of a Micro instance and the greater distribution of load works in our favor. But if our OS and framework use more like 512MB of fixed memory, then we have only 19% of our memory available for users, making Micro instances totally uneconomical.

A more detailed analysis would also need to take into account whether or not a larger fixed overhead results in a lower variable overhead, although in my experience, frameworks that consume a lot of fixed memory also consume more variable (load-dependent) memory. So for now, it feels like if one were to engineer an app for performance and economy in the cloud, memory utilization is the metric to watch.