Dan Croak: A View of the Cloud from the Ground

 A few terms in the technology industry are frequently misused and misunderstood:

 - Web 2.0

 - The cloud

 - Agile

 - NoSQL

 - HTML5

I’d like to examine one of those terms, “the cloud”, from the ground, as a web developer who works mostly with Boston startups.

Layers

Skimming the Wikipedia article, the section that helps me visualize the cloud is “Layers”:

Infrastructure

In the old days, you had to have your own servers running your application. Now you can use virtual machines like Amazon’s EC2 instances or Rackspace Cloud servers.

What’s cool about this setup is that the providers have an API ("application programming interface" - which allows other programs to interact with it) for people like me to access and use computing capacity as needed. If I have a small website that grows into a well-trafficked website, I first scale the site vertically (imagine a glass of water filling up, representing one virtual machine), then I scale it horizontally (add more capacity by adding more glasses - virtual machines).

When traffic spikes subside, I can get rid of the virtual machines. I don’t own them; I’m renting only what I need and it’s all conveniently metered, like the electricity on your house.

Platform

Easier said than done. There’s still a lot of hands-on work that needs to happen to make it work well.

Enter cloud platforms like Heroku and Engine Yard in the Ruby on Rails world and Google App Engine in the Python and Java programming language worlds.

These guys charge more than pure infrastructure services in return for offering a curated environment at all layers of the app (load balancing, fail over, caching, UNIX flavor, server-side programming languages, database, replication, etc.). That parenthetical alone should make it obvious some of the conveniences and cost savings by replacing sysadmins or support staff with hardware and software. Take a look at Heroku’s architecture to get a visual.

What may not be obvious about these platforms is that they cost almost nothing when you’re a tiny startup getting your product off the ground but ideally scale to the heavens with you. That’s the promise, at least. In practice, there are inevitable scaling problems, but in using these tools, you’re getting a head start.

Application

An interesting thing happens when you put your website in the cloud… you start to replace services that used to happen on your UNIX server with cloud applications. These are distributed and often run by third parties.

Example cloud applications I use, often daily, include:

• Amazon S3 for storing user avatars, music, videos, and other files.
• Google Analytics for tracking analytics.
• Hoptoad for capturing and analzing your website’s errors, written by thoughtbot (the Boston firm for whom I work).
• MongoHQ for storing data in MongoDB, a “NoSQL” database created and maintained by 10gen (which has been funded by Boston-based Flybridge Capital Partners).
• New Relic RPM for managing performance.
• Sendgrid (a TechStars Boulder company) for sending transactional emails like account confirmations (“follow this link to confirm your email”) or purchases (“Your group discount is attached to this email as a PDF.”).
• Wufoo for quick forms such as beta sign ups or even processing one-credit cards like we do on the site for our August event, Cape Code.

A Boston-based cloud application I don’t personally use but is worth mentioning is Cloudant. It’s meant for storing data in CouchDB, another “NoSQL” option.

Many of these services are free at the lowest levels or have 30 day free trials. Totally reasonable for a startup.

Also of interest, many applications themselves run on the same underlying Platform or Instracture layers as your application, which gives you the benefit of decreased latency (time delay) as your different components talk to each other. As examples, Heroku and MongoHQ run on Amazon's infrastructure.

Why the cloud is so… cloudy

I’d love to draw the line of what is the cloud and what isn’t right here:

• The cloud is all the stuff that consists of your running web application.

However, you’ll hear people including basically any internet-enabled service in the broad “cloud” term.

In addition to the previously mentioned tools, as a small business, thoughtbot uses a number of awesome services like Gmail, Github, Basecamp, Highrise, Freshbooks, Pivotal Tracker, and Tender to help us run our business. As a consumer, I dig Foursquare, Twitter, and Netflix. Many of these run on the cloud as well.

Aren’t these things just modern life at this point? 

Well, yeah, but… these days just about every application worth it’s salt has an API ("application programming interface" - which allows other programs to interact with it) . So the definition of “all the stuff that consists of your running web application” may very well include Foursquare venues data or Gmail contacts data or some other service as a source of data that's mashed up to create something new.

So that’s the view of the cloud of the ground. If you’re a Boston startup, take a look at some of these services to keep your costs low and maintain focus on building applications instead of infrastructure (see this post by local developer Mike Champion for additional cloud tools). Like the psychology test where you imagine shapes in clouds, cloud computing is all in what you make of it.

Is your startup taking advantage of all the cloud tools available?

Dan Croak is a web developer at thoughtbot. He makes apps for Boston web startups. Email him at dcroak[at]thoughtbot[dot]com.

Photo Credit: kevindooley on Flickr