Decrypt

I’ve been noticing again many questions about the differences inherent in choosing between a Cloud infrastructure such as AWS (Amazon Web Services) and a traditional physical infrastructure. Firstly, there are a certain number of preconceived notions on this subject that I will attempt to decode for you. Then, it must be understood that each infrastructure has its advantages and disadvantages: a Cloud-type infrastructure does not necessarily fulfill your requirements in every case, however, it can satisfy some of them by optimizing or facilitating the features offered by a traditional physical infrastructure. I will therefore demonstrate the differences between the two that I have noticed, in order to help you make up your own minds.

One year already!
Firstly allow me to direct you to a previous article I wrote on this subject nearly a year ago: Outlook is Cloudy … So much the better! (French version). I won’t paraphrase what I have already explained, purely in the spirit of DRY (Don’t Repeat Yourself :o))  and I will capitalize on the experience I have gained in this topic over the past year (Cloud Computing in general and AWS – Amazon Web Services – in particular), to answer some legitimate questions I have encountered since then.

The Framework

Cloud
As a reminder, there are several types of Cloud possibilities and I will stick with the AWS types which are infrastructure-oriented services, rather than the Google-type services (GAE – Google App Engine), to mention just one, which offers a running environment for your web applications developed with the APIs provided (similar to a framework). In fact, regarding the latter, we can’t speak for clients (they are the ones holding the credit card) about infrastructure management, because we upload our application using the APIs provided and leave the entire infrastructure management to the service provider. It doesn’t mean it’s less about Cloud computing, but simply about a Cloud service that’s more PaaS-oriented than infrastructure-oriented.

Several abstraction layers: each editor directs its service to one or more layers.

Physical infrastructure
As far as physical infrastructure is concerned, I will examine the notion of self-hosted infrastructure and equally the notion of infrastructure supported by a hosting provider. Similarly, I’ll also look at infrastructures based directly on hardware, as well as those based on virtualized environments. Cloud computing is also based on virtualization, but we are not so interested in that technology here, rather the way in which it is provided to clients (you). In fact, you can simply start up instances via a console, as you do for EC2s if you have an ESX (VMware) for example, but it involves “only” a hypervisor which partitions the physical servers into several virtual computers. You will still have to take care of buying equipment (blades, etc.), configuring the network, etc. But we will come back to these details later.

Cloud computing = Systems administrators marked down?
Yes, the sales are on! Are you looking for a sweater, a jacket, … a systems administrator? I have often come across people who think that Cloud (in the case of AWS) will enable them to get by without an experienced systems administrator, and to build an infrastructure with inferior competence. The answer is obvious: WRONG!

Perhaps a clever sales pitch can convince you that the various services are so user-friendly, you can do it all yourself, and that prepackaged AMIs (Amazon Machine Image) will make life easy, but it goes without saying that once you have started up the EC2 instances, you connect to the computers (SSH / port 22 for Linux and TSE / port 3389 for Windows), then you have to set the parameters, do the fine-tuning, etc.

What applies to systems administrators faced with AWS applies equally to systems architects in the context of Cloud computing services providing access to higher layers (PaaS like Google App Engine). A person with experience in the field, able to set up the infrastructure requirement is needed: the tool may change but the skills must still be available. Note however that if you use GAEs, you don’t need a systems administrator for the application. If the Cloud service editor is offering a service in a given layer (HaaS, IaaS, PaaS, etc.), there is no longer any need for people to deal with the lower layers. However, we do accept the framework supplied by the Cloud editor.

The systems administrator cannot be done away with, but his role is changing: he is becoming more and more of a developer. Indeed, being able to pull up resources on the fly will enable infrastructure management to be scheduled and automated via scripts which will call up the APIs provided by Amazon enabling communication with its web services. Everything at Amazon is web service: EC2, EBS, SQS, S3, SimpleDB: the only non-SOAP or REST operations that exist are when you connect directly to EC2 instances that you called up via web service or when EC2 instances dialogue with the EBS that you called up via … I’ll let you guess.

The administrator can then, rather than going into the computer room, add a disk, connect a server (which would be the case with a physical architecture) or else pick up the phone and ask the host to do it (fetch a coffee… call again… take a Xanax or a Prozac pill…  ;ob), request resources via a script in Ruby or Python… You can then take automation of a Cloud infrastructure much, much further, with a set of scripts and tools (see Installing an infrastructure on AWS: Best practices! (French version)).

The systems administrator’s craft is therefore evolving between a physical infrastructure and an AWS-type Cloud infrastructure: he is becoming more and more of a developer. But the systems administrator remains essential nevertheless.

Elastic is fantastic!
As I mentioned earlier, one of the crucial differences between the two types of infrastructure is the flexibility and dynamism provided by the Cloud solution, compared to a traditional physical architecture (whether it is based on virtualization or not). That means the elimination of the time it takes to install and set up the logistics (equipment purchase, installation of the OS, connecting to the network – the physical network and the configuration of the interfaces, etc.). Likewise, when you no longer need an item of hardware (EC2 virtual instance, EBS volume, S3 object, etc.), you return it to the resource pool: it will be reinitialized so that none of your data can be retrieved, and made available again until the next web service call.

You also have complete access to certain elements such as security groups (firewall) set for each instance… And that’s very useful. It’s very practical, particularly compared to a traditional hosting provider: do you remember the last time you had to change the firewall rules?  Would you like a Xanax? Are you sure?  ;ob

But it’s not simply about the pros and cons of purchasing servers versus running instances. AWS are backed by datacenters which are already industrially organized and tested. All the safety standards that need to be met in terms of fire protection, computer cooling, redundant electrical power supply, physical security against break-ins, distributing the hardware across 2 or more physical datacenters for disaster recovery, etc. entail a colossal initial investment and even when everything is installed, you still won’t be able to recreate the same quality within your company (99% of the time, in any case). You can get all that however, or part of it, with a traditional hosting provider.

But there are also all the more software-like services, such as data durability management (redundancy/replication as on the EBS and on S3), accessibility or high availability, monitoring hardware (to be alerted when physical components are showing signs of weakening), procedures for breakdowns, etc. I will let you read The 9 Principles of S3 (French version), so you understand just how many concepts are included. You won’t get all that with a traditional hosting provider (and forget about having it @Home ;ob). The quality of the S3 service is effectively a huge advantage, especially compared to current pricing… Let’s talk about prices!

The cost
There are no fixed rules. With Cloud, you pay for the resource by the hour and when you stop using it, you stop paying. Instances (Linux at the time I’am writing the article, but Windows will come soon) can also be reserved on the AWS for one year or for three years: this is known as Reserved Instances: you pay a one-time fee at the start, and afterwards you pay the hourly usage charge at a discounted rate, which leads you to a tipping point starting from a certain percentage of the resource use over the year (or three years). For more information, click here. To easily calculate how much your infrastructure will cost you, take advantage of the new calculator provided by Amazon: Simple Monthly Calculator. In all cases one part is related to the hourly usage and another to your traffic/volume stored.

Simple Monthly Calculator

You can do the comparison with the cost of your local infrastructure or with that of your hosting provider. The Cloud set price is particularly attractive in the following cases:

• The set price is unbeatable for POCs, demonstrations/presentations or architecture load/validation tests.
• It is very attractive for applications or APIs mounted on an SaaS-type economic model and for which you need to spend money on their resources only when the clients pay to use the abovementioned APIs.
• It is a good price for the social applications found on Facebook, for example, and which can take off overnight thanks to the social media phenomenon and may experience a boom (or a drop) in hits.
• It is also cost-effective when you are launching a new company, or a specific project within a larger entity and you do not wish to invest heavily in logistics right at the start.

For all the many other specific cases among you, you’ll have to do your own calculations.

Slacking off? No, of course not …
Usually, no matter what type of infrastructure you have, the same components and mechanisms should be installed. However, it must be acknowledged that often the cosy aspects of “home”-hosted infrastructure can lead to a certain lack of rigor on many issues. The fact that Amazon, with its AWS, offers a dynamic and volatile solution for these EC2s, compels you to install mechanisms (which should be standard) in order to consider failure or disaster recovery plans more seriously, given the volatile nature of the tool, and to identify the important data with the aim of ensuring data durability (EBS, S3 backups, etc).

Conclusion
The evolving duties of infrastructure management can be clearly seen in this first part: from handling physical resources by means of APIs, underlying mechanisms ensuring data durability, availability of services, etc. right up to server power supply and the physical security of datacenters, all of which are supported transparently. The end result: one should “only” see the API which dialogs with a distant server. That is the difference with physical infrastructures. The virtualization (which is one facet of AWS) that we know and have already been using for some time now, is used by Amazon: it’s not so much a technical revolution  - even though I don’t deny the complexity of the implementation and support that goes into it – as the service offered with it, which provides the real added value.  It is matched with a new ‘pay-as-you-use’ aaS (as a Service) economic model. This has enabled the emergence of some applications (such as the games found on social networks), which only a few years ago would otherwise have been compromised by the initial investment.

We will examine the other aspects differentiating these architectures in my second article on the subject.

Frédéric FAURE @Twitter @Ysance