Silly Things in IT – The Infrastructure/Application Mismatch

One thing that always fascinates me is how we humans often behave and work a certain way due to the constraints of technology, but then never pop our heads up and realize how odd many of these activities are and that there can be a much better way to do things. Ever watch a kid try to pinch and zoom on their parent’s older blackberry (crying when it doesn’t work). Or seen less technical friends try to use ctrl-alt-delete on your Mac (you know who you are!). While less accessible to the average person, this behavior is just as true in datacenter technologies.

As an infrastructure guy, I’m as bedazzled and proud of advances in feeds and speeds as anyone. However, it’s far too easy to get sucked into this world and not remember a very key fact… infrastructure is a means to an end, not the end itself. Infrastructure exists to run applications. The infrastructure mission statement should be something as simple as:

I exist to run applications. I shall run them quickly, efficiently, and safely. I should minimize the time it takes humans to use me.

Simple concept, but it’s amazing how far we get away from this at times. I’ve been thinking about all of the silly things we’ve been doing because we forget this mission or because technology constraints force us to do so. A lot of the issues come down to a mismatch of abstraction; the units that infrastructure works with don’t match what an application ideally needs. Let’s explore this oddity across a variety of pieces of infrastructure:

Server (Compute + Memory): In the early days of VMware, we all became obsessed with the assumption that there should only be one application+operating system even on the beefiest physical server. Server virtualization is a well-understood tale at this point, but I’m still amazed that we got to the point of always using 15% of a server’s capabilities just because that was how software, management tools, and processes had evolved to work. What a waste! Thankfully, we’ve gotten past this with virtual machines and other containers being the de facto compute abstraction in the datacenter. The prior approach was the human equivalent of a person driving around town alone in an 8-seater mega-SUV. Oh wait…

Storage: A similar mismatched abstraction has long existed in the world of storage. Ask a random storage admin what they deal with every day, and you’ll hear the word “LUNs” many times. These are big chunks of block-based storage that are often the primary unit of allocation, accounting, and actions (snapshots, replication, etc.); a very infrastructure-centric abstraction. Many applications or VMs are often forced to share a LUN, causing all sorts of inefficiency and management challenges. Our industry has certainly realized this and worked on new abstractions. This includes the very popular file-based abstractions of NFS and CIFS, but more recently we’ve seen this take hold in two key areas:

  • Object-based Storage: Popularized by Amazon S3, this has become the storage target of choice for web developers precisely because it is the abstraction that their application wants to deal with.
  • VM-centric abstractions: Even more recently, we’ve seen the rise of storage systems and converged infrastructures that deal directly with virtual machine abstractions. New companies such as Tintri and Nutanix have become popular precisely because this is a more appropriate abstraction for system administrators to deal with. Moving forward, hypervisors will further bridge the abstraction mismatch to their block-based targets through new APIs (such as VMware’s vVols). NB: This is still one of my favorite keynote clipart items.

Network: And mismatched abstractions thrive in networking land as well. At the highest level, networking infrastructure exists to enable safe and efficient communication between applications (or application components) and their users. In theory, the ability to communicate should be independent of the application’s location, but in practice we have ended up with many abstractions that are very infrastructure-centric. The most telling example is an IP address, a unique identifier for an endpoint (yay!), but one that is tied to the location of that endpoint (boo!). While we’ve created many mechanisms to deal with this (DNS, overlay networks, ADCs), the core abstraction mismatch continues to cause challenges. For example, most datacenters have completely abstracted their compute and storage resources, but still require that applications or VMs reside on specific racks due to IP subnet management challenges or do to the physical placement or networking-related appliances (more on that below).

The IP abstraction mismatch is very similar to the older days of telephony. My fellow 40-somethings will remember that telephone numbers used to be exclusively tied to a physical phone in a physical location. You didn’t call Steve, you called Steve’s house or Steve’s office or Steve’s carphone (remember those?). Just as we now have a mobile phone and number that reaches us wherever we happen to be, networking infrastructure is evolving to do the same through tunneling approaches such as VXLAN, STT, and NVGRE.

Security: Last, but certainly not least, is security. So much of today’s security infrastructure comes in the form of appliances placed at the edge of a network. They act as drawbridges over a castle moat, keeping the good guys safe inside and the bad guys outside. This is a convenient abstraction for the infrastructure provider and administrator, but doesn’t match the abstraction that is most important — the channel that allows an application to safely communicate only with the intended user (a human or another application). Making matters worse, the modern world increasingly consists of:

  • Mobile users needing access to these applications anywhere and any time
  • Applications that are themselves comprised of multiple services coming from many locations
  • The rise of SaaS and public cloud deployments

As I mentioned in my last post, a castle’s moat is only effective if all of the good stuff is inside the castle and all of the bad stuff is outside! As with the other infrastructure technologies, the infrastructure-centric edge abstraction must morph into one that is centered around an application and its users. This transition is well underway with many popular new approaches via authentication, access control, containers, and application firewalls, but we still have a long way to go!

Where does this all end up? As we get to more appropriate abstractions, where do we end up? App deployment will ultimately look something like this:

Application developers construct the components of their service (which increasingly includes the list of external web services required). At provisioning time, the application owners indicate their expectations of the infrastructure in some kind of manifest (remember who works for whom!). The deployment may also inherit certain defaults or mandated expectations from their department or company. It’s then the job of the infrastructure to satisfy these requirements automatically, and as efficiently as it can. And while we’re thinking of the future, this infrastructure may be on- or off-premise, in a private or public cloud. With the right abstractions, this shouldn’t matter. While this may seem a little overly simplistic, we’re heading in the right direction, and you can see glimpses of this in the operations of companies such as Google and Facebook.

I’m passionate about new approaches to datacenter infrastructure that fully embrace the above mission statement and take on the mismatched abstractions that hold it back. We’ve already seen some great new companies built around this premise, and I’m positive there are many more to come!

What are some of the other mismatched abstractions you deal with everyday?

Cheers, Steve

9 thoughts on “Silly Things in IT – The Infrastructure/Application Mismatch

  1. Steve,
    After reading the article on Gigaom I came here for more insight. We at Data Central are working on a very similar architecture and using the VM to get right level of abstraction for a mobile first computing on consumer and small business side. I was wondering if we could I have some more details or a meeting.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s