Tuesday, November 19, 2013

Openstack consumption model : DiY vs Enterprise

Here are some personal comments on Openstack adoption in the industry and their statistics.

Openstack statistics: 

While the Openstack consortium is putting great effort in diffusing its statistics  to market / broaden its adoption ( you can see the latest batch here:  http://www.openstack.org/blog/2013/11/openstack-user-survey-statistics-november-2013/ ) you have to take them with a grain of salt.

One of the thing that is not immediately visible, but you can guess it, is the consumption model of Openstack. Openstack cloud deployment can be broadly classified in two different category: Do it yourself  ( DiY) , and (semi) contracted out  ( aka enterprise cloud).

 DiY vs Enterprise cloud consumption model

My own brew definition (partially adapted-stolen from some of Simon Wardley  blog posts , check out his great blog ):
  • DiY : Complete redesign of an  infrastructure architecture, most of the time as close as possible to a true public cloud architecture ( or exact copy in the case of public cloud provider) and/or often to fit very specific needs. These solution   aim to have the lowest cost possible by reusing as much open source tools out there combine with custom solution.
  • Enterprise: "It's like cloud but the virtual infrastructure has higher levels of resilience at a high cost when compared to using cheap components in a distributed fashion.  The core reason behind Enterprise Cloud is often to avoid any transitional costs of redesigning architecture i.e. it's principally about reducing disruption risks (including previous investment and political capital) by making the switch to a utility provider relatively painless." I.e. they want to be able to run "legacy" application alongside new one without having to throw away a lot of their existing investment ( HW - software - skillset - HR , etc.. ). These Enterprise cloud solution are sometimes delivered/consumed  as (heavily) custom packaged Openstack solution from specific vendor :  HP , mirantis, etc... These solutions are slowly making their way within the heavily regulated company segment ( compliance issues require on premise deployment).
DiY Openstack use internal resource and knowledge, in order to minimize TCO and maximize ROI they heavily rely on open source solution and adapt it to their need. This is why most of the time DiY are almost 100% home-grown solution.
On the other hand the Enterprise ones tend to be hybrid where you have a mix of opensource solution and "enterprise" solution. Sometimes this is also sprinkled with a dose of consulting. End consumer of these cloud are sometime  not aware that the solution they are buying is based on open stack ( ex:  in PaaS – Managed services scenario). By example Swisscom cloud managed services ( offer SAP -ERP - BW - etc..)  relies on Piston cloud who relies on Openstack. 
Often some  key aspect are being bought separately for their enterprise features ( or performance, support)  by ex: -storage : coraid (pNFS) / inktank (CEPH) , and networking : arista- .

Openstack user base is skewed:

Now, why the distinction is relevant to Openstack statistics: Cost savings  open technology , avoid vendor lock-in are the main drivers,  this should be hinting which one of the two is predominantly represented in the users surveyed => DiY.
As a result, the stats are quite skewed toward home grown solution where most of the cloud stack is build using open source solution almost exclusively. This is why CEPH or Ubuntu and other open source solution  heavily dominate the stats but they do NOT dominate  cloud spending ( this is reinforced with the fact that most of the surveyed user have a 1-50 nodes cloud ... ~60% + of the overall and only 20% of the clouds are used for production purpose  ). I would love to see a venn diagram of the different dimensions. If anybody has access to the raw survey data I would be happy to do the diagram for them
By example, for the cloud storage solution of some  Openstack  enterprise deployment in companies I came across  you would see all the usual suspect (EMC - Netapp - HP, etc..). However, they would be sometime relegated as secondary player role while a certain number of newcomers took the front stage. This newcomers obviously leverage the emerging market stage / first mover to grab market share : Inktank ( commercial CEPH version)  , Coraid ( pNFS ,  popular as it allow a natural transition from classical NFS setup ) , and finally the wild card :  Intel who is trying to push its Lustre solution ( from their whamcloud acquisition) into cloud storage . Note: I am not too aware on how glusterFS is fairing..

Openstack , a complex and rapidly evolving environment: 

As you can see Openstack ecosystem is heavily dominated by company using the DiY model.  This is further fuelled by the current state of Openstack which is more a toolkit that you need to (heavily) customize to build your own cloud compared to the self-contained solution like cloudstack, eucalyptus, flexiant, etc.. It makes me feel like openstack is the Debian equivalent for cloud.
However, its adoption is growing heavily in the corporate world (and as a result fragmentation risk too): Oracle has its own home-brew version (but does not contribute back anything and is completely closed) , SAP is starting some effort, Intel is a big proponent (see IDF 13), Big managed services player are using it as their basic building block  ( T-system, Swisscom , Huawei , IBM etc..) .

A tough ecosystem for startups and established company alike:

Complex Ecosystem, fragmentation, heavy reliance on custom solution makes it tricky for company to position themselves  within this environment. Too low and they might be completely ignored due to the heavy DiY proportion , leading to a long struggle ( or death) hoping that the crossing of the desert will come soon.Too high level and they end up fighting with a multitude of Openstack clone as well as with other cloud solution. 
There is no right decision there, maybe using a coherent layered approach across the openstack ecosystem would enable the creation of consistent revenue stream while limiting the race to the bottom competition ( competing on price). I probably will expand more on this concept in a follow up blog post.

PS: as I write this postI came across the following Gartner blog post that echo in some way my thought on Openstack ecosystem http://blogs.gartner.com/alessandro-perilli/what-i-saw-at-the-openstack-summit/ . Again this is to be taken with a grain of salt as Gartner has a long history of being a big VmWare supporter.