Bio Stephen Nelson-Smith is CTO at Strategic Blue and one of the founding members of the emerging worldwide DevOps movement, international speaker, coach, mentor and consultant on cloud automation solutions, recognized as a thought leader in web operations. Stephen has also authored the book "Test-driven Infrastructure with Chef"
Software is changing the world; QCon aims to empower software development by facilitating the spread of knowledge and innovation in the enterprise software development community; to achieve this, QCon is organized as a practitioner-driven conference designed for people influencing innovation in their teams: team leads, architects, project managers, engineering directors.
1. I’m Manuel Pais and I’m here at QCon London with Stephen Nelson-Smith, CTO of Strategic Blue, the company behind Cloud Options. Stephen can you explain briefly what do you mean by cloud computing being a tradable commodity?
Sure, if you look at the way that most people buy and sell cloud at the moment, it’s fairly primitive. You’ve got a large number of cloud providers who sell cloud in a very simple way and in terms of the users they buy the cloud from the cloud providers in a very simple way. What we are beginning to see is the emergence of an ecosystem in which people are starting to look at cloud computing as something which can be traded more like a commodity, like electricity or coal. One of the classic indicators of this is the ability for people to be able to set a price now for something which will be delivered in the future, so this would be what you would call a Forward Contract. So a Forward Contract is something which you can’t get from a cloud provider right now and is not likely to be anything that a cloud provider would want to be able to give you. But a company like Strategic Blue inserts its cloud options service in between the cloud provider and the cloud user as able to buy from the cloud provider on terms which suit the provider and then sell to the cloud user on terms which suit the cloud user. And that interface between the provider via the cloud options service and then to the user enables the ability to do things like offer Forward Contracts and then in the future offer Options Contracts. This is the kind of things that we are talking about.
2. What you are suggesting is a financial model for intermediation between different cloud providers and customers. How does that fit with the kind of lack of standards we see between different cloud providers, each one having their own ways of using their cloud? How do you see those two things fitting together?
Kind of depends when you mean by standards. They certainly have different ways of charging and they certainly have different ways of describing what it is they are providing. But if you look at what the fundamental underlining service that a cloud provider gives you, it’s very much like utility. What they are providing you is some CPU, some memory, some disk, some IO in terms of bandwidth and some IO in terms of writing to the disk. And that set of capabilities is something which is pretty much the same across cloud providers. What you then have is overlays where people then offer a support option or whether they offer something like a customized load balancer or a database service, but the commodity underneath is broadly standards compliant. And especially when you start to look at companies who are building on the OpenStack project as the main way to deliver that cloud, it’s also beginning to look far more like the underlying technologies are the same as well. So while Amazon’s cloud and Rackspace’s cloud may not be the same in terms of the technical implementation is certainly the same in terms of what is delivering.
3. Still there seems to be some technical risk in using a cloud intermediary in more complex environments and when you think about big data or other types of complexity that makes it more difficult to switch between one provider and another or requires more planning in advance, do you think it would make sense for the intermediary in this case to take on some of the technical risks somehow or in terms of support to the client or how do you see that happening?
That is a great question, firstly I guess it depends what you are viewing as the risk. I don’t actually think is very risky moving your environments between cloud providers because we are not doing it in hurry, we are doing it in a way which can be tested, in a way which can be measured, in a way which can be kept very safe. And because the way that we are going about it is by building our own infrastructure as code using Chef what we have is something which we can specify our requirements upfront even wrap acceptance tests around them and then once we’re confident that those tests are passing, we are fairly confident that our infrastructure is built. So I don’t think it’s necessarily a very risky endeavor, but I do agree that when you’re starting to work with large data sets or highly complex clusters, then it’s certainly not a trivial thing to do. But then I think that’s the case whatever happens. If you are going to migrate your systems into a private cloud or moving to a collocation facility, you have these complexities.
As I mention in the question session at the end of my talk, the biggest bottleneck, the hardest thing to get right in the cloud is IO and particularly moving large amounts of data around. But the significant point here is what a service like cloud options provides is what is effectively a pricing signal, so strategically would be able to say to their customers: in 9 months time, if you are prepared to move to Rackspace, we can give you a 17 percent discount. That’s a significant amount of lead time and then if you look at the potential saving then, and obviously I’ve made this number up, but it’s not beyond the bands of possibility, that a very large discount could be offered. If you look at the size of that discount, what’s actually possible to do is for some of the saving to be used to pay an integration partner to help with the process. So that comes back to your next question, to what extend should the financial intermediates get involved in the technical implementation.
I think the answer is not at all, the financial intermediates are responsible purely for the pricing and for the billing part of the structure and this is a standard thing that you see as markets mature. They stop being vertically integrated and specialists fill up, and this is exactly what we are seeing in the cloud market. So in terms of how we actually make that happen, Strategic Blue has alliances with companies that have specific expertise in the area of client migrations. The primary partner that Strategic Blue has is Atlanta Systems, who are experts particularly in Chef but also in Puppet and there are other companies that we’re able to work with. So I think while there is definitely a necessity to hold the users’ hands and to help them, indeed that is absolutely the case, and while it might be necessarily for that to be professional services capability within the financial intermediary, that is really an ace to get the work done and certainly not the core business and not the main area of specialization of the organization.
4. You mention that to move between different cloud providers there would be a timeframe of considerable length for the customer to adapt their infrastructure to work on the new cloud provider and the benefit that you mention on the discount that this customer could have, what would be the timeframe for that? So you are saying for example you’d have 9 months period to plan this move and then how long would the customer have to stay on that new cloud provider to benefit from that discount for example, do you have any idea?
So the cloud options service is basically a risk management service. What’s effectively happening is behind the scenes, Strategic Blue is using their expertise and commodity trading to assess the potential risks of extending offers to customers. So if Strategic Blue says to a customer: we can give you attractive pricing if you move to the cloud for 9 months and will commit for a 6 months period, they are weighting up the relative risks of that. Now what could happen is that 3 months into that position, the customer changes their mind or moves in a different direction in their business, or heaven forbid goes bust. In those situations there’s 3 months of what we had projected we were going to have which we now need to sell but that’s exactly what you have. What you have is the situation where you then need to go back to your customer base and say: “Well a new opportunity has arisen. If you are prepared to move into this 3 month window, we can give you an attractive price and that price might be even more attractive”. This is not uncommon that the ownership so to speak of a commodity will change hands a number of times and I think that you will see as the market emerges in cloud computing that’s not unlike a barrel of oil changing ownership 30 or 40 times as it travels across the ocean. You will see a piece of cloud computing changing hands a number of times and that’s great for flexibility within the cloud ecosystem for technical people, but it’s also great for Strategic Blue because there is a force multiplier in terms of the ability to distribute savings amongst the customers.
5. Going back a little bit to the fact that customers would need to be a bit more prepared to migrate between different cloud providers and would need to plan for that, in your view what would be the most relevant factor to enable that? What should be the first concern of a company that wants to be able to easily migrate between different providers?
The very first thing that you need to be able to do before you even start thinking about this is entering to a relationship with a financial intermediary. So the classic case would be, if you already have an established relationship with a cloud provider and you are a medium sized or large company, the idea of having to go cap in hand to the procurement department asking for permission to use a credit card, start buying for a new cloud provider, it’s not ideal. And then to have to do that a third time is even less ideal. When you then start having to deal with the inflexibilities around the terms and the pricing model, it becomes even more problematic. So this friction between the way that cloud providers sell their services and the way that users want to consume them, is an absolute necessity to be removed. So the very first thing that would need to be done, would be for the person wanting to engage in this process to come and approach Strategic Blue and sign up for the cloud options service. And all that means is you say to Strategic Blue: “I would like to buy my cloud from you and you will buy my cloud from my provider”.
So we are sitting in between the provider and the user in the billing chain- The technical relationship is unaffected, it’s a purely commercial conversation between the two. Once that’s in place, you’ve got a consolidated bill, the ability to add as many cloud providers as you like, because cloud providers are coming to us on a regular basis, asking to be included in our program, we’re signing up new clouds on a regular basis. So if a new emerging cloud emerged and was offering a particularly attractive price, or perhaps were offering a great value add, for example SSD base systems with very high IO capabilities, we would be able to say: “We’ve got this great opportunity for you to work with for example DigitalOcean, maybe you might like to look at DigitalOcean as a person that you could work with”. But because we’ve got that commercial relationship with the cloud user, it’s very easy to start making those things happen.
The next stage once you’ve got the paperwork out of the way which makes all of this possible, and bear in mind that there’s no commitment in that paperwork, it’s just a piece of paper which says: “I authorize you to buy my cloud for me and I will pay you”. Once you’ve got that in place, the next thing is going to be to start thinking about building our infrastructure as code. I think that something that pretty much anybody needs to be doing now anyway if they want to be able to remove risk in terms of the technical complexity of their infrastructure, to be Agile and able to scale and grow and change and also to be able to have disaster recovery capabilities or the ability to move between clouds for financial advantage.
6. You have worked quite long with infrastructure configuration management tools like Chef. Do you think the support that those tools provide in terms of allowing usage of different cloud providers is efficient today to enable the kind of optionality you are talking about? So from the customer point of view, from a technical point of view of managing their infrastructure as code using and being able to migrate between different providers are the tools at a sufficient level of advance for that?
I think it’s important to draw a distinction between the configuration management primitives which Chef provides, these are things like user, package, file, registry edit entry. Those are the primitives which Chef provides. Those have nothing to do with the cloud provider. Those are just an interface with the operating system upon which your infrastructure is running. Those interfaces, those resources and providers are very complete and allow any infrastructure of any complexity to be built. In terms of the interoperability between Cloud providers, what Chef does is makes use of the Fog framework to be able to make API calls to cloud providers. And here I would stick my neck out and say, if you are a cloud provider and you don’t have a REST API, you are not a cloud provider, you are just a virtualization platform with some machines running on it”. So my definition of a cloud provider is somebody close to unlimited capacity which they can give you on demand, fire an API. And if that is what you can do, you are a cloud provider. And if there is an API, Chef can talk to it. Now under certain circumstances it may not be an API which is supported by Fog, it may be an API which needs a custom knife configuration but this is not difficult, it’s not difficult to write and interface for an API which is well documented because we are really just talking HTTP over REST so it’s really very straight forward. So I think the capability of Chef to speak to any cloud goes without saying.
Well speaking specifically of Chef because that is the area that I know most, the ability to start reporting important information, so that your configuration management solution starts to become increasingly looking a little bit like a CMDB, looking like the source of authority within your infrastructure. The idea that you might be able to do significant amounts of audit trail, the idea that you might be able to mine the data for patterns, this is some of the stuff which is coming out of OpsCode’s private Chef and hosted Chef offerings. If you look at the capabilities which are emerging in Chef 11, these are all about making the user experience better, making the performance better and generally enhancing the capabilities of the tool. I don’t think that they are really related to cloud, because cloud is just a utility, is just a commodity, it’s not rocket science anymore. We are just saying here are some computers that we need to use, that is actually so uninteresting that we are able to commoditize it.
Absolutely, I think within the technical delivery aspect, the area that we are seeing as a really emerging market is the area in which traditional Windows based organizations are starting to take DevOps seriously and starting to take configuration management seriously. And this is not only traditional Windows shops thinking about technology refreshes, it’s also people thinking about moving out of their comms room and into the cloud and it’s people replacing legacy applications with new .Net MVC frameworks. And there are a large number of very interesting success stories about this. And this is an area which we are particularly excited about because it’s an area where there is not such a huge amount of specialization and is an area where there is some very exciting technology going on. So I think that is something that we’re seeing a great deal of. I would say I haven’t got the figures but I would say a significant number of the people who approach us are approaching us with Windows based infrastructures that they want to model with Chef and that they want to move into the cloud, and I think that is a really exciting place to be.
Manuel: Ok, thank you very much for your time, Stephen!