BT

The Costs of Versioning an API

by Mark Little on Dec 01, 2013 |

Contract versioning and API/Service versioning has always been a consideration for SOA based systems. Whether because of the impact it has on composability, or client-service governance, it is still something of an art rather than a science. There are many examples of groups giving the benefit of their experiences (e.g., around REST is extremely popular). However, recently Jean-Jacques Dubray has written an article which attempts to inject some scientific objectivity into this problem domain.

I have been asked recently to create an estimate of the costs of versioning APIs (or Web Services). I wanted to share this estimate because I feel a lot of people still don't understand the cost implications of API/Service versioning.

According to JJ, during the work they found that the cost of building APIs was dependent upon the approach used subsequently to version them.

The key point that [you need] to understand is that even if the cost to your consumers may look small to you, it is not just a pure cost, it is risks, disrupted project plans, unavailable budgets... with changes that often have no immediate business value to an existing consumer who was not expecting any change to API.

The article then goes on to classify three different approaches to API versioning (see the full article for a more in depth discussion of each, including how JJ defines a way to measure cost):

  • The Knot. "All API consumers are tied to a single version of the API, when that API changes, all consumers have to change, in essence creating a massive ripple effect across the entire set of consumers / ecosystem."

  • Point-to-Point. "Every service version is left running in production and consumers are required to migrate on their own, when they need to. The maintenance costs increase as the number of version in production increases."

  • Compatible Versioning. "All clients talk to the same compatible API/Service version."

Given these definitions and associated costs computed using the equations JJ describes, it is possible to plot the relative costs as shown below (y axis is cost, x axis is the version number):

 

As JJ says:

[...] a single version forcing every consumer to upgrade when the API changes is the most expensive to the ecosystem. A multiplicity of versions that need to be maintained is better, but still quite costly when you try to keep upgrading each version or alternatively operating older versions. A compatible versioning strategy seem to offer the best efficiency.

So what do others think? Is this way of calculating the cost of versioning APIs applicable beyond the environments in which it was developed by JJ and team? Does the relative cost explanation make sense given your own experiences? Are there other categories which JJ and team don't cover?

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Very nice analysis by Saul Caganoff

Good to see some scientific rigour applied to this topic. I've seen all three models operating in the real world and it's surprising how many people live/struggle with "the knot". The graphs align well with my admittedly qualitative experience of the costs.

We've solved versioning and the fragmentation of APIs by Gabriel Ortiz

Hi! We've just released a new product that can take any API and convert it to any other API type. Solving a lot of the API problems you've described in this article.

Sign up for our beta at Clickslide!

Re: Very nice analysis by Jean-Jacques Dubray

Thank you Saul.

I would say that people often take the wrong path towards API / Service versioning because the fist instance cost a bit more if you lay the foundation for compatibility. There is also a general misunderstanding about how to achieve compatibility.

Great by 胡 毅斌

Great

Not impressed by A Kheyrollahi

While I highly praise the effort to bring some objectivity to a very contentious topic, I believe the model does not fully represent the cost.

The main parameter it is lacking is the technical debt in the case of Compatible versioning (option 3) which will grow exponentially making it very costly after a few iterations.

Guys, there is a reason people do P2P (option 2) and the reason is breaking clean and not carrying the debt along.

Re: Not impressed by Jean-Jacques Dubray

Could you please elaborate as to how technical debt builds up in option 3? Why would it be less than option 2? I would agree that Option 1 has zero technical debt, but on the other hand you force everyone to upgrade when they might never need the new functionality. So you force them to pre-pay the debt, even if they never need it.

The key to a healthy ecosystem around an API/Service is to enable consumers to upgrade on their own schedule, when they have time and budget, while minimizing the development and operations costs on the API side. Would you agree with that statement?

Which versioning pattern comes closest to achieving that goal?

Re: Not impressed by Daniel Zahariev

I would say that it really depends on the system being built. To me the P2P essentially represents a kind of self-service/delivered/pre-built system where you might not choose to update because you have to pay (either technically or with real money). The thing is that not all services can be 'boxed' in a P2P way. And also every now and then the deprecation time comes around.
So i'm not saying anything clear but it's due to fact (as i understand it) that it's all about the business model and goals (with customers in mind) rather than creating a generally perfect solution.

Thanks for the article.

Re: Not impressed by A Kheyrollahi

In option 2 no technical debt is carried along. Using a branching strategy, a branch of the release version gets created and maintained separately. There is, however, cost of additional hosting and some cost (hassle) in terms of keeping expertise in older versions - but you give time to customers to move to newer version. Netflix do this all the time on 100s or 1000s of micro-SOA services.

Also option 3 is not always possible so we need to prepare for 2. Rule of thumb, don't version if you do not have to but if you do, host both and allow time for clients to migrate.

A question regarding the first formula by Gyula Csom

Hi!

I do not understand the first formula. I don't say it is bad:-), just don't understand why should one pay the same update (U) and test (T) cost repeatedly?

For instance as per the formula upgrading from version 1 (S1) to version 2 (S2) would cost:

(1) V2 + (U1 + T1)

and upgrading from version 2 (S2) to version 3 (S3) would cost:

(2) V3 + (U1 + T1) + (U2 + T2)

I do not understand why should one pay the original update (U1) and test (T1) cost more than once? In other words: why does the original update / test cost when upgrading from version 2 to version 3?

Could you clarify it?!:-)

Thanks for the article
Cheers

Re: A question regarding the first formula by Jean-Jacques Dubray

I tried to break down the costs (update the clients of a particular service version + test the new version of the clients with the new version of the service) such that people can plug in different cost structures.

You would pay these costs repeatedly because they are for each consumer.

So, in scenario A) (the knot), you pay for building the first version of the service S1, then for V2 of the service, you'll have the cost of building V2 + the cost of updating each client and testing if they actually work with the new version. That cost U + T would not necessarily attached to any business value, unlike consuming the first version of the service.

So V2's cost to the ecosystem is ~ S1 + V2 + U1 + T1 (cost of updating all the V1 clients to work with V2).

For V3, you have now consumers which came in at V1 and consumers which were onboarded at V2. So V3's cost is ~ S1 + V2 + (U1 + T1) + V3 + (U2 + T2) + (U1 + T1) (V2 consumers where updated once, V1 consumers were updated twice).

The paper really aims at giving trends (not real numbers obviously) so that people better understand the cost implications of their versioning strategy. Overall, people should have a clear idea as to how their versioning strategy is going to impact existing consumers, over and over, for each version.

To be clear, a real-world versioning strategy will involve aspects of all three patterns:
a) force some consumers to upgrade (version no longer supported)
b) leave existing consumers alone
c) bring them along with no effort (compatible)

The question is really what's the ratio of a), b) or c) your ecosystem of consumers is ready to absorb. People should really roll up these versioning numbers with the money the entire ecosystem makes (the service/API provider charging consumers, and consumers charging their customers to use the service).

The bottom line is that if you pay no attention, the versioning costs can kill your API/Service.

Re: A question regarding the first formula by Peter Rajsky

I agree with your result, but formula for point-to-point approach is not fair. You calculate for every new version implementation costs of a new service (using cost of building a new version plus additional deployment cost would be more fair).

Btw. Number of consumers is critical for this analysis. It is totally ignored in your analysis (or assumed that constant number of consumers onboard for each service version - not realistic in the enterprise environment; number of consumers can be much lower than number of versions).

Anyway, i like it. Thanks.

Re: A question regarding the first formula by Jean-Jacques Dubray

Peter,

I agree with your comments, again, everyone's situation is different, I was just trying to point out that there could be a significant difference between each pattern and people should not make these kinds of decisions lightly.

For the point-to-point, I agree again, but I wrote in the article (it may not be clear) that this is a way to cover the additional operational costs which are not part of the formula. The cost is calculated such that eventually the whole ecosystem reaches the latest version, which may not (or never) happen. This is kind of the only way to make a fair comparison. This pattern is the hardest to evaluate because a lot of things can happen to each version that remains in production and consumers can migrate on their own, when they see fit.

I also agree that the number of consumers is critical, again, I was trying to find a way to compare the three patterns. If you assume a non constant consumer population, that makes it harder to compare.

thank you for your detailed feedback! and I am glad you found it useful.

JJ-

Re: A question regarding the first formula by Jean-Jacques Dubray

Peter,

I agree with your comments, again, everyone's situation is different, I was just trying to point out that there could be a significant difference between each pattern and people should not make these kinds of decisions lightly.

For the point-to-point, I agree again, but I wrote in the article (it may not be clear) that this is a way to cover the additional operational costs which are not part of the formula. The cost is calculated such that eventually the whole ecosystem reaches the latest version, which may not (or never) happen. This is kind of the only way to make a fair comparison. This pattern is the hardest to evaluate because a lot of things can happen to each version that remains in production and consumers can migrate on their own, when they see fit.

I also agree that the number of consumers is critical, again, I was trying to find a way to compare the three patterns. If you assume a non constant consumer population, that makes it harder to compare.

thank you for your detailed feedback! and I am glad you found it useful.

JJ-

Re: A question regarding the first formula by Jean-Jacques Dubray

Peter,

I agree with your comments, again, everyone's situation is different, I was just trying to point out that there could be a significant difference between each pattern and people should not make these kinds of decisions lightly.

For the point-to-point, I agree again, but I wrote in the article (it may not be clear) that this is a way to cover the additional operational costs which are not part of the formula. The cost is calculated such that eventually the whole ecosystem reaches the latest version, which may not (or never) happen. This is kind of the only way to make a fair comparison. This pattern is the hardest to evaluate because a lot of things can happen to each version that remains in production and consumers can migrate on their own, when they see fit.

I also agree that the number of consumers is critical, again, I was trying to find a way to compare the three patterns. If you assume a non constant consumer population, that makes it harder to compare.

thank you for your detailed feedback! and I am glad you found it useful.

JJ-

A question regarding test costs by Espen Grønli

Hi Jean-Jacques,

I appreciated your article, it is a good perspective on versioning costs. I do have a question regarding testing in the compatible versioning formula.

Although with the compatible versioning approach you have fewer services to test, to me the formula doesn't seem to reflect that each of these services need to handle, and thus be tested, using a much larger variation of input data. Would you be willing to tell me more about your views on input/output handling in a compatible versioning implementation? I'm interested to see how much flexibility you put into the concept, and which tactics you would apply to keep the complexity from increasing as requirements change over time.

Thank you for the article, it was a good read.

Best regards, Espen

Re: A question regarding test costs by Jean-Jacques Dubray

Espen,

thanks, actually, it is factored in as "cost of certification". You want to go away from a pure "testing" strategy to a "certification" strategy. Just like Telco networks for instance don't regression test with all possible network elements when they introduce a new one.

JJ-

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

16 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT