The Costs of Versioning an API

Good to see some scientific rigour applied to this topic. I've seen all three models operating in the real world and it's surprising how many people live/struggle with "the knot". The graphs align well with my admittedly qualitative experience of the costs.

Hi! We've just released a new product that can take any API and convert it to any other API type. Solving a lot of the API problems you've described in this article.

Sign up for our beta at Clickslide!

Thank you Saul.

I would say that people often take the wrong path towards API / Service versioning because the fist instance cost a bit more if you lay the foundation for compatibility. There is also a general misunderstanding about how to achieve compatibility.

Great

While I highly praise the effort to bring some objectivity to a very contentious topic, I believe the model does not fully represent the cost.

The main parameter it is lacking is the technical debt in the case of Compatible versioning (option 3) which will grow exponentially making it very costly after a few iterations.

Guys, there is a reason people do P2P (option 2) and the reason is breaking clean and not carrying the debt along.

Could you please elaborate as to how technical debt builds up in option 3? Why would it be less than option 2? I would agree that Option 1 has zero technical debt, but on the other hand you force everyone to upgrade when they might never need the new functionality. So you force them to pre-pay the debt, even if they never need it.

The key to a healthy ecosystem around an API/Service is to enable consumers to upgrade on their own schedule, when they have time and budget, while minimizing the development and operations costs on the API side. Would you agree with that statement?

Which versioning pattern comes closest to achieving that goal?

I would say that it really depends on the system being built. To me the P2P essentially represents a kind of self-service/delivered/pre-built system where you might not choose to update because you have to pay (either technically or with real money). The thing is that not all services can be 'boxed' in a P2P way. And also every now and then the deprecation time comes around.
So i'm not saying anything clear but it's due to fact (as i understand it) that it's all about the business model and goals (with customers in mind) rather than creating a generally perfect solution.

Thanks for the article.

In option 2 no technical debt is carried along. Using a branching strategy, a branch of the release version gets created and maintained separately. There is, however, cost of additional hosting and some cost (hassle) in terms of keeping expertise in older versions - but you give time to customers to move to newer version. Netflix do this all the time on 100s or 1000s of micro-SOA services.

Also option 3 is not always possible so we need to prepare for 2. Rule of thumb, don't version if you do not have to but if you do, host both and allow time for clients to migrate.

Hi!

I do not understand the first formula. I don't say it is bad:-), just don't understand why should one pay the same update (U) and test (T) cost repeatedly?

For instance as per the formula upgrading from version 1 (S1) to version 2 (S2) would cost:

(1) V2 + (U1 + T1)

and upgrading from version 2 (S2) to version 3 (S3) would cost:

(2) V3 + (U1 + T1) + (U2 + T2)

I do not understand why should one pay the original update (U1) and test (T1) cost more than once? In other words: why does the original update / test cost when upgrading from version 2 to version 3?

Could you clarify it?!:-)

Thanks for the article
Cheers

I tried to break down the costs (update the clients of a particular service version + test the new version of the clients with the new version of the service) such that people can plug in different cost structures.

You would pay these costs repeatedly because they are for each consumer.

So, in scenario A) (the knot), you pay for building the first version of the service S1, then for V2 of the service, you'll have the cost of building V2 + the cost of updating each client and testing if they actually work with the new version. That cost U + T would not necessarily attached to any business value, unlike consuming the first version of the service.

So V2's cost to the ecosystem is ~ S1 + V2 + U1 + T1 (cost of updating all the V1 clients to work with V2).

For V3, you have now consumers which came in at V1 and consumers which were onboarded at V2. So V3's cost is ~ S1 + V2 + (U1 + T1) + V3 + (U2 + T2) + (U1 + T1) (V2 consumers where updated once, V1 consumers were updated twice).

The paper really aims at giving trends (not real numbers obviously) so that people better understand the cost implications of their versioning strategy. Overall, people should have a clear idea as to how their versioning strategy is going to impact existing consumers, over and over, for each version.

To be clear, a real-world versioning strategy will involve aspects of all three patterns:
a) force some consumers to upgrade (version no longer supported)
b) leave existing consumers alone
c) bring them along with no effort (compatible)

The question is really what's the ratio of a), b) or c) your ecosystem of consumers is ready to absorb. People should really roll up these versioning numbers with the money the entire ecosystem makes (the service/API provider charging consumers, and consumers charging their customers to use the service).

The bottom line is that if you pay no attention, the versioning costs can kill your API/Service.

I agree with your result, but formula for point-to-point approach is not fair. You calculate for every new version implementation costs of a new service (using cost of building a new version plus additional deployment cost would be more fair).

Btw. Number of consumers is critical for this analysis. It is totally ignored in your analysis (or assumed that constant number of consumers onboard for each service version - not realistic in the enterprise environment; number of consumers can be much lower than number of versions).

Anyway, i like it. Thanks.

Peter,

I agree with your comments, again, everyone's situation is different, I was just trying to point out that there could be a significant difference between each pattern and people should not make these kinds of decisions lightly.

For the point-to-point, I agree again, but I wrote in the article (it may not be clear) that this is a way to cover the additional operational costs which are not part of the formula. The cost is calculated such that eventually the whole ecosystem reaches the latest version, which may not (or never) happen. This is kind of the only way to make a fair comparison. This pattern is the hardest to evaluate because a lot of things can happen to each version that remains in production and consumers can migrate on their own, when they see fit.

I also agree that the number of consumers is critical, again, I was trying to find a way to compare the three patterns. If you assume a non constant consumer population, that makes it harder to compare.

thank you for your detailed feedback! and I am glad you found it useful.

JJ-

Peter,

I agree with your comments, again, everyone's situation is different, I was just trying to point out that there could be a significant difference between each pattern and people should not make these kinds of decisions lightly.

For the point-to-point, I agree again, but I wrote in the article (it may not be clear) that this is a way to cover the additional operational costs which are not part of the formula. The cost is calculated such that eventually the whole ecosystem reaches the latest version, which may not (or never) happen. This is kind of the only way to make a fair comparison. This pattern is the hardest to evaluate because a lot of things can happen to each version that remains in production and consumers can migrate on their own, when they see fit.

I also agree that the number of consumers is critical, again, I was trying to find a way to compare the three patterns. If you assume a non constant consumer population, that makes it harder to compare.

thank you for your detailed feedback! and I am glad you found it useful.

JJ-

Peter,

I agree with your comments, again, everyone's situation is different, I was just trying to point out that there could be a significant difference between each pattern and people should not make these kinds of decisions lightly.

For the point-to-point, I agree again, but I wrote in the article (it may not be clear) that this is a way to cover the additional operational costs which are not part of the formula. The cost is calculated such that eventually the whole ecosystem reaches the latest version, which may not (or never) happen. This is kind of the only way to make a fair comparison. This pattern is the hardest to evaluate because a lot of things can happen to each version that remains in production and consumers can migrate on their own, when they see fit.

I also agree that the number of consumers is critical, again, I was trying to find a way to compare the three patterns. If you assume a non constant consumer population, that makes it harder to compare.

thank you for your detailed feedback! and I am glad you found it useful.

JJ-

Hi Jean-Jacques,

I appreciated your article, it is a good perspective on versioning costs. I do have a question regarding testing in the compatible versioning formula.

Although with the compatible versioning approach you have fewer services to test, to me the formula doesn't seem to reflect that each of these services need to handle, and thus be tested, using a much larger variation of input data. Would you be willing to tell me more about your views on input/output handling in a compatible versioning implementation? I'm interested to see how much flexibility you put into the concept, and which tactics you would apply to keep the complexity from increasing as requirements change over time.

Thank you for the article, it was a good read.

Best regards, Espen

Espen,

thanks, actually, it is factored in as "cost of certification". You want to go away from a pure "testing" strategy to a "certification" strategy. Just like Telco networks for instance don't regression test with all possible network elements when they introduce a new one.

JJ-