BT

REST and transactions?

by Mark Little on Jun 15, 2009 |

There has been a long debate about the benefits of REST versus other approaches (notably Web Services), or simply the benefits of using it for Web-based integrations. The discussions continue, but several people have begun to try to move the debate to whether or not there are certain capabilities we take for granted elsewhere that are missing in REST, or specifically in REST-over-HTTP. One of those things is distributed transactions and there has been a recent to-and-fro on the REST-discuss mailing list, triggered when Bill Burke pointed people at an implementation done on RESTeasy and asked for comments.

Pretty clean API. I want to see if Atom Links can replace some of the published URI schemes so that we can limit the number of URIs exposed by the system and give more flexibility to the system as a whole. I'm also wondering if we standardize on Link Relationships rather than data format, this may free a DTX standard from having to define a data format altogether.

This initially triggered a lively debate about whether you need ACID transactions or compensation-based transactions, with Bob Haugen (ex-Choreology and OASIS BTP) pointing out:

The basics are: 1. In the first phase, all participants update their resources provisionally (whether by state or by separate provisional resource), 2. Upon commit, all participants update their resources in their final state (or create final resources). 3. Upon abort or cancel, all participants delete their provisional resources, or mark them cancelled, or create new cancelled resources. The pattern also allows selective commits or cancels, for example for a bidding process.

There seemed to be general agreement that if you needed transactions at all then some form of extended transaction (of which compensations is a specific example) were the way to go, for the same reasons as with Web Services. Mike Amundsen added:

I prefer using the Saga model since it is an "optimistic" pattern and I find that easier to model over HTTP. On the more pragmatic side, I can model the initial interaction set w/o employing the details of the saga (implementing either 'forward compensation' or 'backward compensation' steps). I can then add the compensation work later in the implementation process (sometimes weeks or months!) without much disruption to clients or proxies, etc.

After a while Roy Fielding joined in the discussion, having said several years previously ...

This topic has come up a few times on webdav and http-wg lists. The transaction is a resource, but the relationship between it and the requested resource can be accomplished via a header field that defines each request as a sequenced resource within a hierarchical transaction. In other words, ask the server for a transaction begin, send the URI it gives the client in each request as a header field with a request number appended to it, and finally abort or commit the transaction as a final request to the transaction's URI. That's basically how I did it for the still-vapor waka protocol.

However, over the years Roy's opinions have changed, based on his experiences.

If you find yourself in need of a distributed transaction protocol, then how can you possibly say that your architecture is based on REST? I simply cannot see how you can get from one situation (of using RESTful application state on the client and hypermedia to determine all state transitions) to the next situation of needing distributed agreement of transaction semantics wherein the client has to tell the server how to manage its own resources. Most likely, the system you are thinking of is just doing CRUD operations on multiple servers. Each of those actions might be based on a RESTful architecture. When all of them are done and the client makes a final request to approve or cancel the changes, it might be interacting with a TM-style manager resource that tells all of the other services to commit the associated changes to a more persistent or public set of resources, just like a staging server might be used to prepare content prior to publication. The sum of all those actions might be equivalent to an ACID transaction. None of that matters to the REST client. As far as the client is concerned, it is only interacting with one resource at a time even when those interactions overlap asynchronously. There is no "transaction protocol" aside from whatever agreement mechanism is implemented in the back-end in accordance with the resource semantics (in a separate architecture that we don't care about here). There is no commit protocol other than the presentation of various options to the client at any given point in the application. There is no need for client-side agreement with the transaction protocol because the client is only capable of choosing from the choices provided by the server.

There was a lot of follow-up to this, with examples of where extended transaction protocols (not ACID) could and are being used in what the posters believe to be a RESTful manner. However, Bill Burke interjected that that wasn't really the point of his original post:

This was what I meant in my original post that defining and standardizing a few link relationships might simplify things a lot, both at the client resource level and resource/TM interactions.

This did seem to resonate with a several of the people on the mailing list. In fact Alexandros Marinos then pointed people at a protocol (ACID based) that he and colleagues have been working on implementing. The discussion continues, but there seems to be no clear answer to whether or not (extended) transactions really do fit in a REST world. However, it does seem conclusive that many people believe they need them for one reason or another.

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

The circle of life.. by Michael Burke

..repeats. The complexity creeps in. One size does not fit all. The revolutionary change dies a slow death through incremental accretion of cruft.

All those people being shot down by the REST school for asking "what about transactions?" must feel vindicated right about now!

Transactions are bad for REST by Arnon Rotem-Gal-Oz

I fully agree with Roy. I think distributed transactions are bad for REST just as much as they are bad for SOA and just as much as ws-atomictransactions are bad for SOAP

I blogged that in more detail here

Re: Transactions are bad for REST by Cristian Opris

REST will be hijacked by the same SOA people that think changing protocols will somehow magically solve their distributed data consistency problem.

What you need is a new way of thinking about doing distributed systems where you manage consistency and data aggregation differently, rather than how to encode 2PC in HTTP

Re: Transactions are bad for REST by Mark Little

I suggest you read the entire discussion thread on the mailing list. Most people agree that ACID transactions, e.g., WS-AT, are bad for REST let alone SOA and Web Services. That argument was debated ad nauseam many years ago, so do we really need to revisit those same arguments?

Re: Transactions are bad for REST by Arnon Rotem-Gal-Oz

You tell me :). After all this news bit, written by you and which appear on a wider forum than the rest list brings this issue up again. Efforts like RETRO (which triggered my post) also do that...

Re: Transactions are bad for REST by Mark Little

Arnon, remember this markclittle.blogspot.com/2007/07/is-anyone-out-...

Meet back here in another 2 years for the same discussion? ;)

Re: Transactions are bad for REST by Arnon Rotem-Gal-Oz

Mark, of course I do :)
I didn't say that the topic isn't discussed to death - but it does seem to come up again and again. So now there's a new distribution hype going on (REST) and the same ol' topics have to be rehashed (e.g. my next post will explain why CRUD is bad for REST). I guess we never learn :)

Re: Transactions are bad for REST by Mark Little

There's a glitch in the Matrix. It's deja vu ;-)

Transactions in SOA and REST by Guy Pardon

Hi,



We at Atomikos did not support WS-AT because we figured nobody would want it. And yet, people today are asking us to provide this... I am talking about ACID, which is of course bad for inter-company solutions but actually acceptable for intra-company solutions where interop is needed.




AFAIK interop with, say, MS is the only good reason to use web services in the first place...




When it comes to SOA there seems to be consensus that ACID is bad and something based on compensation is needed. Here is our approach to that:




www.atomikos.com/Publications/TryCancelConfirm





Also see the whitepaper that can be downloaded from our front page:



www.atomikos.com



I guess this same model could just as well be applied to REST solutions. There would be no tight coupling nor locks, and you would get the best of the loosely coupled REST world with the guarantees of transaction monitors.




Best

Guy


www.atomikos.com

"Framing the Future", a RESTful two-phase commit pattern by Diederik van Leeuwen

Hi,

We are working on a REST API for the server-side of a graphical modeling platform. Our resources - models, diagrams and elements - are hierarchical by nature and our server keeps track of revisions of these models.

The platform allows users to work on the same models simultaneously. Concurrency is realized using a git-inspired strategy; when a user starts working, he starts working on a new revision that is not yet shared with others. The unshared update a user is working on, does reside on the server, so the user can close his session and continue later wihout having to share his changes.

As a user's work evolves, the system claims the resources he touches, making sure he can effectuate his changes at a later point in time. This pessimistic approach appears to fit our customers the best. Of course claims have a time-out to avoid dead-lock and/or starvation.

To expose the functions that we support in a RESTful way, we came up with a two-phase-commit-pattern we like to refer to as the 'framing the future'-pattern: A user can frame a resource by posting a frame to it, create or update child resources by posting these to the frame rather than to the resource and have these changes take affect somewhere in the future:

We use Frames, Catches and Patches as a means to let clients transfer provisional state as if it were final state, with the guarantee that this state can be finalized later. The terminolgy should be understood as follows: A Frame represents a snapshot view on your resource, a Catch refreshes your snapshot and a Patch effectuates your changes to it. Frames present resources as if your are the only one changing them, without actually putting your changes to effect. So you don't bother others with your changes and others don't bother you with theirs... Yet. Because you can choose to catch up with patches others made by posting a Catch. And if you would choose to post a Patch, others may in turn post a Catch to catch up with your changes.

From within a Frame you see exactly the same resources as from outside a Frame, at least for the time you don't do any (provisional) updates from within that Frame and as long as nobody else effectuates any updates. When you do update resources from within your Frame, these resources only reflect your changes within that Frame; from outside your Frame these resources will show as before. One could say a Frame provides you with a view on your resources as if your changes have actually taken place and as if changes someone else made after you created your Frame have never taken place.

Only when you post a Catch to your Frame, it will reflect changes that were finalized from outside your Frame and only when you post a Patch to your Frame, your changes will in turn be effectuated. After posting a Patch, all resources will reflect your changes, both from within your Frame and from outside your Frame; only other Frames that were created before you patched won't show your changes as long as these don't catch up. From this point on you can keep on using your Frame to make new changes. These changes will be treated like before you posted your first Patch; you will need to post another Patch to finalize these new changes.

A Frame provides certainty when it comes to finalizing your changes, but only to a certain extend. A Frame comes with a time-out; when the Frame expires it won't accept any postings anymore, which means you will loose your changes after your last Patch. After each Patch, a Frame's expiration date will be reset. As long as a Frame does not expire, it will use a pessimistic claiming strategy to ensure you can finalize your changes later. As a consequence, updates you make from within a Frame, may bounce as resources may already be claimed by other users from within their Frames.

All can be illustrated using the following scenario with two users that add an element to the same diagram of a model in parallel:

// before posting a frame User 1 sees a single element in diagram 1 of model 1
// after posting a new element within his frame, the diagram shows elements 1 and 2
User 1 GET @ /models/1/diagrams/1/elements -> [element 1]
User 1 POST frame @ /models/1/frames -> frame 1
User 1 POST element @ /models/1/frames/1/diagrams/1/elements -> element 2
User 1 GET @ /models/1/frames/1/diagrams/1/elements -> [element 1, element 2]

// as the updates of User 1 are provisional, User 2 also sees only one element
// after posting a new element within his frame, the diagram shows elements 1 and 3
User 2 GET @ /models/1/diagrams/1/elements -> [element 1]
User 2 POST frame @ /models/1/frames -> frame 2
User 2 POST element @ /models/1/frames/2/diagrams/1/elements -> element 3
User 2 GET @ /models/1/frames/2/diagrams/1/elements -> [element 1, element 3]
User 2 POST patch @ /models/1/frames/2/patches -> patch 1
User 2 DELETE @ /models/1/frames/2

// even though User 2 posted a patch, User 1 still only sees elements 1 and 2
// however, after posting a catch, frame 1 does reflect the changes of User 2
User 1 GET @ /models/1/frames/1/diagrams/1/elements -> [element 1, element 2]
User 1 POST catch @ /models/1/frames/1/catches -> catch 1
User 1 GET @ /models/1/frames/1/diagrams/1/elements -> [element 1, element 2, element 3]
User 1 POST patch @ /models/1/frames/1/patches -> patch 1
User 1 DELETE @ /models/1/frames/1

// now that both users shared their updates by posting a patch to their frame
// the actual resource reflects all the updates and shows the same to all users
User 1 GET @ /models/1/diagrams/1/elements -> [element 1, element 2, element 3]
User 2 GET @ /models/1/diagrams/1/elements -> [element 1, element 2, element 3]

I’m curious what other’s think about this approach, so please share your thoughts on this pattern.

Regards,
Diederik van Leeuwen

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

10 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT