Recorded at:

RESTful Approaches To Financial Systems Integration

Sections

Process & Practices,

Architecture & Design,

Enterprise Architecture

Topics

Business Process Modeling ,

Enterprise Architecture ,

Architecture ,

SOA ,

REST

Tags

QCon London 2009 ,

Business/IT Alignment ,

Architecture Evaluation ,

QCon

The next QCon is in London March 5-9, Join us!

Share |

RelatedVendorContent

Free Gartner Cloud Services Brokerage Report

Getting Started with Stratos - an Open Source Cloud Platform

18 agile and lean practices for effective software development governance

SOA All-In-One Guide: KPIs & Best Practices, ESB Report

ALM Buyer's Guide, by IBM

Related Sponsor

*Free Gartner Cloud Service Brokerage Report
*REST Security Reference Card
*Free Gartner Cloud Security Report
*Securing your APIs in the Cloud

1. Back to top

   Good presentation

   by Brian Edwards

   Nice presentation. A lot of ground covered and high energy is always appreciated.

   Reply

2. Back to top

   Any chance to get the source media (presentation & vidio) of this ?

   by Sakesun Roykiattisak

   I want to put some sub-title in, and show it to the team.

   Reply

3. Back to top

   Handling Upgrades

   by Jean-Jacques Dubray

   Kirk:
   
   glad to see people are finally producing some pragmatic content. I am obviously most focused on handling upgrades and it is clear that you have thought a lot about it. The point that you don't seem to have grasped yet is that if you use non brittle parsing techniques, and yes, XML/XSD is pretty much the only technology today that will give you that capability, you don't need as much MIME types as you seem to indicate. XML/XSD allows you to define "Forwards" or "Upwards" compatible schemes that handle upgrades nicely. People could have done that ever since the XML architecture was in place (XML/XSD/XPATH), but most vendors (who wanted to sell you their wondering routing technologies) dismissed it and most people felt "extensibility" was actually a bad thing. Kjell and I wrote this article some while ago: www.infoq.com/articles/contract-versioning-comp2
   
   The second point, and the major issue in any kind of REST versioning is that there is no precise boundary between the Entity (i.e. Resource, i.e. data) and the Business Logic that you access it with. So when you access a URL with MIME type vX what do you mean? do you mean vX of the resource? Do you mean all the access logic (getters and business operations) get upgraded at once? or do you have a single MIME type per version per URL. Ultimately the "business service layer" on top of entities exists, REST or not, HATEOS or not, and versioning this BSL in a RESTful implementation simply does not work, because REST does not acknowledge that this layer exists.
   
   About HATEOS, how someone can ever articulate that in a server-to-server integration scenario "you don't need to agree on the interaction"? If I have a Certificate of Deposit and we add a "roll-over" operation to CDs, how -again in a server-to-server integration scenario- have someone invoke the "roll-over" operation without ever knowing what it was before? When the client is a browser, with a human behind that can understand what a roll-over operation means, I can really appreciate how well HATEOAS works, but again how can anyone make any kinds of claim in a server-to-server integration scenario? UDDI claimed "discovery" was possible 10 years ago. We all saw where it ended. HATEOS for server-to-server interactions will end up in the bin.
   
   Overall, it would be helpful to stop untertaining the confusion between how REST applies to system integration and building user operated clients, as well as showing precisely how versioning works with a complex set of operations (lots of POSTs) on a resource.

   Reply

4. Back to top

   Re: Any chance to get the source media (presentation & vidio) of this ?

   by Kirk Wylie

   Hi, Sakesun,
   
   I've put in a set of links in the comments on my blog link back to this, but I'm not sure how to get the video. Truth be told, this is the first time I've seen the video (and I haven't even had time to work through the whole thing as of yet!).
   
   Kirk

   Reply

5. Back to top

   Re: Handling Upgrades

   by Kirk Wylie

   glad to see people are finally producing some pragmatic content. I am obviously most focused on handling upgrades and it is clear that you have thought a lot about it. The point that you don't seem to have grasped yet is that if you use non brittle parsing techniques, and yes, XML/XSD is pretty much the only technology today that will give you that capability, you don't need as much MIME types as you seem to indicate. XML/XSD allows you to define "Forwards" or "Upwards" compatible schemes that handle upgrades nicely. People could have done that ever since the XML architecture was in place (XML/XSD/XPATH), but most vendors (who wanted to sell you their wondering routing technologies) dismissed it and most people felt "extensibility" was actually a bad thing. Kjell and I wrote this article some while ago: www.infoq.com/articles/contract-versioning-comp2
   

   
   Definitely. If I didn't make it clear enough in the presentation, this is always preferrable, but I've seen too many "badly behaving" clients that can't handle it. Furthermore, there are always structural reasons why you might want to reorganize the entity's wire representation in a non-backwards-compatible way, and you need to support it.
   
   FOSSA started with versioned MIME types for each minor revision, until we wised up and just started using them for major versions that broke backwards compatibility, and that worked out pretty well for us.
   
   

   
   The second point, and the major issue in any kind of REST versioning is that there is no precise boundary between the Entity (i.e. Resource, i.e. data) and the Business Logic that you access it with. So when you access a URL with MIME type vX what do you mean? do you mean vX of the resource? Do you mean all the access logic (getters and business operations) get upgraded at once? or do you have a single MIME type per version per URL. Ultimately the "business service layer" on top of entities exists, REST or not, HATEOS or not, and versioning this BSL in a RESTful implementation simply does not work, because REST does not acknowledge that this layer exists.
   

   
   
   Ultimately, this is an AI-complete problem, which is I think why REST works well in completely disregarding it: you can't have an automatic solution to the problem, so why bother?
   
   In my case, I find that you have to work with the end-users of the application to figure out what the right semantics are, and HATEOAS allows you to do that: use the version to determine whether to just send the new version of the content, or to a whole different area of the application (maintained for backwards compatibility reasons).
   
   

   
   About HATEOS, how someone can ever articulate that in a server-to-server integration scenario "you don't need to agree on the interaction"? If I have a Certificate of Deposit and we add a "roll-over" operation to CDs, how -again in a server-to-server integration scenario- have someone invoke the "roll-over" operation without ever knowing what it was before? When the client is a browser, with a human behind that can understand what a roll-over operation means, I can really appreciate how well HATEOAS works, but again how can anyone make any kinds of claim in a server-to-server integration scenario? UDDI claimed "discovery" was possible 10 years ago. We all saw where it ended. HATEOS for server-to-server interactions will end up in the bin.
   

   
   
   Personally, I agree with you that the notion of discovery just doesn't work. There has to be human interaction, which is where this stuff ends up coming down: the HATEOAS factors are there for different versions of the human-known API, and to allow for things like machine renames/moves and URL changes. The generic navigation from the top-level elements will still need to be done by a human being, but there are a lot of things you can do with that with HATEOAS that simplifies your life.
   
   

   
   Overall, it would be helpful to stop untertaining the confusion between how REST applies to system integration and building user operated clients, as well as showing precisely how versioning works with a complex set of operations (lots of POSTs) on a resource.

   
   
   Yes, I agree. Couldn't cover all that in an hour though I'm afraid. :-)

   Reply

6. Back to top

   Re: Handling Upgrades

   by Jean-Jacques Dubray

   Kirk:
   
   I am glad we are in such a broad agreement. I still think that MIME types, though perfectly logical, are not the long term view of versioning in REST as it creates the need for a very large number (i.e unmanageable number) of Media types. My understanding is that you would need one per URI per (major) version since you have no way to group URIs locally: URI represent resources not a unit of business logic and what is versioned here is not the resources (which may have their own versioning mechanism of course) but the business logic invoked to access them.
   
   I was wondering if you could comment on how many media type do you see in practice and where do they apply? Have you found a way to logically group URIs that belong to a single unit of versioning?

   Reply

7. Back to top

   Re: Handling Upgrades

   by Kirk Wylie

   In general, since I've been working on systems that didn't have to cover the Whole Internet, I've just done simple things like application/xml+vN and use that for a variety of different entities, since in general I'm going to know what the entity is when I follow the link. Probably wouldn't satisfy the Roy Fielding test, but it's worked out pretty darn well for me for internal applications.
   
   Also, I did in fact address the brittle parsing problem in the original talk. I wasn't 100% sure, but after rewatching the talk, I definitely addressed it in the slides and in the talk.

   Reply

8. Back to top

   Re: Handling Upgrades

   by Jean-Jacques Dubray

   yes, I know you did, but I think it was important to emphasize that parsing helps limit the number of MIME types that you need. A lot of people I know simply dismiss extensibility because, well traditional programming models don't allow any kind of extensibility.
   
   As I understood it, I am not sure it came clear to everyone that there is a connection between versioning and brittle parsing, and MIME types are just for major versions.
   
   Again my concern is that when you have an entity that have multiple operations, the versioning approach that you are recommending does not have any "boundaries" of what is really being versioned.
   
   For instance, a certificate of deposit is going to have multiple operations:
   
   POST /customer/CDs/cd-123/offer (noun)
   POST /customer/CDs/cd-123/investment
   POST /customer/CDs/cd-123/rollover
   POST /customer/CDs/cd-123/payout
   POST /customer/CDs/cd-123/cancellation
   
   usually you would version all these at the same time, yet, each operation entity (offer, investment, rollover,...) is going to have its own MIME type and independent versioning from one noun to another.
   
   This is the problem I see with contract-less versioning. This is not a problem of how you encode your operations: if you are using SOAP actions or a noun-encoded scheme (via a uniform interface + noun).

   Reply

9. Back to top

   What a great presentation

   by Thomas Tarnow

   inspiring to see people having success using REST for non-trivial systems integration.

   Reply

10. Back to top

    Re: Handling Upgrades

    by Stephen Anderson

    The fact that you refer to "operations" concerns me. I feel vaguely pedantic restating this, but the only operations in REST are those provided for by the underlying protocol (in this case, HTTP). If you're defining other operations, you're not doing REST (Mr. Fielding came up with the term, he's allowed to define what it means). What you're doing seems more like XML-RPC.
    
    In REST you transfer Representations of Resources. The representation of the resource may be versioned. REST requires that the semantics of the resource do not change. You can expand the representation of the resource, but if a client that is not aware of the expansion will cause unwarranted side-effects when PUTing new resources, then that's a semantic change.
    
    If the semantics change, then it's not the same resource, and it should have a different URL.

    Reply

11. Back to top

    Re: Handling Upgrades

    by Jean-Jacques Dubray

    Stephen:
    
    but of course, when you have a business entity "you don't have operations". In case you have not noticed, there is a mechanical transformation between "business operations" and noun:
    
    invest -> POST investment
    purchase -> POST order
    login -> POST session
    
    ...
    
    The uniform interface IS-A protocol. If you use is as a set of operations then all you can do with it is CRUDing.
    
    Nobody really cares about building RESTful systems as defined by Roy, the only thing people care is how to use HTTP to build distributed systems.
    
    >> The representation of the resource may be versioned. REST requires that the semantics of the
    >> resource do not change. You can expand the representation of the resource, but if a client that
    >> is not aware of the expansion will cause unwarranted side-effects when PUTing new resources,
    >> then that's a semantic change.
    Again, either you are CRUDing (REST and HTTP were never invented as an application model) or you are simply using HTTP to build distributed systems acknowledging that HTTP was built with REST in mind. Trying to apply rules that were invented for a completely different purpose to a programming model is quite inefficient. Requesting that you can't PUT variants of a schema at the same URL is quite a big architectural problem considering that REST badly couples access to a resource with its (non unique) identity.
    
    At the end of the day, HTTP is just a better CORBA, it is a Distributed Object technology, not a Service Oriented technology, it offers:
    
    # A Global naming service, accessible from anywhere, including a simple browser, by anyone (even my mum can do it)
    
    
    # An unbreakable interface, CRUD does wonders, no versioning needed
    
    # Coarse grained interactions by default
    
    # Free cache for everyone
    
    # Scalable (Web Server based) Activation Model
    
    # Getters and business operations (via POST) can be added at will without impacting existing clients
    
    # Simple Query Language
    
    # Object Collections
    
    # No proprietary broker, just pure interoperability
    
    # ...
    
    HTTP is the perfect IIOP.

    Reply

12. Back to top

    Re: Handling Upgrades

    by Stephen Anderson

    Jean-Jacques:
    
    but of course, when you have a business entity "you don't have operations". In case you have not noticed, there is a mechanical transformation between "business operations" and noun:
    
    invest -> POST investment
    purchase -> POST order
    login -> POST session
    

    
    
    There is often a correspondence, yes. I had noticed that ;). If they're 1:1 mappings, why not use the nouns?
    
    

    
    The uniform interface IS-A protocol. If you use is as a set of operations then all you can do with it is CRUDing.
    

    
    
    Sorry, I didn't quite understand this. Do you mean that without defining operations, all you can do with REST over HTTP is CRUD on business entities? If so, I can't say that I agree. There is nothing to say that your resources are 1:1 representations of your business entities - in fact that should probably be discouraged. Also, REST as a style does not constrain you to GET, POST, PUT and DELETE. You can use any supported HTTP verb.
    
    

    
    Nobody really cares about building RESTful systems as defined by Roy, the only thing people care is how to use HTTP to build distributed systems.
    

    
    
    I must disagree, there's at least 2 people who care, Mr. Fielding and I ;). I would argue that, if you don't want to build RESTful systems as defined by Mr. Fielding, then not only are you not building RESTful systems (as he says, "Please try to adhere to [the rules] or choose some other buzzword for your API"), but you're also not getting the full set of advantages that REST has to offer. There's a set of problems (quite neatly described in the presentation) that REST addresses, so why not use it to solve them?
    
    

    Requesting that you can't PUT variants of a schema at the same URL is quite a big architectural problem considering that REST badly couples access to a resource with its (non unique) identity.

    
    
    I don't see why this is a problem. URLs are cheap, aren't they? I do see a problem if I have asserted that updating a resource at a certain URL has a particular behavior, and then I change that behavior without notifying existing clients, or making it possible for them locate a resource with the original behavior.
    
    By the way, I'm not saying you shouldn't PUT multiple versions of a schema to the same URL, just that you need to be very careful that you don't introduce unwanted side-effects when doing so.
    
    I don't understand the point about identity. The only identity a resource has is its URL, which should obviously be unique. However, in REST, there is no 1:1 correspondence between a resource and its business entities or storage location. I am free to define multiple "views"; these views could easily relate to versions of the schema.
    
    

    At the end of the day, HTTP is just a better CORBA, it is a Distributed Object technology.

    
    
    I'm sorry, that's not correct. HTTP relates to resources, not objects. There is a difference. To get to an equivalence with CORBA, you would need HTTP, UDDI, SOAP, in fact most of the WS-* stack. I would hesitate to describe that as better.

    Reply

13. Back to top

    Re: Handling Upgrades

    by Jean-Jacques Dubray

    There is often a correspondence, yes. I had noticed that ;). If they're 1:1 mappings, why not use the nouns?

    
    

    There is always a correspondence... so why bother telling people what they should use?

    
    
    

    The uniform interface IS-A protocol. If you use is as a set of operations then all you can do with it is CRUDing.

    
    
    

    Ah.. this is the most important point, a protocol lets you encode semantics, if you literally use the REST operations you can (literally) do only CRUD, or you are randomly and 'ad hocly' some unclear semantics, which understanding is unsharable between the resource representation consumer and the resource representation provider.

    
    
    

    With SOAP I can encode a bunch of semantics, the protocol itself is not related to a programming model: Send/Receive. Using 4 verbs in your protocol still requires that you encode semantics. No matter how you look at it.

    
    
    

    To get to an equivalence with CORBA, you would need HTTP, UDDI, SOAP, in fact most of the WS-* stack. I would hesitate to describe that as better.

    
    

    Hence REST-*, again you need to encode semantics, there is no other way around...

    
    
    

    I don't see why this is a problem. URLs are cheap, aren't they?

    
    

    The only (little) problem is people are encourage to keep URIs as identities. If I have a PO application under a different network authority from my Customer application, they will cross-link each other, so no you can't really use "cheap" URL. Coupling access and identity is one major flaw.

    Reply

14. Back to top

    Re: Any chance to get the source media (presentation & vidio) of this ?

    by Rasheed Muhammad

    I would like to have option to download the presentation.

    Reply

15. Back to top

    Re: Any chance to get the source media (presentation & vidio) of this ?

    by Kirk Wylie

    Hi, Sakesun, I already responded, and if you go to my blog posting about this presentation you'll find all the various source links.

    Reply

• Older >