RESTful Approaches To Financial Systems Integration
Recorded at:
- Share
-
- |
Read later
My Reading List
Bio
Kirk Wylie worked for Broadbase, BEA, Radik, and M7 before relocating to London and developing software solutions for the financial industry. At SmartSpread he developed a bespoke risk and analytics platform for the Vega Asset Management family of hedge funds. At KBC Financial Products he built a CDO system and developed the RESTful techniques he applied to later projects.
QCon is a conference that is organized by the community, for the community.The result is a high quality conference experience where a tremendous amount of attention and investment has gone into having the best content on the most important topics presented by the leaders in our community. QCon is designed with the technical depth and enterprise focus of interest to technical team leads, architects, and project managers.
Good presentation
by
Brian Edwards
Any chance to get the source media (presentation & vidio) of this ?
by
Sake .
Handling Upgrades
by
Jean-Jacques Dubray
glad to see people are finally producing some pragmatic content. I am obviously most focused on handling upgrades and it is clear that you have thought a lot about it. The point that you don't seem to have grasped yet is that if you use non brittle parsing techniques, and yes, XML/XSD is pretty much the only technology today that will give you that capability, you don't need as much MIME types as you seem to indicate. XML/XSD allows you to define "Forwards" or "Upwards" compatible schemes that handle upgrades nicely. People could have done that ever since the XML architecture was in place (XML/XSD/XPATH), but most vendors (who wanted to sell you their wondering routing technologies) dismissed it and most people felt "extensibility" was actually a bad thing. Kjell and I wrote this article some while ago: www.infoq.com/articles/contract-versioning-comp2
The second point, and the major issue in any kind of REST versioning is that there is no precise boundary between the Entity (i.e. Resource, i.e. data) and the Business Logic that you access it with. So when you access a URL with MIME type vX what do you mean? do you mean vX of the resource? Do you mean all the access logic (getters and business operations) get upgraded at once? or do you have a single MIME type per version per URL. Ultimately the "business service layer" on top of entities exists, REST or not, HATEOS or not, and versioning this BSL in a RESTful implementation simply does not work, because REST does not acknowledge that this layer exists.
About HATEOS, how someone can ever articulate that in a server-to-server integration scenario "you don't need to agree on the interaction"? If I have a Certificate of Deposit and we add a "roll-over" operation to CDs, how -again in a server-to-server integration scenario- have someone invoke the "roll-over" operation without ever knowing what it was before? When the client is a browser, with a human behind that can understand what a roll-over operation means, I can really appreciate how well HATEOAS works, but again how can anyone make any kinds of claim in a server-to-server integration scenario? UDDI claimed "discovery" was possible 10 years ago. We all saw where it ended. HATEOS for server-to-server interactions will end up in the bin.
Overall, it would be helpful to stop untertaining the confusion between how REST applies to system integration and building user operated clients, as well as showing precisely how versioning works with a complex set of operations (lots of POSTs) on a resource.
Re: Any chance to get the source media (presentation & vidio) of this ?
by
Kirk Wylie
I've put in a set of links in the comments on my blog link back to this, but I'm not sure how to get the video. Truth be told, this is the first time I've seen the video (and I haven't even had time to work through the whole thing as of yet!).
Kirk
Re: Handling Upgrades
by
Kirk Wylie
glad to see people are finally producing some pragmatic content. I am obviously most focused on handling upgrades and it is clear that you have thought a lot about it. The point that you don't seem to have grasped yet is that if you use non brittle parsing techniques, and yes, XML/XSD is pretty much the only technology today that will give you that capability, you don't need as much MIME types as you seem to indicate. XML/XSD allows you to define "Forwards" or "Upwards" compatible schemes that handle upgrades nicely. People could have done that ever since the XML architecture was in place (XML/XSD/XPATH), but most vendors (who wanted to sell you their wondering routing technologies) dismissed it and most people felt "extensibility" was actually a bad thing. Kjell and I wrote this article some while ago: www.infoq.com/articles/contract-versioning-comp2
Definitely. If I didn't make it clear enough in the presentation, this is always preferrable, but I've seen too many "badly behaving" clients that can't handle it. Furthermore, there are always structural reasons why you might want to reorganize the entity's wire representation in a non-backwards-compatible way, and you need to support it.
FOSSA started with versioned MIME types for each minor revision, until we wised up and just started using them for major versions that broke backwards compatibility, and that worked out pretty well for us.
The second point, and the major issue in any kind of REST versioning is that there is no precise boundary between the Entity (i.e. Resource, i.e. data) and the Business Logic that you access it with. So when you access a URL with MIME type vX what do you mean? do you mean vX of the resource? Do you mean all the access logic (getters and business operations) get upgraded at once? or do you have a single MIME type per version per URL. Ultimately the "business service layer" on top of entities exists, REST or not, HATEOS or not, and versioning this BSL in a RESTful implementation simply does not work, because REST does not acknowledge that this layer exists.
Ultimately, this is an AI-complete problem, which is I think why REST works well in completely disregarding it: you can't have an automatic solution to the problem, so why bother?
In my case, I find that you have to work with the end-users of the application to figure out what the right semantics are, and HATEOAS allows you to do that: use the version to determine whether to just send the new version of the content, or to a whole different area of the application (maintained for backwards compatibility reasons).
About HATEOS, how someone can ever articulate that in a server-to-server integration scenario "you don't need to agree on the interaction"? If I have a Certificate of Deposit and we add a "roll-over" operation to CDs, how -again in a server-to-server integration scenario- have someone invoke the "roll-over" operation without ever knowing what it was before? When the client is a browser, with a human behind that can understand what a roll-over operation means, I can really appreciate how well HATEOAS works, but again how can anyone make any kinds of claim in a server-to-server integration scenario? UDDI claimed "discovery" was possible 10 years ago. We all saw where it ended. HATEOS for server-to-server interactions will end up in the bin.
Personally, I agree with you that the notion of discovery just doesn't work. There has to be human interaction, which is where this stuff ends up coming down: the HATEOAS factors are there for different versions of the human-known API, and to allow for things like machine renames/moves and URL changes. The generic navigation from the top-level elements will still need to be done by a human being, but there are a lot of things you can do with that with HATEOAS that simplifies your life.
Overall, it would be helpful to stop untertaining the confusion between how REST applies to system integration and building user operated clients, as well as showing precisely how versioning works with a complex set of operations (lots of POSTs) on a resource.
Yes, I agree. Couldn't cover all that in an hour though I'm afraid. :-)
Re: Handling Upgrades
by
Jean-Jacques Dubray
I am glad we are in such a broad agreement. I still think that MIME types, though perfectly logical, are not the long term view of versioning in REST as it creates the need for a very large number (i.e unmanageable number) of Media types. My understanding is that you would need one per URI per (major) version since you have no way to group URIs locally: URI represent resources not a unit of business logic and what is versioned here is not the resources (which may have their own versioning mechanism of course) but the business logic invoked to access them.
I was wondering if you could comment on how many media type do you see in practice and where do they apply? Have you found a way to logically group URIs that belong to a single unit of versioning?
Re: Handling Upgrades
by
Kirk Wylie
Also, I did in fact address the brittle parsing problem in the original talk. I wasn't 100% sure, but after rewatching the talk, I definitely addressed it in the slides and in the talk.
Re: Handling Upgrades
by
Jean-Jacques Dubray
As I understood it, I am not sure it came clear to everyone that there is a connection between versioning and brittle parsing, and MIME types are just for major versions.
Again my concern is that when you have an entity that have multiple operations, the versioning approach that you are recommending does not have any "boundaries" of what is really being versioned.
For instance, a certificate of deposit is going to have multiple operations:
POST /customer/CDs/cd-123/offer (noun)
POST /customer/CDs/cd-123/investment
POST /customer/CDs/cd-123/rollover
POST /customer/CDs/cd-123/payout
POST /customer/CDs/cd-123/cancellation
usually you would version all these at the same time, yet, each operation entity (offer, investment, rollover,...) is going to have its own MIME type and independent versioning from one noun to another.
This is the problem I see with contract-less versioning. This is not a problem of how you encode your operations: if you are using SOAP actions or a noun-encoded scheme (via a uniform interface + noun).
What a great presentation
by
Thomas Tarnow
Re: Handling Upgrades
by
Stephen Anderson
In REST you transfer Representations of Resources. The representation of the resource may be versioned. REST requires that the semantics of the resource do not change. You can expand the representation of the resource, but if a client that is not aware of the expansion will cause unwarranted side-effects when PUTing new resources, then that's a semantic change.
If the semantics change, then it's not the same resource, and it should have a different URL.
Re: Handling Upgrades
by
Jean-Jacques Dubray
but of course, when you have a business entity "you don't have operations". In case you have not noticed, there is a mechanical transformation between "business operations" and noun:
invest -> POST investment
purchase -> POST order
login -> POST session
...
The uniform interface IS-A protocol. If you use is as a set of operations then all you can do with it is CRUDing.
Nobody really cares about building RESTful systems as defined by Roy, the only thing people care is how to use HTTP to build distributed systems.
>> The representation of the resource may be versioned. REST requires that the semantics of the
>> resource do not change. You can expand the representation of the resource, but if a client that
>> is not aware of the expansion will cause unwarranted side-effects when PUTing new resources,
>> then that's a semantic change.
Again, either you are CRUDing (REST and HTTP were never invented as an application model) or you are simply using HTTP to build distributed systems acknowledging that HTTP was built with REST in mind. Trying to apply rules that were invented for a completely different purpose to a programming model is quite inefficient. Requesting that you can't PUT variants of a schema at the same URL is quite a big architectural problem considering that REST badly couples access to a resource with its (non unique) identity.
At the end of the day, HTTP is just a better CORBA, it is a Distributed Object technology, not a Service Oriented technology, it offers:
# A Global naming service, accessible from anywhere, including a simple browser, by anyone (even my mum can do it)
# An unbreakable interface, CRUD does wonders, no versioning needed
# Coarse grained interactions by default
# Free cache for everyone
# Scalable (Web Server based) Activation Model
# Getters and business operations (via POST) can be added at will without impacting existing clients
# Simple Query Language
# Object Collections
# No proprietary broker, just pure interoperability
# ...
HTTP is the perfect IIOP.
Re: Handling Upgrades
by
Stephen Anderson
Jean-Jacques:
but of course, when you have a business entity "you don't have operations". In case you have not noticed, there is a mechanical transformation between "business operations" and noun:
invest -> POST investment
purchase -> POST order
login -> POST session
There is often a correspondence, yes. I had noticed that ;). If they're 1:1 mappings, why not use the nouns?
The uniform interface IS-A protocol. If you use is as a set of operations then all you can do with it is CRUDing.
Sorry, I didn't quite understand this. Do you mean that without defining operations, all you can do with REST over HTTP is CRUD on business entities? If so, I can't say that I agree. There is nothing to say that your resources are 1:1 representations of your business entities - in fact that should probably be discouraged. Also, REST as a style does not constrain you to GET, POST, PUT and DELETE. You can use any supported HTTP verb.
Nobody really cares about building RESTful systems as defined by Roy, the only thing people care is how to use HTTP to build distributed systems.
I must disagree, there's at least 2 people who care, Mr. Fielding and I ;). I would argue that, if you don't want to build RESTful systems as defined by Mr. Fielding, then not only are you not building RESTful systems (as he says, "Please try to adhere to [the rules] or choose some other buzzword for your API"), but you're also not getting the full set of advantages that REST has to offer. There's a set of problems (quite neatly described in the presentation) that REST addresses, so why not use it to solve them?
Requesting that you can't PUT variants of a schema at the same URL is quite a big architectural problem considering that REST badly couples access to a resource with its (non unique) identity.
I don't see why this is a problem. URLs are cheap, aren't they? I do see a problem if I have asserted that updating a resource at a certain URL has a particular behavior, and then I change that behavior without notifying existing clients, or making it possible for them locate a resource with the original behavior.
By the way, I'm not saying you shouldn't PUT multiple versions of a schema to the same URL, just that you need to be very careful that you don't introduce unwanted side-effects when doing so.
I don't understand the point about identity. The only identity a resource has is its URL, which should obviously be unique. However, in REST, there is no 1:1 correspondence between a resource and its business entities or storage location. I am free to define multiple "views"; these views could easily relate to versions of the schema.
At the end of the day, HTTP is just a better CORBA, it is a Distributed Object technology.
I'm sorry, that's not correct. HTTP relates to resources, not objects. There is a difference. To get to an equivalence with CORBA, you would need HTTP, UDDI, SOAP, in fact most of the WS-* stack. I would hesitate to describe that as better.
Re: Handling Upgrades
by
Jean-Jacques Dubray
There is often a correspondence, yes. I had noticed that ;). If they're 1:1 mappings, why not use the nouns?
There is always a correspondence... so why bother telling people what they should use?
The uniform interface IS-A protocol. If you use is as a set of operations then all you can do with it is CRUDing.
Ah.. this is the most important point, a protocol lets you encode semantics, if you literally use the REST operations you can (literally) do only CRUD, or you are randomly and 'ad hocly' some unclear semantics, which understanding is unsharable between the resource representation consumer and the resource representation provider.
With SOAP I can encode a bunch of semantics, the protocol itself is not related to a programming model: Send/Receive. Using 4 verbs in your protocol still requires that you encode semantics. No matter how you look at it.
To get to an equivalence with CORBA, you would need HTTP, UDDI, SOAP, in fact most of the WS-* stack. I would hesitate to describe that as better.
Hence REST-*, again you need to encode semantics, there is no other way around...
I don't see why this is a problem. URLs are cheap, aren't they?
The only (little) problem is people are encourage to keep URIs as identities. If I have a PO application under a different network authority from my Customer application, they will cross-link each other, so no you can't really use "cheap" URL. Coupling access and identity is one major flaw.
Re: Any chance to get the source media (presentation & vidio) of this ?
by
Rasheed Muhammad
Re: Any chance to get the source media (presentation & vidio) of this ?
by
Kirk Wylie
Hello stranger!
You need to Register an InfoQ account or Login to post comments. But there's so much more behind being registered.Get the most out of the InfoQ experience.
Tell us what you think