New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.
Posted by Miko Matsumura on Aug 10, 2006
The problem of large binary attachments to SOAP has long been an issue in Web Services based SOA implementations.
Colin Adams of WebServices.org in his blog, posts a great overview of SOAP attachment technology standards and specifications. Referring to the practice of encoding binary as text and stuffing it into SOAP, he writes:
There is an obvious drawback. Performance takes a noticeable hit for larger messages. Firstly, the Base 64 adds 30% to the size of the original binary format due to the 4(characters):3(binary bytes) ratio, incurring a greater latency over the wire, and secondly there is some expensive decoding and encoding to be done at either end. Particularly the decoding and some tests indicate factors of 3-4 times slower performance. However, for the most part it is certainly an option for smaller messages and is guaranteed to have excellent interoperability.
For larger messages and applications that require speedy operation, Base 64 is not the solution.
People recognized fairly early on after the birth of SOAP that they needed a way to attach a binary file to a SOAP message. This was exactly the approach of the first “attachment” specification that came about. But it failed.
The response to this was the development of SOAP with Attachments specification, which was criticized in this blog for bringing two data models and breaking the composability of the message and the ability to work with standards such as WS-Security. Colin goes on to describe the state of the art in resolving some of these complex challenges.
SOA takes on many incarnations, including large attachment, asynchronous document styles. Attachments can include huge PDF documents, images or even executable binaries (much less common). In some use cases, email messages can be transformed through an intermediary into SOAP and back, and attachment handling can be one of those use cases. Attachments may not be a crucial issue in your SOA, but should they be neccesary, this blog entry provides a terrific review of the state of the art for this practice.
Indeed a great article! The MTOM/XOP recommendations finally brought a consensus.
Still, it remains strange to see how ebXML back in 2001 already defined SOAP with attachments with digital signatures based on XML signing! Many other B2B standards such as RosettaNet, AS2 and the BizTalk framework are also based on a MIME structure. Almost all vendors endorsed/implemented SOAP with Attachments (SwA), with one notable exception, Microsoft. Microsoft came up with the awful DIME.
SwA made sense and its disadvantages are exaggerated in my opinion: a WS mediation framework or ESB will have to manage extra context information (e.g. HTTP headers) outside the XML infoset anyway.
Anyway, the attachments story is a very nice example of the growth pains of web services.
How bad?
Read my findings in chapter 4 of my new book FastSOA at www.xquerynow.com/thebook. The writer's draft of the chapters are available for free download for the next month or two. In this chapter I show the scalability problems in using SOAP with Attachments and other encoding styles.
-Frank Cohen
www.xquerynow.com
www.pushtotest.com
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
2 comments
Watch Thread Reply