New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.
Posted by Boris Lublinsky on Feb 10, 2010
With numerous publications describing SOA implementation design and very few concentrating on the interface (contract) design, one might assume that implementation’s design is significantly more important and deserves more attention. A common justification for that is the fact that contracts change over time and spending too much time on their definition upfront contradicts the Agile development approach.
In his recent post Steve Jones disagrees with this common misconception by saying:
To my mind that view point is just like the fake-Agile people who don't document because they can't be arsed rather than because they've developed hugely high quality elements that are self-documenting. It’s basically saying that everyone has to wait until the system is operable before you can say what it does. This is the equivalent of changing the requirements to fit the implementation. Now I'm not saying that requirements don't change, and I'm not advocating waterfall, what I'm saying is that as a proportion of time allocated in an SOA programme the majority of the specification and design time should be focused on the contracts and interactions between services and the minority of time focused around the design of how those services meet those contracts.
Steve notes that there are several reasons why contracts are the most important part of SOA implementation:
- Others rely on the contracts, not the design. The cost of getting these wrong is exponential based on the number of providers. With the contracts in place and correct then people can develop independently which significantly speeds up delivery times and decreases risk
- Testing is based around the contracts not the design. The contract is the formal specification, its what the design has to meet and its this that should be used for all forms of testing
- The design can change but still stay within the contract
The notion of importance of contracts is not new. According to Dimuthu Leelarathne:
If you do not design the contracts between services first, there will be complex integration problems between your services.
In reality, creation of good services contracts is not simple and requires good understanding of the enterprise’s core business. Although there are some well established approaches to service interface design it is still more art than science. As a result, both developers and software vendors are typically focusing on what they are trained to do (and sell) - designing and coding a service implementation. In Steve’s opinion:
... IT concentrates.. far too little on ensuring the external interfaces are at least correct for a given period of time. Contracts can evolve, and I use the term deliberately, but most often older contracts will still be supported as people migrate to newer versions. This means that the contracts can have a significantly longer lifespan than the designs.. Contracts matter, designs are temporary.
As we continue to promote usage of SOA for business/IT alignment, the role of service contracts aligned with business requirements continues to grow.
After the interface contract is complete, coding a "stubbed-out" version with hard-coded results is a quick job. Then the service's clients can immediately begin testing against the contract to flush out missing functional requirements.
It must be because I'm tired ... but these debates are so .... i don't know, lacking in context? ... the contract is important ... and ensuring the design is fit for purpose and fit to be changed is also important. the contract has an impact in terms of dependencies so i do agree that is usually more important but it does depend on the specific characteristics of the problem being considered. If a particular subsystem's performance or getting quick customer feedback are going to be a critical business differentiators/success factors then they might be more important than the service contract design.
putting aside whether the subsystem/service design is fit for purpose and change, the main question is how do you get to the right contract design? there aren't any silver bullets here (although decoupling is a good place to start). if you don't understand the problem then you probably are going to get the contract wrong. how do you get to understand the problem? well, you probably need to iterate. In a large programme of work with many subcontracting companies iterating is a pain in the arse, so the tradeoff is between the cost of iterations vs time spent try to work subsystem contract/external interface designs out without understanding the problem sufficiently.
both approaches have advantages and disadvantages and which approach is appropriate is mostly about trying to understand the constraints of the situation and which risks are most significant. Also, lots of variations/combinations of iterate/upfront/option based design depending on each individual context, so I'm not that interested in debating the finer details - although might be if pushed :-). Probably the most important attributes for solving these problems are understanding the context/constraints, experience (btw, i don't mean age) and an open mind!
In terms of authors who have something interesting to say IMHO, must say that I do recommend Don Reinertsen. Also Mary Poppendieck's presentation uk lean conference presentation is worth a watch, although strangely I didn't really enjoy DonR's presentation at the same conference. I'm a big fan of Steve Vinoski. Ian Robinson and Jim Webber have interesting things to say and i still think consumer led contracts has something of value about it (reminds me of TDD for contract design). Probably other authors/practitioners that I can't think of at the moment.
Service evolution is a pain and anything the attempts to reduce the pain is usually good. Put the money where the pain is ... quite often that is in the service contract design (and maybe more importantly, the ability to evolve those designs/implementations) ... but poor quality subsystem design or subcontracting companies can hurt just as much.
Anyway, rant over.
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
2 comments
Watch Thread Reply