New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.
Posted by Steven Robbins on Jul 24, 2008
When the data models used to persistently store data (whether file systems or database management systems) and the data models used to write programs against the data (C++, Smalltalk, Visual Basic, Java, C#) are different, this is referred to as the "impedance mismatch" problem.While there has been much debate over the definition of and even the existence of the O/R impedance mismatch, many of the people in the cases said that they had experienced it in their projects. John Davies, Technical Director of Iona, UK, said in answer to "Do you have an impedance mismatch problem?"
This "impedance mismatch" is a serious problem in the enterprise, upwards of 25-33% of development time is being wasted squeezing objects into relational persistence, usually termed Object Relational Mapping (ORM). While the examples of such tools demonstrate the simplicity of ORM, real life is usually several orders of magnitude more complex and the whole idea breaks down. Even the best tools create an incredibly inefficient model, resulting in serious performance issues.Richard Aherns, a Director at Merrill Lynch, also agreed that the problem is real
We absolutely have this issue. In the equity derivatives business, agility and time to market are incredibly important. New products are introduced regularly, and flexible technology is required to adapt and keep up with the rapid pace of change in the industry. In our domain of order and quote management, having to maintain object-to-relational mappings across a wide variety of product and asset times is a drag on developer productivity and limits our ability to scale.Gerd Klevesaat, Software Architect at Siemens AG in Germany, also acknowledged the mismatch problem and went on to explain how the ORM tools don't necessarily help the developers. He stated that "you are forced to define a query in a special query language. It would be beneficial to use the programming language to leverage compile time checking of query statements and navigation capabilities." Gerd pointed out that there are technologies and tools that are beginning to bridge the gap, specifically "LINQ in .NET, native queries in db4o or DataSets in Groovy."
I was using ORMs that leveraged the compile time checking features of the language years ago (LLBLGEN Pro anyone?), and its been around since longer than that, and it wasn't the first to do it...
Its not new, and works quite well. The issue is mostly with architects that are trying to mix modern technology with obsolete practices and methodologies from 20 years ago. (That happens -away- too often... )
I second this interpretation. Everybody grew up studying the relational model of data analysis. The Object Oriented model that OODB propose has little academia value, so when analysist analyze, they often think in terms of tables, not objects. I experience this quite often, with my fellow collegues who work on the business analysis, when I correct them "you don't see the t_business_unit table, you have the BusinessUnit objects...".
Michele
There's something to be said though for having an ORM solution which can easily be decomposed or allows you to 'skip' the ORM piece and go straight DB when the use case is justified. Yes, there is an initial cost in adopting ORM, but chose an implementation which will not tie your hands when you need to get out.
"you are forced to define a query in a special query language. It would be beneficial to use the programming language to leverage compile time checking of query statements and navigation capabilities."
That's not a problem of O/R mapping itself, that's a problem of the existing tools. There are some tools that solve the problem, for example LINQ in .NET. It's a bit tricky to do that in Java, but it's possible. See for example the JaQu tool (part of the H2 database):
Product p = new Product();
List<Product> expensiveInStockProducts =
db.from(p).
where(p.unitsInStock).bigger(0).
and(p.unitPrice).bigger(3.0).
orderBy(p.productId).select();
See also other examples.
All good ORMs support mapping stored procedures to strongly typed methods. That pretty much lets you handle all messed up and weird scenarios right there.
Use the ORM for 90-95% of cases, stored procedures mapped to methods for the last 5%. Problem solved.
so when analysist analyze, they often think in terms of tables, not objects. I experience this quite often, with my fellow collegues who work on the business analysis, when I correct them "you don't see the t_business_unit table, you have the BusinessUnit objects...".
Michele
My experience is exactly the same. Even with 'senior' analysts. They just 'analyze' in tables and fields and cannot up the level of abstraction. The similar thing also happens on the other side (although I saw that much less often), where OO 'analysis' is done in the context of the particular OO programming language, instead of abstract model expressed by, say, UML.
One of the reasons for this is that modeling tools are not up to their tasks and tend to tie the modeler to the implementation paradigm, either relational database or OO language.
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
6 comments
Watch Thread Reply