New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.
Posted by Jean-Jacques Dubray on Aug 21, 2008
Cloud Computing represents a significant paradigm shift for our industry. David Chappell note that:
One of the most important parts of that shift is the advent of cloud platforms...this kind of platform lets developers write applications that run in the cloud, or use services provided from the cloud, or both.
Even though these platforms look familiar, Dave provides a word of caution:
While it’s useful to look at on-premises platforms and cloud platforms through the same lens, the two aren’t identical. When platform functions move into the cloud, they sometimes change in significant ways.
Dave divides an application architecture into three categories:
- foundation
- infrastructure services
- application services
and argues that:
a cloud application can be built on a cloud foundation, just as an on-premises application is built on an on-premises foundation. Both kinds of applications can access infrastructure and application services provided on-premises and in the cloud. Just as on-premises platforms support today’s applications, cloud platforms provide services for the applications we’re likely to build tomorrow.
Out of all possible infrastructure services, "Data Services" are arguably the most important as information systems cannot be built without them. Data represents also a major strategic asset as the Cloud Platform that provides the most popular Data Services will likely command the largest market share.
Dave sees different types of Data Services:
remote storage in the cloud comes in different styles. For example, Amazon’s Simple Storage Service (S3) provides basic unstructured remote storage. The model it exposes to developers is straightforward: objects, which are just bunches of bytes, are stored in buckets ...
Another approach to cloud storage is to support more structured data. In Microsoft’s SQL Server Data Services (SSDS), for example, a container includes one or more entities, each of which holds some number of properties
Arnon Rotem-gal-Oz wonders though whether the "Database as a Service" is a good idea. This is a general trend in the industry with Microsoft, IBM, Amazon, LongJump or EnterpriseDB trying to provide essentially the same type of functionality. He explains:
So why is exposing the database through a web-service (RESTful or otherwise) is wrong? let me count the ways
- It circumvents the whole idea about "Services" - there's no business logic It makes for CRUD resources/services
- It is exposing internal database structure or data rather than a thought-out contract
- It encourages bypassing real services and going straight to their data
- It creates a blob service (the data source)
- ItIt encourages minuscule [half]-services (the multiple "interfaces" of said blob) that disregard few of the fallacies of distributed computing
- It is just client-server in sheep's clothing
Andrea James, reporter in Seattle, sees another pressing issue:
For a business, the electricity that flows from an outlet seems endless. And water will stream out of the tap without worry -- businesses pay only for what they use. But computing power hasn't been so seamless.
We are certainly barely scratching the surface of Cloud Platforms which will not emerge without some form of structured data management. It seems still too early to tell which Data Services and Cloud programming model will win. Would you trust your enterprise data to Data-as-a-Service? Would you prefer to access data through application services or simply CRUD your system of record?
While I take on board all the points that mean that Data Services can be seen as "client-server" re-invented, I'd like to offer another viewpoint.
Firstly, many companies are now past the point of trying to do large "get-everything-perfect-first-time" SOA engagements, and have moved to a more pragmatic, continuous improvement, iterative model of SOA. And in this model, quickly unlocking data to the rest of the enterprise is vital.
Secondly, just because you have the ability to make a Data Service from a database doesn't mean you have to expose the internal structure. You can use an ESB or Mashup server to transform, aggregate, and massage your data service into something that truly fits the business. This is a common pattern and one I see users asking for a lot.
I see Data Services as one of the killer applications for SOA - by allowing secure, reliable, managed access to data across a distributed domain you unlock huge capabilities and enable a new set of SOA applications incredibly rapidly.
This podcast offers some further discussion of the area.
Paul Fremantle, CTO, WSO2
Disclosure: WSO2 has an open source Data Services solution: wso2.org/projects/solutions/data-services/java which is currently available in our WSAS server and Mashup Server and in beta as a standalone system.
Paul, thanks for your comment, I think the question is "Can all enterprise data be exposed that way?" Clearly MDM or CDI makes total sense. Is it true for transactions (in the OLTP sense) as well? Now if you think that Data-as-a-Service is a killer app for SOA or if you if you are architecting a Cloud Platform, what is the programming model for developing solutions on a DaaS foundation? Aren't we one more time trading the opportunity to become process-centric for the convenience of being data-centric? Don't you think that DaaS will lead to IaaS (Integration as a Service). You speak about: "transform, aggregate, and massage", but is that enough to go beyond MDM and CDI? I really like this statement from Dave:"When platform functions move into the cloud, they sometimes change in significant ways." I would add, if the architecture of a Cloud Platform is about taking a "traditional solution architectures" and distributing the capabilities and aspects atomically, what are we gaining compared to EC2 for instance? Cloud computing should be more than a question of topology, this is a unique opportunity to reinvent the programming model.
I wrote a post on this topic some time ago...
Data virtualization promises to provide a single access point to manage and view all enterprise data regardless of data source or physical location. By providing information as a service, data virtualization software has the potential to greatly impact the deployment of SOA.
it.toolbox.com/blogs/the-soa-blog/soa-data-serv...
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
3 comments
Watch Thread Reply