New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.

Posted by Rathina Dhandapani on Sep 29, 2009
Service Oriented Architecture (SOA) has been widely accepted as an approach that facilitates business agility by aligning IT with business. The prime differentiator with this approach is the ease with which such an agility is quickly achieved at a relatively lower cost. At a high level, this approach attempts to drive down the incremental cost of addressing the 'n'th business change to zero or closer to zero. Organizations pursue SOA initiatives in order to achieve this elusive 'n'th iteration as early as possible in their SOA journey. In practice, it may take years, if not longer to achieve such an optimized state. This article is intended for business analysts and enterprise/system architects involved in a SOA initiative.
Service identification is a crucial first step in the long journey towards SOA end state. It's an iterative step that needs to be organized and initiated early on during analysis and planning. It determines the overall service landscape with services being identified, named, categorized, prioritized and even associated with appropriate roadmap phase for implementation. Service identification phase lays the foundation for a service based ecosystem and this article highlights best practices associated with business service identification.
Based on the context in which SOA initiatives are taken, they can be classified under categories below:
Figure 1: Initial business agility and functional maturity of various categories are shown here. It also shows the preferred state to be achieved through SOA.
Figure 2: Relative ROI over time for four hypothetical but related changes is shown here. An enterprise under category SOA transition (i) leverages functional maturity to reap quick benefits early on but has relatively flat returns over time. On the other hand, an enterprise under category SOA embarkment (iii) has steep returns in the long run leveraging predictable and shorter time-to-market potential and less integration costs.
Identifying the category would help set the focus and expectations. It also helps in narrowing down the service identification approaches.
There is an elevated need to show tangible and quick Return on Investment (ROI) against the backdrop of a gloomy global economy. Business and IT teams should work closer and leverage each other's capability to swiftly support drastic and unconventional business moves in an increasingly competitive environment. Need for service identification cannot be overemphasized and this is largely a business driven, IT guided step.
Inputs from strategy, vision and long term business goals/objectives are critical to establish a strong base that accommodates changes. Such inputs are accounted for in SOA roadmaps and associated phases. It impacts service definition by injecting a level of abstraction future-proofing service landscape and extending service contract's useful life (e.g. A financial enterprise specializing in credit based products may have the intention to venture in to insurance or brokerage and this may add few product agnostic services to the landscape).
These are predominant processes that form the bulk of the process landscape comprising of core and noncore functionality. Such processes can usually be represented at various abstraction levels referred to as process levels in a process model. Services can then be extracted from multiple such levels with a top-down approach. Higher abstraction levels provide inputs for composite services while lower levels provide inputs for fine grained candidates. Such a focus on processes and service candidates would help identify functional redundancy across enterprise. Regardless of the nature of the SOA initiative (highlighted above), this practice is vital.
SOA planning and governance tools may streamline and expedite service identification and communication process. They can even visually depict service relationships. Service repository related tools can be useful to identify impact of a proposed service change.
Over the years, there has been significant progress in SOA space within industry verticals like telecom, utilities etc spearheaded by relevant groups (IFX, SWIFT, OAGi, Accord, ETIS, ISO etc). Few cross-industry artifacts too are available. Such standardized service contracts (interfaces and operations), data models and third party service lists can be reviewed and leveraged. It could jump-start the move towards SOA and for the most part the artifacts (XML based) are likely to be extensible and backward compatible.
However, such artifacts must undergo context specific scrutiny before adoption. Examples of questions that enterprises might need to ask when adopting these standards are: Should the data model be adopted in part or whole? Is it necessary to provide native support to the canonical model or would it suffice to limit it to external integration touch points? Is there any hidden cost? Does it command broader vendor support?
A service contract should highlight both functional and non-functional capabilities. A baseline needs to be established by identifying relevant attributes that are part of a contract. Functional attributes may include details like service description, message structure and data model. Non-functional attributes may include details like Quality of Service (e.g. Response time, Availability), Cost base (Count or Period) and Security. Such a baseline standardizes the content of WSDL files, XML Schemas and WS-Policy definitions (Metadata for enforcing behavioral constraints) and ensures contract consistency across the enterprise.
It's quite natural to just extend processes, business goals etc to identify a huge list of candidate services with corresponding contracts. The services can be just as good as their sources in aiding agility and may lead to service proliferation. A matrix with service attributes (e.g. reusability, composability, abstraction, competitive differentiation etc) and their relative weighted average can be used to screen, rate and refine the candidate list. This rating should not be less than the predetermined target rating based on category, roadmap phase etc to make the cut. Granularity can be modified and contract can be repetitively changed to meet matrix requirements.
This can be a very handy step to evaluate the agility of a system well before implementation. Business scenarios and use cases from a subsequent roadmap phase or typical business scenarios not catered to in current phase would need to be compiled. Service inventory with contracts would then be used to address these scenarios through simulation, while assessing the impact on the system. Key metrics for assessment can include:
i) Service reuse ratio (Number of services reused / Total number of services in scenario)
ii) Service leverage ratio (Number of services reused / Total number of services in inventory)
iii) Service revision ratio (Number of services revised / Number of services reused)
iv) Service creation ratio (Number of new services created/Total number of services in scenario) and
v) Service utilization ratio (For a given service, Number of service consumers identified / Total number of services in scenario )
These IT metrics can be normalized for complexity and analyzed together to assess impact on time-to market characteristics. This may result in further restructuring impacting service inventory. An enterprise that has gone past initial roadmap phases can include historical data to generate business and financial metrics that are even better during BASS.
The hallmark of a service based system is to accept both planned and unplanned changes. The service inventory is a key enabler and accommodating an iterative service identification process that would fit hand in glove with SOA governance processes is critical. The iteration may align with roadmap phases, continuous improvement initiatives or result from internal/external triggers (e.g. Technical advancements that even out service overhead may allow for additional fine grained services). Such changes may result in new services, service revisions and service retirements.
Composite applications assembled from an inventory of services enable business agility. Service identification yields this list of business and technical services. It's relatively easy to identify a set of services, however the ROI would be governed by the nature of SOA initiative, frequency of changes and the magnitude of changes. This article highlights key best practices to identify, validate and verify service inventory content well before implementation.
Hello
Note on Point 9-III (Number of services revised / Number of services reused) :
If an update is made on a Service and this change is not used by all the consumer of the service, I don’t think it should be added in this statistic.
Example: 1 Service used by 5 consumers, one of the consumer need a new tag or logic. These types of changes should not be added. Actually, the overall statistic should go down because you might have to do some regression testing for all other consumers.
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
1 comment
Watch Thread Reply