Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Articles Book Excerpt and Review: Smart (Enough) Systems

Book Excerpt and Review: Smart (Enough) Systems

Smart (Enough) Systems by James Taylor and Neil Raden introduces a new IT concept: Enterprise Decision Management (EDM).
EDM is a systematic approach to automating and improving operational business decisions...To make your systems smart enough, your core problem is knowing what's the right decision to make and how to make it when required.
While business processes often become standard across industries, the authors explain that decisions and the data they are based on are one of the enterprise most strategic assets since they are virtually impossible to replicate by a competitor.
Many organizations have found a gap between gaining insights from business intelligence and taking action to exploit that insight in operational decisions
Charlie Bess, EDS fellow, notes that:
The focus of the book is moving from a data or system centric perspective to a decision centric approach.

The book claims that you can greatly enhance your company’s performance by automating decisions, specifically micro-decisions, by creating sophisticated decision services.

The benefits of an EDM approach are illustrated by many examples from different industries: Banking, Insurance, Retail, Healthcare, High Tech, Telecom, Manufacturing, Government Agencies, Publishing, Software,...

The book also contains an EDM readiness assessment section. Other sections detail a road map to develop EDM within your organization as well as the impact on IT. The authors and publishers provided InfoQ with Chapter 10 - EDM and the IT department. In this chapter, the authors establish that EDM builds on some of the most important trends in IT, such as Service Oriented Architecture, while  addressing some of IT’s most persistent problems. This chapter also discusses how decision services are deployed. The impact of EDM on software development life cycles and methodologies is also considered.

The book's wiki can be found here.

InfoQ talked with James Taylor who gave us an introduction on the architecture and benefits of decision services.

InfoQ: Business rules are at the heart of decision services. Business Rules Engines have historically integrated well with applications models. Is it still true in a Service Oriented World?

J. Taylor: In the past few years, the leading business rules engines have become true Business Rule Management Systems. A BRMS has to run as part of a standard enterprise architecture. You can no longer put constraints on consumers - it has to work the way developers and architects think it should work within a heterogeneous environment. For instance when the architecture mandates an application server, a rules-driven decision service has to behave as well as a regular piece of code would do. Leading vendors have also made great progress on performance. Today most of the performance bottlenecks are data related not rules-related. So, yes, I would say that business rules are good citizens in a connected world.

InfoQ: Do you see any challenges in consuming decision services?

J. Taylor: SOA has helped a lot. Decision services represent a growing class of services in the enterprise. Until recently, applications were monolithic. The temptation was just too big to write it all in one language. In most cases, it was only when there was no other alternatives that architects and developers replaced code with rules. As people have thought about SOA more cleanly, most good development teams are thinking in terms of reusable, manageable services. Suddenly the barrier to create and reuse decision services does not seem so high. Business rule vendors see SOA as a major driver behind BRMS adoption.

Of course, you have to write rules against a business object model and there is a need to describe these objects in your systems and somehow share some of these semantics in with your consumers. But you still have the flexibility to share as much or as little you need. A lot of decision services are XML-in & XML-out without any further interactions. XML and SOA are helping here again, because your can pass a lot of the context to the decision service seamlessly. Naturally, a lot of decision services require additional information, in that case the best case scenario is to implement a stateless invocation to fetch this information or cache it whenever possible.

The problems are by and large no longer technical you can deploy the same decision services to multiple endpoints in different technologies. Some SOA vendors, such as Oracle, even have a decision service construct in their SOA infrastructure.

We have also made some progress on the standards front. JSR 94 is available today but is a fairly low level API to execute rules. There is more work on the way at the OMG (PRR – Production Rule Representation) and W3C (RIF - Rules Interchange Format). We will also be starting to work on the second generation of these standards.

InfoQ: Moving on to Business Process Management Systems, what is the integration with BRMS today?

J. Taylor: The BPMS vendors as a group are more focused on having a way to integrate decision services in their architecture. EMC, FileNet, MetaStorm, Savvion, Webmethods,… they all have partnership with several BRMS vendors.

It’s interesting to look at when a decision service comes into play. Human workflow starts in a very ad hoc unstructured way with few opportunities to use decision services. It is only when end-to-end processes are automated that the need for this type of services becomes apparent. Integration-centric BPMS vendors hit this point sooner because integration scenarios feature a high degree of automation. At the same time they are used to running code in the middle of the process, so running a decision service similarly is a non-issue for them.

Today, customers have to deal with complex enough problems that they need both a BRMS to build decision services and a BPMS. As users become more sophisticated, we see new needs. They may need to log how the process and the decisions executed in an integrated way for instance. They may need to re-run transactions the way they executed at the time or perform simulations for potential changes. We are heading for a more sophisticated type of integration between BRMSs and BPMSs.

InfoQ: In EDM, What are the “management” capabilities offered today?

J. Taylor: There is a clear difference between Business Rule Management Systems and their older brothers the Business Rules Engines. Leading vendors are aggressively heading in this direction. A BRMS offers a repository with the capability of extracting a subset of the rules for deployment, it needs to support versioning, access controls, audit trails. The steps to assemble meaningful releases of decisions services are quite complex and this is where a BRMS can assist. Some customers commonly manage 30 to 40,000 rules. We even have customers that have 100,000 rules. This is when you need strong management capabilities.

These customers have typically hundreds of decision services deployed. Customers are just starting to think about how the pieces work together. For instance, we see some level of convergence happening between a service repository and a rules repository. We also start having a lot more metadata about rules.

InfoQ: What are the applications of BRMS that are enabled by SOA?

J. Taylor: Mobile enablement is a great example of where BRMS and SOA come together to deliver a lot of value. As soon as you think about mobile devices you have 3 problems:

  • You have huge amounts of data from RFID, GPS-enabled devices etc. This data is real-time and changes fast,
  • You want to have ability to plug in location oriented data (you know where people are and want to use it)
  • Mobile devices have a very different form factor

The ways information used to be consumed via a browser or even an IVR unit does not seem fit any more. Mobile devices demand automation if a mobile user is going to make decision or be impacted by a merchant’s decisions. Basically, you can’t have a person in the loop.

For instance, Best Buy is doing a company-wide customer centric initiative. They personalized the layout of stores based on local demographics, taking into account the kinds of people that are shopping in a particular store. The next step is to find a way, while you are in the store, to personalize and target messages to you without invading your privacy. When you walk in a store, they have already done a lot of work to predict what you are going to buy or the kinds of offers you could respond to. Their next challenge is make their store as efficient to sell as their web channel.

The goal moving forward is to make the data work for you and leverage the devices that customers use to transform the data into revenue.

InfoQ: What is the risk of implementing EDM?

J. Taylor: A couple of major ones. The first one is organizational. How do you see your staff? What you value in them? For instance, EDM takes underwriting to a new level in the insurance industry. Underwriting becomes about handling the exceptions. Underwriters should be spending more time with agents, focusing on the book of business as a whole. This is a tectonic shift in your workforce. All of a sudden a very productive employee who works with high precision may no longer be your greatest asset, compared to someone who can establish a better contact with your agents.

Similarly, the best cross-seller is no longer the one who had the best memory. Now with automation, the person that gets the best rapport with the customer will become a star performer. You need to manage that transformation. It is not just the consumers, knowledge workers dealing with decisions will be also be impacted.

The second one is that EDM changes the balance between the business and IT. Some business and some IT organizations don’t like it. If you change the dynamics everyone is in it together, you may end up upsetting both sides. The business and IT should come together and improve how a given system work, each bringing their side of the equation. This can be a challenge.

One unexpected consequence is that you can actually scare the hell out of a customer if you display too much insight. Now Netflix customers, for instance, love the recommendation engine and they rate the decisions it makes better than their own judgment. On the other hand, if a bank sees a new deposit and immediately sends you an “investment opportunity” message, we noticed that customers have a negative reaction. The optimal time is 1 to 3 days. It does not feel intrusive and you still have a chance that the customer has not thought about what to do with the money.

InfoQ: What are the trends in marketing and business models that benefit from and EDM?

J. Taylor: The “long tail” benefits strongly from using EDM. Today a company can generate more revenue by selling a few each of the many things as compared to a lot of a few things. In this model automating decision becomes critical. You simply have no other choice. The cost of doing business would be way too high if you had to rely on manual decision-making. The question becomes how do you that when you have a lot of stores, or no stores at all. How do you recreate the corner store experience that knows you and can sell you the little something you are going to like. The long tail is a sustainable business model. These companies have a huge investment in decision management.

Another pressure point is growing complexity. For instance in the Insurance industry, leading companies like Progressive have adopted EDM. They have gone from 3-4 pricing tiers to 250. The odds of pricing them competitively are much higher. However, 250 tiers cannot be managed by traditional underwriting processing, most of it has to be automated.

Another trend is outsourcing, the globalization of certain kinds of business. The question is where do I run my processes? where do I do certain activities? If you think of some of the problems that come out in terms of quality, privacy, risk management... there are huge benefits in retaining control on these critical decision steps. Many kinds of business rely on how well customers are treated … EDM lets you keep control of these decisions.

InfoQ: How is EDM impacting the implementation of a solution?

J. Taylor: One area is in data mining. Developers and architects need to stop looking at data as a fixed asset, as a set of data fields. Given the data that they have, they should look at leveraging the data, not just the structure, to predict things. Data modeling is not just about the structure of the data, the million dollar question is what can I infer and how can I influence the way the system works? Industries that do a lot of analytics do it without thinking, this is second nature.

There are also a lot of problems that can be solved without capturing data just by inferring it. If you have been running SAP for 10 years, you’ve got Data. Not mining it is a gross waste of critical company assets. How do turn this data into revenue opportunities or better customer service? Say, I have a customer on the phone, their contract is coming up for renewal in a month. What should I do? Give them an incentive? One of the key areas of innovation today is to take your customer information and match it up with publicly available information. Your data is the only asset your competitors cannot get. You can transform it into a great competitive weapon.

InfoQ: Thank you !

Rate this Article