Agile Business Rules
The differences between "Agile software development" and "Business Agility" often cause confusion. Sometimes an organization says it needs business agility as a way of giving itself permission to use Microsoft Excel to solve a problem, to use end-user programming tools or otherwise avoid its IT organization. Sometimes business agility is used as a synonym for adopting a Business Process Management System (BPMS) or using a diagramming notation like BPEL. Sometimes there is an unspoken assumption that adopting SOA will result in agility. For even the most complex systems, however, agile software development can deliver business agility - this is especially true when the practice combined with the right development technology.
Let's start with Gartner's definition of Agility (in the sense of Business Agility):
"...the ability of an organization to sense environmental change and to respond efficiently and effectively to that change."1
Gartner includes the following as the five basic steps2 to delivering business agility:
If you are in the middle of a project that is using an agile approach, you are going to see Sensing happen in both retrospectives (where the team senses changes that are required) and outside the project (where the business users sense a change needed based on performance management or other monitoring). The Strategize-Decide-Communicate-Act sequence will likely repeat within an iteration regardless of how the need for change was sensed.. If you are "done" with a system, then the result of Strategizing might well be to start an agile project to deliver what is needed (I would collapse Decide-Communicate-Act into the project myself). In this case Sensing is going to have some either from an end-of-project retrospective or from ongoing business monitoring.
Let's apply a couple of key agile principles3 to this process:
- Welcome changing requirements, even late in or after development. Agile processes harness change for the customer's competitive advantage.
- Business people and developers must work together daily throughout the project.
Clearly applying these principles can help you deliver business agility - they ensure that you will respond to changes that are needed because of a changing environment or competitor, and they will ensure that the Strategize-Decide-Communicate sequence runs smoothly and quickly. A challenge arises, however, when the new requirement is not really a requirement at all, but a new or changed business rule. For instance, while new legislation or a court ruling might force new requirements on a project, it is more likely to generate new rules.
But aren't business rules the same as requirements? No, not really. As Kulak and Gurney4 say, "Use cases ... cannot portray all the subtleties of how a business is run. For this, we need business rules ... A list of business rules is not the same thing as a list of requirements ... Business rules are the written and unwritten rules that dictate how a company or agency conducts its business. Requirements relate to a specific application being considered or developed."
If business rules are not the same as requirements, then how do I make sure my agile development processes work just as well for business rules as they do for other kinds of requirements? If a core practice in agile modeling is the idea of single source information - that something required for development should be recorded once in a suitable artifact - what do you do with business rules? Our agile method will have a perfectly good way to record requirements but business rules are different from requirements. Can we have "agile rules"?
"Agile software development teams embrace change, accepting the idea that requirements will evolve throughout a project"5 and indeed after one. To do this, agile projects focus on the highest priority requirements in each iteration. Given that we need to consider business rules as separate from requirements, we need to think about how to manage business rules alongside our other requirements. There are three key artifacts here: decision services, rule templates (or definitions of the kinds of rules), and actual rules.
The decision service "Confirm Eligibility" access a rule repository that contains two templates – Data Completeness and Knock-Out – that allow for controlled entry of critical information from which the four rules shown can be generated.
The highlighted elements of the rules are those editable through the templates while the remaining elements are fixed for each instance of the template.
These artifacts typically require domain experts, developers and knowledge engineers to work in collaboration6 . The best way is for developers to define the decision services, for developers and knowledge engineers together to define the rule templates or types and how they fit into the processing of the decision service, and then knowledge engineers and domain experts manage the rule instances.
Defining decision services is just like defining the interface to any component - the only difference is that you are going to explicitly identify these services as being "decision-centric." These services require the rules for how the company does business to be applied as part of the system. While there are many ways to identify services as decision services, the simplest is to think about the rules or logic being implemented in the service. If any of the following are true you have a candidate decision service:
- Lots of rules - hundreds or thousands
- Rules that change often - monthly, weekly, daily or even hourly
- Rules that are very complex or interact in complex ways
- Rules that require domain knowledge to understand - legal rules and medical rules for instance
Services with one or more of these characteristics are going to benefit from this approach.
Defining templates involves identifying the kinds of rules that exist, categorizing them, and identifying what constructs and data elements are likely to be needed. Some templates are very restrictive, with little variation between rules, and others are very flexible, allowing for a wide range of rules to be managed in the template. Templates can be actual or logical depending on the business rules management system used to implement the business rules.7
Rules, or rule instances, are defined and managed by domain experts. The templates keep them "honest" and ensure the rules conform to the data in the system and the place in the decision service to which they relate. The instances reflect the domain expert's understanding of the business at that time and the target range of test cases - in other words, just the rules needed to manage the test cases identified based on our Test Driven Development.
Remember, Test Driven Development is about writing the test before the program:
"... it leads the developer to first think about ‘how to use' the component (why do we need the component, what's it for?) and only then about ‘how to implement'. First think of the goal, the required functionality then test then code."8
When developing a decision service - something that will use business rules to answer a business question - Test Driven Development is ideal. Most decision services have a very simple interface: a modest amount of data is passed in, and a simple answer is passed back. The complexity is all within. Thus, developing a test or set of tests for the decision service when it is first defined is both natural and straightforward. Domain experts can provide real examples and explain what decision was made in each case. Each of these becomes a test case. Typically, only a core set of cases will be automated to start with, as described above, and new test cases must be known before new rules are written (as the rules will be defined to handle specific cases not handled before) making adding test cases before making changes completely logical.
The process of defining templates is iterative, and the process of defining rule instances highly so; as a result, these processes are well suited to agile approaches. One of the key benefits of using business rules in this way is that it enables real collaboration. When the rules are written in a syntax that everyone can understand, then agreement on them and editing of them is greatly enhanced. In addition, domain experts can develop rules independently, and consequently, some of the change management during an iteration is in their hands. They can change as fast or slow as they like, and cost is controlled, as there is a defined difference between change within expected boundaries (that only requires new or changed rules) versus unexpected change (that requires new decision services or templates). Evolving a system means targeting a certain percentage of decisions and then gradually increasing it by automating exception handling, identifying patterns etc.
By taking this approach, you essentially record a business rule once as human-readable but implementable code, so you can re-use it. This helps deliver business agility in all five stages. If we go back to our definition of business agility, we can see how using business rules contributes during each of the stages:
- Sense - makes it clearer and more explicit what decision was taken
- Strategize - gives business and IT a shared understanding of what could be done with the rules
- Decide - makes it easier to analyze and test possible rules so as to decide what option to take
- Communicate - communicates the rules effectively using formats common to both IT and the business
- Act - allows action in an automated yet agile way
Software to manage and execute these human-readable but implementable rules is well established. Called either Business Rules Engines (BREs) or Business Rules Management Systems (BRMS) they are widely used in decision-heavy industries like Insurance and Financial Services as well as in high-tech companies like Dell and eBay. Fair Isaac with Blaze Advisor and ILOG with JRules are the best known vendors and the two with the largest market shares by far. There are also a number of smaller players such as Haley, Corticon and Yasutech filling out the market, each with its own slightly different approach.
Let's wrap up by considering how all this matches back to the Agile Manifesto3. The manifesto has four key tenets:
- Tenet 1: Individuals and interactions over processes and tools. One of the key interactions is between developers and domain experts. The use of agile rules facilitates this conversation.
- Tenet 2: Working software over comprehensive documentation. Business rules can deliver working software that is easier for domain experts to read and manipulate making it more "self-documenting" and lessening the pressure for documentation.
- Tenet 3: Customer collaboration over contract negotiation. The fact that both developers and domain experts can read and understand business rules allows true collaboration over the implementation of business logic.
- Tenet 4: Responding to change over following a plan Business rules deliver business agility by making the actual code you write easier to change both during the project, and after it.
About the author
James Taylor is vice president for Enterprise Decision Management technologies at Fair Isaac, where he is responsible for working with clients to identify and bring to market advanced decision management solutions that will better solve their business needs. Mr. Taylor has experience in all aspects of software development, including designing, developing, researching and marketing application development technology and platforms.
Truly passionate about decision automation, Mr. Taylor is one of the leading experts and visionaries in enterprise decision management. He actively maintains two blogs on business rules and decisions automation, frequently commenting on subjects from Business Activity Monitoring and Business Agility to Compliance and Business Process Management. http://www.edmblog.com/ and http://www.ebizq.net/blogs/decision_management/
Mr. Taylor is a highly sought speaker, appearing frequently at industry conferences, events and seminars, and in university lecture halls. Along with writing numerous contributed articles for industry publications and reports, he is currently working on a book about the field. His co-authors are Neil Raden, Business Intelligence expert and founder of Hired Brains, and Knowledge Partners Inc.'s Barbara von Halle, who is one of the first pioneers to propose and practice the Business Rules Approach as it is defined today.
1 Gartner: Achieving Agility: SOA Will Build Organizational Agility, but Watch the Hype, available at http://www.gartner.com/DisplayDocument?ref=g_search&id=491111
2 Gartner: Achieving Agility: Defining Agility in an IT Context, available at http://www.gartner.com/DisplayDocument?ref=g_search&id=491393
3 Agile Alliance: Manifesto for Agile Software Development, available at http://www.agilealliance.org
4 Kulak and Guiney: Use Cases - Requirements in Context Second Edition. Boston: Addison-Wesley, 2005
5 Scott Ambler: Agile Requirements change Management, available at http://agilemodeling.com/essays/changeManagement.htm
6 Holger Knublauch: Extreme Programming of Knowledge-Based systems, available at http://www.agilealliance.com/articles/knublauchholgerextrem/file
7 Geoffrey Wiseman, Real-World Rules Engines, available at http://www.infoq.com/articles/Rule-Engines
8 Christoph Steindl: Test-Driven Development , available at http://www.agilealliance.com/articles/steindlchristophtestd/file
Martin Thompson Jul 27, 2014