It all started with Spaghetti… in the late 90′s, Business Rules technology started to become popular based on the premise it would remove the need for spaghetti code. Those independent statements, which all contribute to checking that an applicant is eligible or that a transaction is possibly fraudulent, could be externalized for easier maintenance. It quickly turned into the vision that business users would be empowered to maintain that decisioning logic of course.
Over a decade later, we are back to Spaghetti! Certainly BRMS technology helped significantly reduce the complexity in the software code, leading to much greater agility but…….. we moved the problem from spaghetti code to spaghetti rules!
When I talked to BRMS practitioners, the maturity curve looks something like:
- Learning Curve: “But where do I start???”
- Bliss: “Wow, this is powerful!”
- One step too far: “Oooooh … I have too many rules!”
Business Rules Engines have evolved, as well as infrastructure, allowing for execution of very large decision services in very little time. Runtime performance is not much of an issue. The issue with Spaghetti is really the understandability of decisioning logic.
As a subject matter expert, the temptation is great to keep adding business rules into the pot. After all, it is what business rules are for: they allow you to manage those rules independently and let the engine worry about execution. The problem comes when the number of business rule grows beyond your ability to cope intellectually.
A practical example
You might start with a rule for credit score requirement on a mortgage product, e.g. “Credit score must be at least 650″.
Then comes segmentation based on the loan amount. A jumbo loan only applies if the loan amount is greater than $417k but above this threshold, we could have increasing strict requirements for credit worthiness. For example, 650 might be required for a Jumbo loan but you would want at least 700 for any Jumbo loan above $1M. I am making those up of course.
Then, with still high rates of foreclosure, you may further discriminate the requirements based on neighborhoods that have been particularly affected, based on actual appraised value of the property, based on prediction of job stability…
Before you know it, your decision service for Jumbo loan origination is turning into thousands of rules. The only way to wrap your head around the volume of rules is to stop thinking about them and have the computer think instead… This is where automated verification and validation can take over and pinpoint anomalies such as gaps and overlaps.
The issue with this paradigm is that you really want to leverage the expertise of the skilled knowledge worker. It is critical to provide him/her the opportunity to understand so they can contribute and improve the quality of the decisioning.
Addressing the complexity of exceptions
Carlos described in his own words the challenge of exceptions that pollute clean decision services. I also covered the importance of User Experience in the past. Those are critical aspects that help cope with complexity. If you can identify the best graphical representation and you can structure your knowledge, break it down into chunks, then you can push the limit of what volume you can understand.
The issue is that a single representation may not be appropriate for all people at any point in time in the decision management life-cycle.
Let me illustrate.
The first point we beat to death in the past… You start with a decision table then get exceptions after exceptions and your table ends up being too large for visualization and full of holes. It can’t work any longer. The same could be said for any representation, there is a point at which any will break.
An Epiphany for Carole-Ann!
The point we have not yet discussed much is life-cycle. I am not referring here to the traditional Governance Process but rather the fact that business rules have a life-cycle of their own and the activities involving them are quite diverse. It only occurred to me in the past year or so that my needs for looking at business rules keep changing based on my role as a user in addition to the nature of those business rules.
- Before I start, I need to understand the whole decision service and what piece of decisioning can be improved and how
- When I author the rule, I only care about that one rule and its test cases to make sure it does what I want it to do
- After it is unit-tested, the rule needs to be tested in aggregation in the whole decision service; this is where I want to understand interdependencies between the rules
- When it is ready for approval, I want to ensure it is compliant with my business objectives and corporate guidelines / industry regulations
As such, looking at a Decision Table makes a lot of sense when you have regular rules for which you need to change range limits. It is also very good when you want the visual validation that all rules are consistent: the higher the age, the lower the risk for example. But if I want to add an exception that refers to a brand-new attribute, I would likely prefer to add nodes in a Decision Tree. If I am focusing on a specific use case, I do not care about the whole table or tree and I only want to see the rules that apply to that one use case. If I am the Compliance guy, I may only care about negative outcomes and ensure that we did not apply logic that conflicts with regulations.
The fact is that the nature of the user interface is NOT ONLY applicable to the type of rules that you have… but it is ALSO a function of the task at hand.
As a result, Tom and Mary may need to interact with different perspectives of the same piece of logic at the same time because they are involved in different tasks with different objectives.
Visual Logic
We have called this concept Visual Logic as it refers to the ability to visualize the logic for the best understanding, the best interaction at the time. One person we talked to said, “It is like Business Intelligence on rules”. I think this is very true. There is a real need for better intelligence on the knowledge before, during and after its automation.
I’d love to hear from you whether you had your epiphany much sooner than me and how you have been dealing with those challenges.