Some comments on the second day of the October Rules Fest:
The day starts with Dr. Dan Levine talking about brain science. Obviously it isn’t a talk about optimizations for forward chaining production systems. However, it is an interesting talk and goes well with the coffee to get my brain working (I was still on West Coast time). It’s a slightly unusual talk for this conference, but by the end I felt it was a good start for the day.
Next up was Rolando Hernandez for his second talk. Rolando had an interesting graphic on rules vs. decision tables vs. decision trees, and he made an interesting verbal observation about the different thinking styles involved in each. I’m probably going to do a separate post on this since Rolando has graciously made his slides publicly available. Rolando is also thinking about Directed Acyclic Graphs these days, and they turned up later in the day as well.
Next up were Carole-Ann Matignon and Carlos Serrano-Morales from Fair Isaac. Their talk was heavily focused on building scorecards (they use the term ‘predictive analytics’), especially for financial applications such as insurance and mortgages. This is important stuff, but the folks I was sitting behind weren’t paying attention.
Dr. Jacob Feldman (OpenRules) was up next on the topic of how constraint solving can partner with business rules. Quote from Dr. Feldman: “In constraint solving, Rete and sequential are just two choices among many algorithms.” His slides are online at OpenRules.
After lunch, we resumed with John McQuary of Fluor Corporation. John talked about the systems they have built at Fluor, and the organizational issues of adopting such system. The question and answer session had some interesting points. Jason Morris asked about adjusting the corporate culture to work with the knowledge management system. John stated that employees cannot get promoted past a certain point (they can’t be Technical Fellows) unless they work with the knowledge management system. He also said that individuals cannot declare themselves to be a SME, the community as a whole must make that determination. James Owen grabbed me after to ask if I thought the talk was technical enough – in other words would the talk have fit better at the Business Rules Forum? My response was that this was an important topic to be addressed at this conference. System implementers need to be aware of the organizational issues involved in adopting these systems. You can’t simply download a rule engine off the interweb, plug some rules in and deploy to production. (Well, you can, but…) The organization has to adopt the knowledge management system and set up the appropriate lifecycles and processes around it. I’d like to see this topic given treatment again at future ORFs. John also had a final point learned from his experience conducting knowledge engineering: “Some true experts cannot explain how they did it – if they try they may get it wrong.” (John didn’t mention it, but Malcolm Gladwell’s Blink is a survey of research in this area.)
Next up was Carlos Serrano-Morales (Fair Isaac) for his second presentation of the day. Carlos asked the question “Decision trees: do they really scale?” and then proceeded to try to answer that question. There was a nice graphic in the slides comparing a 75 node decision tree to a 1750 node decision. Carlos then showed examples using Directed Acyclic Graphs – which Rolando had already touched upon earlier in the day. (Of course, we all already know that Fair Isaac has been investigating this approach.) Carlos then showed a number of diagrams comparing decisions trees to DAGs. Regarding one slide, Carlos stated “Both [tree and DAG] are unreadable, but this [DAG] is less unreadable.” Another quote from the presentation: “It turns out that Rete is a pretty good way to execute decision trees”.
The day ended with Daniel Selman (Ilog) and Keith Lindsey (USB Bank) discussing a system built with JRules. Daniel had a series of slides outlining typical customer scenarios (compliance/validation, computation, correlation, stateful session) and which of the Ilog execution modes are typically used for each. There was also a chart showing the features of each execution mode – most of this data came straight out of the JRules documentation. This was setting up a discussion of the solution they used: sequential execution mode. Daniel then showed some graphics comparing the performance of each execution mode, although he stated that the numbers were old and out of date – so the graphic was useful for comparing Ilog against Ilog, but not for Ilog against other tools. Over all, the point of the presentation was that we are now selling rules to a range of users and a range of applications – and therefore vendors should provide a range of tools to fit those users and applications. This was a somewhat surprising talk given the Rete-centric nature of the conference, but I appreciated having this approach represented and hope to see similar consideration given to the application of sequential algorithms at future ORFs.
The presentation ended as James Owen focused in on an Ilog execution mode labeled as ‘Rete without node sharing’. (Update: Daniel Selman offered up the appropriate link for this on James Owen’s blog.) I’m sure that there is something on the web about this feature, but haven’t quickly been able to find it.) James was quick to point out that he views node sharing as mandatory in order to be considered an implementation of Rete. James took the conversation to the bar afterward where he managed to get Dr. Forgy to agree with his point.
Thanks for the report, Karl. Our goal for this ORF was to provide some insight on aspects of the decision management eco-system around BRMSs that the pure BRE community tends not to look too much into.
The whole scorecard building business, as you point out, is important. And the key message was around the fact that when building these decision-management centric apps, you need to account for the choices that will enable you to build explainable / legally challengeable predictive analytics into the solution. That will lift the value of the decisions beyond what you can extract from the heads of experts and procedures, while remaining explainable and manageable, and easy to deploy. MS has a good play to make there,
The DAG thing is both graphically entertaining and interesting. How do you best represent knowledge to minimize translation and management risk – key issues that we take for granted. DAGs and EDAGs (where you actually express the logic by saying “in general, the decision is X, except if …, in which case it’s Y, expect if … in which case it’s actually Z, etc…”
The challenge for us in the BRMS / EDM world is to ensure that while we provide support for expressing the logic the most appropriate way possible for the problem and context (organization, training, culture, …), we still maintain support for all the value features in the “management” aspect: debugging, verification, etc.
Interesting stuff.
Is that something you are working on?
Thanks, Carlos. I agree with you on the challenge facing us. Much of my comments here are driven by my prior years of experience in the rules space, and shouldn’t be taken as indications of Microsoft’s direction. When I finally am able to disclose about things that Microsoft is working on, those postings will be pretty obvious.
Yes, I am sure they will be!