What Makes a Good Decision Model?

A good decision model is one that is reviewable and maintainable, not merely one that returns the right answer. Two models can be equally correct while only one of them survives the next amendment to the law it encodes. What separates them is traceability, rationale, constrained types, clean factoring, and tests.

“The problem with regulatory law is when you automate it, it’s not good enough just to get the right results. You have to be able to justify them.” — Dr. Jan Purchase

Key takeaways

1. Point every decision back to its source. Annotate decisions, and individual decision table rows, with the article they implement.
2. Use knowledge sources. The one DMN symbol whose job is to connect a model to the law or policy it automates.
3. Capture rationale, not just the result. Let the why percolate up the model even when the caller does not strictly need it. An unexplained correct answer is still a failure.
4. Keep data validation out of the legal logic. Trap bad data at the boundary; never let one decision both validate input and express a legal definition.
5. Constrain every type. Avoid open, unconstrained string as a model input. Enumerations let the tool prove your table is complete and stop whitespace from silently killing a rule.
6. One decision, one job. Logic buried in an oversized context box is logic the decision requirements diagram no longer describes.
7. Put the tests inside the model. Their real value is not to check “did I break something” but “did I break only what I meant to.”
8. If you generate models with AI, put all seven rules in the prompt. Generated models fail in predictable ways unless the prompt heads them off.

---

Most DMN material teaches you how to build a model that works. Far less of it teaches you how to build a model that someone else can still maintain years later, after the law or policy it encodes has changed.

That gap was the subject of the inaugural Aletyx industry webinar, featuring Dr. Jan Purchase, a co-author of Real-World Decision Modeling with DMN (the DMN “blue book”), with over 25 years of experience modeling decisions in investment banking and longstanding involvement in the OMG DMN specification.

The law presented for the modeling example is the DORA ICT Incident Classification, based on Regulation (EU) 2022/2554, the European Digital Operational Resilience Act. It requires fintech organizations to determine whether a systems incident is “major” and, if so, report it to a national competent authority. Article 9 defines materiality thresholds: if more than 10% of your clients are affected by a single incident, that is a threshold violation, and enough violations make the incident reportable by law.

It is a good case precisely because correctness is the easy part. Two models were assessed, and both models pass testing. So the answer to “what makes a good decision model?” has to be found somewhere other than the output, and it turns out to live in eight specific, checkable practices.

---

1. Make every decision point back to its source

The first model works correctly and tells you nothing about why it exists. Looking at the decision requirements diagram, there is no indication which article of DORA any given decision implements.

“There’s no clue about which articles in the act are supported by each of these decisions. And that’s a bad thing, because a business user reading this may be a bit disoriented at first. And secondly, if that law changes, and it will change, you want some sort of clue as to what parts of your model need to be redeveloped.”

The better model decorates essentially every decision with an annotation naming the articles it supports. This runs all the way down: in the improved model, each row of the decision table carries its own article reference, so a reviewer can see exactly which line of legislation each rule encodes.

The maintenance argument here is the strong one. Regulatory law changes on roughly a yearly cycle. When Article 9.5b is amended, traceability is what turns “audit the entire model” into “look at these three decisions.”

And this holds regardless of who does the editing:

“You need that clue, whether or not you’re an AI. If an AI is doing the modifications, it still needs to know the relationship between the articles and the decisions in here.”

2. Use knowledge sources

One of the most neglected symbols in DMN is the knowledge source: the element whose entire job is to express the relationship between a model and the document it automates, such as the law, the company policy, or the standard.

In the good model, the knowledge source carries a link to the DORA text itself. The model is no longer a free-floating artifact; it states, in the notation, what it claims to implement.

“Please use knowledge sources. So few people do, but they are vital in connecting models to underlying documentation.”

3. Capture the rationale, not just the result

The weak model classifies an incident as major. That is all it does. But there are two distinct ways to reach a “major” classification under DORA, and the output gives you no way to tell which one fired.

The fix is rationale capture: where a part of the model generates information that explains an outcome, you let that information percolate up the model even when the recipient does not strictly need it.

The materiality threshold table already knows exactly which thresholds were breached, such as clients impacted over 100,000 or percentage of clients impacted over 10%. Each violation already carries its article address. In the weak model that detail is computed, counted, and thrown away. Only the count survives.

The improved model outputs the classification and the reason: which part of the act was triggered, how many violations occurred, and what they were.

“The problem with regulatory law is when you automate it, it’s not good enough just to get the right results. You have to be able to justify them.”

Regulatory models are held to a higher standard than ordinary automation: an unexplained correct answer is still a failure.

4. Separate data validation from the legal logic

Before the type discussion proper, there is a structural point that is easy to skip past. If you do not trap bad data at the boundary, you end up handling it inside the model. Now a single decision is doing two unrelated jobs: validating input and expressing a legal definition.

“You’ve got to keep those two things separate. You don’t combine them, otherwise things get very messy.”

Which leads directly to the next practice, because the cleanest place to enforce data validity is the type system.

5. Constrain every type

This is where the weak model is at its worst, and the failures are worth listing individually because they are so common:

Field	Weak model	Problem
Clients impacted	number	DMN’s number includes negatives, so a negative client count is representable
EU state	string	Any string at all counts as a European state
Impact class	string	Unconstrained, and it drives the main decision table

The improved model replaces each with a constrained user-defined type: count (constrained to be greater than or equal to zero), EU state (an enumeration of the actual member states), and service (the four impact classes, documented with the article they come from).

Robustness is the obvious benefit. The subtler benefits matter more:

The tool can now verify completeness. With an open string, the modeling environment raises an information error on the decision table. It cannot confirm that every possible input value is covered, because it has no idea what the possible values are. Once the type is an enumeration, that error disappears: the system can prove the table is complete.

You stop losing rules to whitespace. In a free-form string cell, a stray space turns a rule into one that can never fire:

“I could accidentally put a space in here and that would instantly mean that that rule can never be satisfied… This row would not fire. So the rule essentially has a gap now which cannot be detected.”

With an enumeration you get a dropdown instead of a text box. You can still pick the wrong value, but you can no longer create an invisible, undetectable gap.

6. One decision, one job

Every decision should do exactly one thing.

“This keeps decisions easy to understand, and it means that the Decision Requirements Diagram (DRD) properly explains the scope of the whole decision. This diagram includes everything. There’s no hidden logic in here that’s not in the right place.”

That last clause is the real argument. The DRD is supposed to be a complete map of the model’s logic. Every piece of logic smuggled into a context box that belongs somewhere else is a piece of the model the diagram no longer describes.

The weak model’s threshold-checking table also computes percentages of clients and counterparties affected, a necessary calculation, but one that belongs to incident aggregation, not threshold checking. The improved model moves it there.

“We have to factorize our logic carefully. One decision, one determination.”

On how bad this gets in practice: “I have seen decisions with contexts in them which have 50 or more entries” as modelers pile logic into one place. Refactoring support, grabbing a set of context entries and peeling them off into their own decision, was named as the capability most wanted from tooling today.

7. Put the tests inside the model

The weak model has no tests at all, despite being an executable model. The improved model carries seven test cases, each with a set of inputs, the expected outputs, and an assertion that the two match.

Tests earn their place twice over.

They confirm behavior. Each case records the correct answer and flags you when the answer changes. The tooling also makes an active test traceable back into the model: with a test selected, you can see which rules fired, which decision table rows were evaluated, and how the rationale was assembled.

They show the blast radius of a change. This is the more valuable application. In the demonstration, Article 9.5b is changed so malicious access is always classified as minor, and a test immediately goes red, because a case that expected major now returns minor.

“If you have a set of tests and you make changes to your model, you can use the test to make sure that the consequences were no graver than you expected. You can see in real time the impact of the changes you’re making.”

For a model that tracks legislation, that is the whole game: not “did I break something” but “did I break only what I meant to.”

The honest caveat is that writing test cases is real work. The session points to AI-assisted test generation, and to tools such as the RapidGen test case generator, which imports a DMN model, lets you set distributions and constraints per input, and generates cases directly into your modeling environment.

8. If you generate models with AI, put these rules in the prompt

Everything above applies just as much when a model is AI-generated, and generated models tend to fail in predictable ways unless the prompt heads them off:

• Provide examples in the prompt.
• Ask for structures that bind related information together. Without it, “it inputs almost everything as a single data unit. It gets very complicated very quickly.”
• Demand constraints on every type, especially strings. “There is virtually no justification for having an open, unconstrained string as an input to a decision model. It doesn’t make any sense. Use enumerations instead.”
• Be explicit about traceability: annotations in the DRD, article references on decision table rows, grouping, and knowledge sources.
• Require rationale on node outputs, not just values.
• Enforce one decision, one goal, mapped to one part of the source document.
• Populate comments. Modelers under-comment DMN for the same reason developers under-comment code, and it costs the same.
• Ask for tests.

One structural note that applies beyond AI: for larger regulatory work, define your types in their own model and include them in the models that use them, rather than defining types alongside the logic as both demo models do.

---

Where Decision Modeling is heading

On the tooling side, the stated aim is to bring these practices into the AI assistant itself, so that generated models arrive already traceable, already constrained, and already tested, shifting the modeler’s time from authoring toward review and refinement.

The framing of the industry problem is worth repeating: decision management has always aspired to put modeling in the hands of business experts, and in practice most real-world implementations still lean heavily on IT. The bet is that AI assistance is what finally closes that gap.

The counterpoint from Dr. Jan Purchase is the more interesting half of the exchange. AI currently delivers most of a model, a running start from selected passages of regulatory law, and that share is expected to climb. But:

“One has to be aware, a little bit concerned, about how we keep the modeler interested… you get the same effect you can with pilots and autopilot. They get so used to allowing the automation to do the job that they become atrophied in their skills, and a certain degree of confirmation bias might make them see, ‘yeah, that looks good, I’ll go with that.’”

Which is exactly why the eight practices above matter more as generation gets better, not less. Traceability, rationale, constrained types and tests are what make a model reviewable, and review is the job that remains.

---

FAQ

Is a correct decision model good enough?

No. Correctness is the entry requirement, not the goal. A model that returns the right classification but cannot show which article produced it, which rules fired, or what would break if the law changed is a liability rather than an asset, especially under regulation, where you may have to defend a decision years after it was made.

Should I ever use an unconstrained string as a decision model parameter?

Almost never. As Purchase puts it, “there is virtually no justification for having an open, unconstrained string as an input to a decision model.” Use an enumeration instead. Two things improve immediately: the tool can verify that your decision table covers every possible input value, and you get a dropdown rather than a text box, so a stray space can no longer create an undetectable gap in your rules.

How many test cases does a regulatory decision model need?

Enough to cover distinct paths, including each separate route to the same classification. The improved DORA model carries seven, each with inputs, expected outputs, and an assertion that the two match. The number matters less than the coverage property: when you change a rule, the tests that go red should be exactly the tests you expected to go red.

Where should types live in a larger regulatory model?

In their own dedicated model, included by the models that consume them, not defined alongside the logic. Both demonstration models define types inline, which is acceptable at demo scale and stops being acceptable once several models share a vocabulary.

Can AI generate a good decision model?

Partly, and increasingly. AI already delivers most of a model from selected passages of regulatory law, and that share is expected to grow. But generated models fail in predictable ways, including flat data structures, unconstrained strings, missing annotations, and no tests, unless the prompt explicitly demands otherwise. Ask for structures, constrained types, article-level traceability, rationale on outputs, one decision per goal, populated comments, and tests.

Does traceability still matter if AI maintains the model?

Yes, and arguably more. “You need that clue, whether or not you’re an AI. If an AI is doing the modifications, it still needs to know the relationship between the articles and the decisions in here.” Traceability, rationale, constrained types and tests are what make a model reviewable. Review is the part of the job that does not get automated away.

---