Automating with or without AI: the question to ask before picking a tool

For two years, nearly every request we get opens with “we would like to automate this with AI”. In most cases, after twenty minutes of mapping, the process does not need AI. It needs a well-built workflow. In a minority of cases it is the reverse: someone tried to automate it with rules and it never held, because the input cannot be reduced to rules.

The difference between the two is not a tooling question. It is a question about the nature of the process, and it gets settled before looking at a single product.

Two families, not one

Rule-based automation is deterministic. The same input always produces the same output. An invoice arrives, the amount exceeds a threshold, it goes to approval. You can write it down, test it, explain it to an auditor. It costs nothing to run and it never fails the same way twice: when it fails, the rule was incomplete, and you fix the rule.

Judgement-based automation is probabilistic. A model interprets a variable input and produces a plausible output. A client email arrives, the model extracts the intent and the useful fields. The same input can yield two slightly different outputs. It costs per call, it is tested by sampling rather than case by case, and it fails unpredictably.

	Rules	Judgement (AI)
Input	Structured, fixed shape	Variable, natural language, documents
Output	Identical every time	Plausible, may vary
Errors	Predictable, fixed once for all	Unpredictable, monitored continuously
Testing	Case by case, exhaustive	By sample, never exhaustive
Cost	Fixed, near zero to run	Per call, grows with volume
When a decision affects a person	Explainable line by line	Needs careful documentation

That last row matters in Quebec. Our post on the CAI’s recommendations explains why a decision based on automated processing carries obligations, and why the Commission wants to extend them to partially automated decisions. A written rule is defended in one sentence. A model’s judgement is defended with a file.

Three questions to decide

1. Can you write the rule? Not “we know what to do”. The complete rule, with its exceptions, on one page. If you can, it is rule-based automation, and putting a language model in it adds cost, variability and documentation for nothing. If you cannot after an honest hour of trying, the process rests on judgement.

2. Does the input have a fixed shape? A form, a database field, a file in a known format: fixed shape, rules. An email, a scanned PDF, a conversation, a free-text request: variable shape, and that is exactly where AI earns its place. Not because it decides better, but because it turns a variable input into a structured one.

3. What does a wrong answer cost? A suggested category a human corrects in one click: the error is cheap, judgement is acceptable. A rejected file, an amount paid, data sent to the wrong recipient: the error is expensive, and the final decision must belong to a rule or a person, never to the model alone.

The layout that works in most cases

Real processes are rarely pure. The layout that holds best has three tiers:

AI at the edge, where the input is variable. It reads, classifies, extracts. It turns an email into five fields. It decides nothing.

Rules in the middle. Once the five fields are filled, everything that follows is deterministic: thresholds, routing, validations, writes into systems. It is testable, and it is the part an auditor will examine.

A person at the decision point, where the error is expensive. Not everywhere, which would kill the gain. At the precise spot where question three answered “expensive”.

This layout also settles the compliance question by construction: the model does not make a decision about a person, it prepares the decision. The line is clean and the architecture itself documents it.

The two mistakes we see most

A language model where an if would do. Checking that a field is filled, extracting an order number from a known format, routing by department code. It works, it costs on every call, and the day it gets it wrong, nobody understands why.

Rules where the input is free-form. Three years of regular expressions to read client emails, an exceptions spreadsheet that grows every month, and an employee hand-correcting what the rule misses. That is the case where AI at the edge makes the exceptions spreadsheet disappear.

Before the tool

Map the process, ask the three questions of every step, and only then look at tools. Our post on choosing the first process gives the mapping method, and the one on measuring the gains tells you whether the layout paid off.

To rough it out in two minutes: our selection tool asks the 7 questions that matter, without collecting anything. And if you are unsure about one specific process, the initial audit starts with exactly these three questions.