Designing AI workflows that survive production

Start from the failure, not the happy path

A demo workflow is designed around the case that works. A production workflow is designed around the case that does not: the malformed document, the ambiguous record, the upstream service that returns a 200 with an empty body.

Before modelling any steps, write down what the process should do when it cannot decide. If the answer is 'a human looks at it', that human's queue is a first-class part of the design, not an afterthought bolted on when accuracy disappoints.

Separate judgement from mechanics

The most durable workflows keep intelligence steps narrow. A model extracts fields or scores a case; deterministic rules decide what happens as a result. This split is what lets you change models without re-testing the entire business process.

• Intelligence steps produce structured, schema-validated output
• Rules consume that output and own every branch and side effect
• Confidence is an explicit field, not an implicit vibe
• Every automated decision records the rule that produced it

Make the confidence threshold a dial

Launch with the threshold set conservatively so most work routes to review. The corrections operators make become your labelled data. As accuracy on that data proves out, raise the threshold and watch the queue shrink.

This turns a risky launch into a gradual handover, and it gives you a defensible answer when someone asks why you trust the automated path.