Reliable document extraction is built in two layers that most people collapse into one. A model reads the document and pulls out the fields; a separate layer of deterministic rules checks those fields before anyone trusts them. The model handles the messy reading; arithmetic and format checks catch the errors. Keeping the two apart is what turns a clever demo into a system finance will actually use. This is how to build extraction that holds up on the messy real documents, the ones a demo never shows.
The problem is not typing, it is the silent errors
Document automation is usually justified by the time saved retyping invoices and contracts into another system. The typing is the visible cost. The expensive cost is what goes wrong underneath: a tax ID with two digits swapped that the accounts reject weeks later, a supplier invoice that enters twice and gets paid twice, a contract that renews itself because nobody noticed the notice period lapsed. None of these are carelessness. They are the predictable result of a process where the only defence against a mistake is one person’s attention on repetitive work.
That reframes the goal. Extraction is about removing the silent errors a tired human misses, with speed as the by-product. A system that types fast but repeats those mistakes has automated the wrong thing.
Two layers: extraction and validation
The central design decision is to separate what the model does from what the rules do. Collapse them into one “AI reads the invoice and posts it” step and you get a system nobody can trust, because when it is wrong there is no way to know. Split them and you get a system that is right, or that says clearly it is not sure.
Extraction, done by the model
The model reads the document and identifies each field: supplier, tax ID, date, line items, base, tax, total, and for contracts the parties, dates and terms. Crucially, it attaches a confidence score to each field, high when the value is printed unambiguously, lower when it was inferred from a poor scan. Confidence is what lets the system route uncertain fields to a human instead of guessing.
Validation, done by rules with no model at all
The extracted fields then pass through deterministic checks that use no AI whatsoever. It is arithmetic and format verification: the tax ID checksum, the line items summing to the base, the base plus tax matching the total, the tax rate being a legal one, the due date falling after the issue date, a duplicate reference matching a document already processed. These rules are certain. They do not have opinions, they either pass or fail, and that certainty is what makes the output defensible.
A document that passes every rule with high confidence on every field posts without a human touching it. Anything that fails a check, or carries a field below the confidence threshold, rises to a review queue with the reason attached. That division is the whole system in one sentence: the model reads, the rules decide whether to trust it.
Read the structured data first, model the rest
Wherever a document already carries structured, certified data, read that first and only use the model for the parts that are genuinely ambiguous. Many invoices now carry machine-readable codes that hold the key fields already validated at source: supplier and buyer identifiers, date, total, the tax breakdown. Where that code is present and legible, those fields enter at full confidence and skip review entirely. The model then handles only what is left, the line descriptions, the payment terms, the free text, which is exactly where ambiguity lives.
This changes the economics of the whole problem. The automatic processing rate ends up depending mostly on supplier quality: the ones sending clean digital documents pass untouched, the ones sending photos of crumpled paper fill the queue. That split is useful information in itself, and it tells you where to push suppliers rather than where to add staff.
The review queue is where trust is won or lost
The exception queue is the part that decides whether people keep using the system or quietly abandon it. Two design choices make the difference.
First, show the source. The screen splits the document on one side and the extracted fields on the other, and clicking a field highlights the exact spot on the document the value was read from. That small touch decides whether a reviewer confirms in two seconds or hunts for the number across a full page, twenty times a day.
Second, explain every flag in plain language. “Validation error” leaves the reviewer guessing. The flag should read “the lines total 1,845 but the document says 1,854, a 9 difference, likely two transposed digits”. The reviewer reads the reason and acts, instead of investigating what the system meant. And they keep three clear exits: correct and approve, archive without exporting when it is a duplicate, or delete. The system never decides on its own when it is unsure.
Contracts are harder than invoices
Invoices have predictable fields. Contracts do not: what matters depends on the type and on what the business wants to monitor, the parties, the object, start and end dates, automatic renewal, notice period, value, jurisdiction. The field that earns its keep is usually the notice period, because it feeds the renewal alerts that stop a contract auto-renewing unnoticed. It is also the trickiest, because a contract will often say one thing in one clause and something different elsewhere. The right behaviour there is to flag the contradiction rather than silently pick one. A system that guesses on a contradiction is a system that will eventually renew something expensive by mistake.
Measure the right thing
One metric matters: the share of documents processed without human intervention. Everything else is secondary. And one common metric quietly misleads, average extraction accuracy. A system at 95% average accuracy can be useless if the 5% of errors are spread through documents that look correct, because then there is no exception queue, just silent mistakes flowing into the accounts. The goal is the opposite: err rarely, and when you do, know it, so the mistake lands in review instead of the ledger. A system you can trust is one that is honest about what it does not know.
Settle the non-technical questions first
When a system handles a firm’s client documents, the hardest questions are not the model. Three things have to be decided before code, not after: where the data is hosted and under what data-processing terms, who is allowed to see and approve what, and how long originals are kept and what happens when that period ends. For confidential or regulated work this means hosting in the right jurisdiction, a proper data-processing agreement, and a model provider on zero retention, plus access logging. Treating these as part of the build is what separates a system that can go live from a prototype that never leaves the demo.
This two-layer pattern, model reads and rules decide, is the same one behind every system we build. The wider approach, and where document extraction fits among other operational systems, is covered in AI systems that do your business’s repetitive work.
Frequently asked questions
How is AI extraction different from OCR?
OCR turns an image into text. AI extraction identifies which text is the tax ID, the total, the notice period, and attaches a confidence score to each. The important part is the layer after it: deterministic rules that check those fields before anyone trusts them.
Can I trust AI to enter invoices into accounting automatically?
Only with a validation layer. The model reads, and separate arithmetic and format rules, the checksum, the sums matching, the rate being legal, decide whether the result is safe to post. Documents that pass every rule with high confidence post automatically; the rest go to review.
What should the review queue show?
The source document beside the extracted fields, with a click highlighting where each value came from, and every flag explained in plain language rather than as a generic error. The reviewer should be able to confirm or correct in seconds.
How do you measure if document extraction is working?
By the share of documents processed without human intervention, not average accuracy. Average accuracy hides silent errors; the goal is to err rarely and to catch it when you do, so mistakes land in the review queue rather than the accounts.
Drowning in invoices or contracts?
Document extraction is often the clearest first system to build. Request a strategy call and we will look at your documents and what could be automated safely.
