- Document lands in a watched folderTrigger fires on file creationThe drop folder is the entire interface. Nobody logs into anything, which is why it gets used.
- Fetched and prepared for the modelPreparation is where scans and native PDFs stop being the same problem. Everything downstream assumes this step normalised them.
- First pass — extractionFields pulled against a fixed schemaA fixed schema rather than a summary. Free text cannot be checked, joined or totalled; fields can.
- Second pass — verificationThe extraction is read back against the sourceThis is the step most pipelines skip. It is also the step whose value is easiest to overstate — see the failure section.
- Validate and flagStructural checks — missing fields, totals that do not sum, unparseable datesThe only errors the pipeline can prove it caught. Everything else is inference.
- Rows built and written outAppended to a running record, and exported as a spreadsheetTwo destinations because two audiences: the running record is the history, the export is what somebody opens today.
On failureAnything that fails validation is flagged rather than written, so a questionable row never enters the record silently.
Document extraction is the task these systems are genuinely good at, and the one most often shipped without any way to tell whether it worked.
The usual shape is a single model call that returns fields, written straight to wherever the fields were going. It produces output on the first try, which is exactly what makes it dangerous: there is nothing in the pipeline that would behave differently if the extraction were wrong.
This build adds one thing — a second pass that reads the extraction back against the source before anything is written.
Why a second pass at all
The argument for it is straightforward. An extraction is a claim about a document. Claims can be checked against the thing they are about, and checking is cheap relative to the cost of a wrong row entering a system of record.
The argument against it is the one worth taking seriously, and it is in the failure section above: two passes from the same model family are not independent. When they agree, you have learned that the model is consistent. Consistency and correctness are different properties, and only one of them is what you wanted.
So the verification pass earns its place for a narrower reason than it first appears. It catches the extraction that was unstable — the one where a second look produces a different answer. That is a real class of error and it is worth catching. It does not catch the extraction that was confidently and consistently wrong, and nothing in this pipeline does.
What the validation stage actually establishes
Structural checks are the part of this build that can be defended without qualification, because they do not depend on a model being right about anything.
A required field is present or it is not. A set of line items sums to the stated total or it does not. A date parses or it does not. These are decidable, they are decidable cheaply, and a failure is a fact rather than an inference.
They are also, unglamorously, where most real errors surface. A document that extracted badly usually fails arithmetic before it fails judgement.
The transferable part
Nothing above is specific to the document type. The shape — watched folder, extract to a fixed schema, verify, validate structurally, write only what passes — applies to any set of documents where the fields are known in advance and somebody downstream is going to rely on them.
The part worth carrying across is the ordering. Structural validation comes after the model passes and before the write, which means the cheapest and most reliable check is the last thing standing between an extraction and a system of record. Most pipelines put it first, or nowhere.
Failure modes
The verification pass is the point of the build and also its weakest claim. A second call from the same model family can agree with the first for the same reason the first was wrong — shared training, shared priors, shared blind spots. Agreement between the two passes is therefore evidence of consistency, not of correctness, and the build does not distinguish them. There is no evaluation set: nothing here has been scored against documents with known answers, so no accuracy figure is claimed and none should be inferred from the presence of a verify step. The validate-and-flag stage catches structural problems — a missing field, a total that does not sum, a date that will not parse — and those are the only errors the pipeline can actually prove it caught.
Sources and methodology
- Scope
- An open architecture pattern showing two-pass extraction and structural validation. It is not a benchmark for a named model or legal document class, and agreement between model passes is not presented as accuracy.
- How this was produced
- Documented from the working automation canvas. Claims are limited to the pipeline's structure and deterministic validation checks because no human-labelled evaluation set was built.
Read the editorial standards, corrections policy and AI-use disclosure.