The Failure Test

The demo passed. The job did not.

An invoice extractor can look flawless on five clean PDFs. The acceptance test starts when the input stops cooperating.

Draft for review. Not published. This page is excluded from the public feed and sitemap.

By Sonny Saggar · Drafted 2026-09-22 · Updated 2026-09-22

The invoice arrives sideways. The purchase-order field is empty. A credit note looks almost exactly like a bill. The demonstration did not include any of these.

Consider a hypothetical invoice tool sold on its ability to extract a supplier, a total, and a due date. On five tidy examples it works beautifully. That is evidence about those examples. It is not yet evidence that an accounts-payable team can rely on the workflow.

Specify the decision, then test the output

The useful unit of testing is the job the output enables. If the extraction feeds a payment queue, confusing a credit with an invoice is not merely a formatting error. An unreadable date should not become a plausible date. A missing supplier should not become the closest familiar name.

Write the acceptance rule before tuning the system: preserve the source page, identify each extracted value's location, mark missing fields, and route specified ambiguities to review. Decide which mistakes must stop the workflow and which can be corrected downstream. Those are proposed buyer acceptance criteria, not universal thresholds.

NIST describes its AI Risk Management Framework as voluntary guidance for incorporating trustworthiness into AI design, development, use, and evaluation. Its core organizes work around Govern, Map, Measure, and Manage. That supports treating evaluation as part of operation rather than a one-time demonstration. It does not certify this hypothetical tool. [1]

Build a small hostile sample

Use documents the team is authorized to test, with private information removed where necessary. Include duplicate invoices, a credit note, poor scans, an unfamiliar layout, and an amount that appears in both a subtotal and a total. Keep a held-out set that the builder does not tune against.

Record the expected outcome for each case. Sometimes the correct outcome is a refusal to guess. Count review work as work: a system that needs a person to inspect every field has not demonstrated the promised labor reduction merely because its first pass was quick.

Ask for the failed case

The most informative demo may be the one the supplier cannot complete. Can the operator see why it stopped? Is the source still available? Can the task be retried without creating a duplicate? Does the user have a clear next action?

Do not invent a single accuracy target before understanding the consequence of each error. Agree on the critical cases, the allowed failure behavior, and the evidence the buyer will receive. Then run that test on the actual delivery.

The purchase decision becomes simpler when everyone can point to the same job. A polished screen is welcome. A preserved source, a caught ambiguity, and a safe failed run are often stronger evidence.