Four AIs read the same 12 messy docs.
The priciest was the least accurate.
Same 177 fields — smudged receipts, two-column resumes, a handwritten contract override. One shot each. Accuracy landed a coin flip apart: 98.9% vs 93.2%. The bills didn't. Guess which one ran up the biggest bill, then reveal the receipts.
The bill has nothing to do with the accuracy.
Four models, 177 fields each. Every one landed between 93.2% and 98.9% — a coin flip apart. Yet the priciest cost 4.5× the cheapest, and scored dead last.
Most extraction pipelines are hard-wired to one expensive model, on faith that the price buys accuracy. On these 12 documents it didn't. The premium bought nothing you can measure — it was pure overpay.
The handwritten override that separates reading from parsing
A logistics services agreement printed a 30-day termination notice. Someone crossed it out by hand and wrote in 60, then initialed it. A model that only reads the typed text gets this wrong. The correct answer is 60 — and every model had to see the ink to know it.
Agreement upon thirty (30) days'
written notice to the other party.
— per rider, initialed J.M.
Truth = 60 days
The verdict
Every model read the documents. Smudged OCR, GBP vs USD, balance-forward and credit-memo totals, two-column resumes, both handwriting traps — all four landed within 5.7 points on the same 177-field rubric. On accuracy it's effectively a coin flip.
So the thing that most separates them is the bill, and the bill ranges 4.5×. The most expensive model in the field scored last. You paid a premium for a worse answer.
Pick by brand and you're gambling with your budget. Pick the cheapest that still clears the bar — which is all routing is — and you keep the accuracy while cutting the bill to a fraction. "We'll send every doc to whatever's cheapest that works" is a sentence no single lab can say. We can.
Cost to extract all 12 documents once. Accuracy in parentheses. Blind labels — we don't map model to letter.
| Model | Cost | vs cheapest |
|---|---|---|
| A (97.2%)CHEAPEST | $0.0335 | 1.0× |
| B (98.9%) | $0.0425 | 1.3× |
| D (98.9%) | $0.0891 | 2.7× |
| C (93.2%)PRICIEST | $0.1499 | 4.5× |
Four leading models ran this — Claude Fable 5, Claude Opus 4.8, GPT-5.5, and one more. We're not telling you which is which, because on accuracy you couldn't tell either. That's the point.
One shot each
Identical prompt and identical 12 documents, no retries, no cherry-picking. Each result is exactly what the model returned.
Graded against ground truth
177 leaf fields with a fixed rubric: case-insensitive strings, numbers to 2dp, dates ISO. Accuracy = fields correct ÷ 177.
Real bills, one yardstick
Two models ran as Claude agent runs (output chars ÷ 4 × published rate); GPT-5.5 and one more billed on real usage tokens. Same yardstick for all four.
Labels withheld on purpose
We don't map model to letter. If price bought accuracy you'd spot the expensive one on the bars. You can't — because it didn't.
Ran 2026-07-04 · same prompt, one shot each · Two models ran as Claude agent runs via the Anthropic API; GPT-5.5 and one more ran through our at-cost gateway · Labels withheld on purpose.
Send us your AI bill.
Your extraction pipeline has a spread like this too. We route every document to the cheapest model that clears your accuracy bar — and prove it nightly. What you spent, what you'd have spent, receipts attached.