§ Case study · Multi-modal AI
Contract-vision extraction for a UK legaltech scale-up.
A multi-modal pipeline that turns scanned contracts into structured records — accuracy high enough to skip the manual QA step on 82% of intake.
a UK legaltech scale-up
Client
Founder & SMB
Tier
10 weeks
Duration
2026
Delivered
The context
What they came to us with.
A small legaltech team was ingesting five thousand scanned contracts a month, extracting a defined set of fields, and pushing the structured records into their downstream product. Every document went through a manual QA step — the founding team's biggest operational cost.
They'd built a first-pass OCR + LLM extraction pipeline that worked, in the sense that it produced answers, but nobody trusted it enough to skip QA on any document. Field-level accuracy was measured only anecdotally.
They wanted two things: a real number on field-level accuracy that would satisfy their compliance officer, and a confidence-gated hand-off so QA could focus on the documents that actually needed it.
The approach
What we did, in order.
-
Ground-truth set from real intake.
We anonymised and hand-labelled 400 real documents across their contract-type mix, per field, with edge cases explicitly annotated. That set became the accuracy bench everything was measured against.
-
Vision for structure, LLM for reconciliation.
OCR gave us tokens with layout metadata; a lightweight layout model recognised sections and tables; an LLM stitched fields together with the section context. Structured output validation at each stage.
-
Confidence at the field level.
Every extracted field carried a calibrated confidence. Thresholds per field type, tuned against the ground-truth set to trade false accepts against manual QA cost.
-
The confidence-gated hand-off.
Documents with all fields above threshold bypassed QA. Documents with any field below threshold were routed to a QA screen that highlighted just the low-confidence fields — not the whole page.
-
The loop back into eval.
QA corrections became new ground-truth. Weekly re-scoring against the growing set kept us honest about drift and let the compliance officer point at a number that was less than a week old.
How we measured
The bar we held ourselves to.
- Per-field precision and recall against the ground-truth set, with a target of ≥99% on each named field.
- Confidence calibration — for each threshold, what proportion of documents cleared and what error rate did they carry.
- QA time-per-document, measured before and after the hand-off went live.
- Drift monitoring — weekly re-scoring on the growing labelled set, alerts when any field crossed 0.3pp.
The outcome
Where it landed.
82%
Intake bypassing manual QA
99.1%
Field-level accuracy (evaluated)
4×
Documents per analyst-hour
Eight weeks in, field-level accuracy was 99.1% on the ground-truth set and 82% of incoming documents were bypassing manual QA. The compliance officer had a rolling weekly number they could point at; the operations lead had a QA queue that was actually manageable for the first time in a year.
The client owns the whole pipeline — the labelling harness, the confidence-gating layer, the drift dashboard. We handed over on the last week and they've since expanded the pipeline to two adjacent document types.
Retrospective
What we'd do differently.
- We hand-labelled the first 200 documents ourselves. In hindsight we'd have the client team do half of them from the start; they'd have caught two field ambiguities we didn't.
- The confidence calibration took longer than we'd budgeted. Next time we'd start it in parallel with the extraction work, not after.
- We would have specced the QA screen with a designer. The functional version worked; a designed version would have moved the QA time-per-document another 20%.
Services used
The moves in this engagement.
S3 · Build
LLM Integration
Chat, RAG, and the wider integration stack — measured, regression-tested, and cost-aware from the first prompt.
S4 · Build
Multi-Modal AI Integration
Video, speech, vision, and image capabilities — orchestrated across vendors, evaluated end-to-end, cost-controlled by design.
Related work
Other engagements in the same shape.
Have something with this shape?
A 30-minute discovery call. We'll say honestly whether the pattern fits.