Speech-in, structured-out — a small pattern that scales.

The voice interface has to produce a structured record on the other side. The voice is the front door; the record is what the downstream system actually uses. If the record is wrong, the pretty voice interface is worse than useless — it’s misleadingly good.

We’ve built variations of this pattern a handful of times now, and it’s converged onto a shape that reliably scales. Three stages, one schema, one repair loop, one measurement bar. Nothing exotic. All of it worth writing down.

The pattern

Three stages between the microphone and the record:

1. Speech-to-text with domain-tuned vocabulary. Off-the-shelf ASR is good and getting better. It is not good enough on your domain vocabulary, your accents, or your acoustic environment. Take the STT model that’s closest to your workload and give it a domain-specific hotword list, a custom vocabulary, and a small evaluation set drawn from real recordings.

Rule of thumb: your word-error rate on your own evaluation set is the number that matters, not the vendor’s advertised WER.

2. LLM reconciliation to a schema. The transcript goes into an LLM call with two things: the schema you want the record to conform to, and enough context to disambiguate what’s missing.

Given the transcript and the schema below, produce a valid record.
For any field you cannot fill from the transcript with high confidence,
return { field: null, reason: "..." } — never guess.

Schema:
  patient_id: string
  presenting_complaint: string
  duration_days: integer
  medications_mentioned: [string]
  urgency: "routine" | "urgent" | "emergency"

Two design choices matter here:

• Explicit “null with reason” is a first-class output. Guessing is the failure mode you’re preventing; the schema has to make “I don’t know” easier than making something up.
• Enum fields wherever possible. Free-text fields are hallucination surfaces. Enums are checkable at parse time.

3. Schema validation + bounded repair. The output goes through a schema validator. If it fails — invalid enum, wrong type, missing required field — the failure and the invalid output are fed back to the model with a short repair prompt.

The previous output failed schema validation with the following errors: ...
Return a corrected output that conforms to the schema. If a field cannot
be filled from the transcript, return null with a reason.

The repair loop has a hard cap (usually one or two retries). Beyond the cap, the whole record is routed to a human — not “returned with best effort”, not “returned with null” — routed to a human with the transcript and the last invalid output attached.

The measurement bar

Two rubrics, both applied to a small ground-truth set of real transcript-and-expected-record pairs.

Field-level accuracy. Per field, precision and recall. Miss a medication mention and the record is wrong in a way that matters; return a medication that wasn’t said and the record is wrong in a much worse way. Track both separately.

“I don’t know” honesty. For each null field, was the transcript actually silent on that field? A model that returns confident answers on absent information will produce records that look complete and are subtly, dangerously wrong.

Both rubrics run on every merge, against a fixture set that grows over time. Regressions on either fail the build.

Why this scales

The pattern scales because none of the stages are load-bearing on their own. If STT quality drops on a new accent, only field-level accuracy drifts; the schema still catches malformed outputs; the repair loop still recovers most cases; the routed-to-human path is still there.

Every stage is a floor, not a ceiling.

Where it goes wrong

Three failure modes worth watching:

Silent schema drift. Somebody adds a field to the downstream consumer without adding it to the model’s schema. The record has null where it should have data; the record is technically valid; the consumer crashes on a null it wasn’t expecting. Solution: schema versioning at both ends and a compatibility check in CI.

The “helpful” repair. Repair prompts are supposed to fix validation errors. They are not supposed to change the underlying answer. If your repair loop starts producing meaningfully different content from the first pass, you’ve turned it into a two-shot generator by accident. Constrain the repair to structural fixes only.

Fixture-set staleness. The world moves. New medications get approved. New enum values get added. New accents show up. The fixture set has to grow with the world; if it doesn’t, you’ll be measuring against a workload that doesn’t exist any more.

So what?

Boring plumbing. Three stages. One schema. One repair loop. One measurement bar. Nothing that would look impressive in a keynote demo, and everything that reliably produces records the downstream system can trust.

Voice-in, structured-out systems succeed by being unimpressive in production. That’s the target.

Related reading

• RAG for the meeting-notes-hostage — retrieval-shaped version of the same “boring plumbing” argument.
• We measured our prompts. Here’s what changed. — the measurement discipline the repair loop assumes you’re running.