§ Case study · Multi-modal AI
A conversational virtual avatar for a public figure with a decades-deep media catalogue.
A retrieval-grounded conversational avatar of an internationally recognised author and broadcaster — voice, likeness, and a domain-specific RAG over their own books, columns, and interviews. Fans converse with the persona; the persona never invents facts outside the corpus.
an internationally recognised author and broadcaster
Client
Enterprise
Tier
22 weeks
Duration
2026
Delivered
The context
What they came to us with.
A globally recognised public figure with a media catalogue spanning twelve books, twenty years of magazine columns, several hundred hours of long-form interviews, and a substantial documentary back-catalogue. Their team wanted a way for fans and audiences to interact with the persona at scale — not a chatbot with a name on it, but a conversational avatar that looked and sounded like the figure, and that only ever answered from the actual body of work.
Two hard constraints. First: absolute fidelity to source. The figure had built a reputation over decades on the specificity of their arguments; anything the avatar said had to trace back to a passage they'd actually written or said. Second: voice and likeness quality high enough to hold up in front of a lifelong reader — not the uncanny-valley approximation.
They had also had a previous attempt built by a well-funded generative-AI vendor. It looked good in the demo and answered questions the figure had never taken a position on. It never shipped.
The approach
What we did, in order.
-
The corpus is the ground truth. Everything else scales to it.
Books, columns, and interview transcripts were the canonical corpus. Video was transcribed with speaker-attributed diarisation and cross-linked back to the source book chapter where the argument was made in longer form. Nothing entered the RAG index that couldn't cite itself back to a published or on-record passage.
-
RAG that respects the author's structure.
Chunking followed argument boundaries — chapter, section, and transcript-turn — not fixed windows. Hybrid retrieval and a cross-encoder re-ranker fine-tuned on a fixture set of canonical questions. Every generated sentence cited a supporting passage; anything unsupported was replaced or dropped.
-
Voice built from the actual voice, not a proxy.
Custom voice model trained on isolated interview segments — not the mix-and-match style transfer that starts strong and drifts. Prosody adapted to conversational context so the persona sounded different in a lightning answer than in a considered one.
-
Likeness with a boundary.
A rendered avatar produced from a controlled reference set, running at real-time frame rate. Refused to answer prompts that tried to place the persona in unrelated scenes or endorse products it never endorsed — hard rules, testable, auditable.
-
The team who ran it after us.
Everything shipped with the client's own team paired in from week four — the eval harness, the corpus-refresh pipeline, the moderation dashboard, the guardrail policy. When we left, they were adding new source material without our involvement.
How we measured
The bar we held ourselves to.
- Per-topic hallucination rate on a 300-question fixture set drawn from real fan queries, refreshed monthly.
- Citation precision — every generated claim mapped back to a specific passage in the corpus.
- Voice reference-listener evaluation against a hidden test set (blind A/B against genuine short interview clips).
- Guardrail effectiveness on a suite of adversarial prompts designed to place the persona in situations they hadn't actually spoken to.
- Latency budget — sub-second first-word latency; sub-3-second full response for typical questions.
The outcome
Where it landed.
12
Books ingested into the corpus
400+
Hours of video / interview transcripts
0.6%
Hallucination rate on the fixture set (from 14%)
At launch, hallucination rate on the fixture set was 0.6% — down from 14% on the earlier vendor prototype. Citation precision was 100% by construction. The reference-listener evaluation ranked the generated voice within noise of the genuine short clips on a blind panel. Real-time frame-rate rendering held on commodity client hardware.
The figure's team runs it end-to-end today. The corpus grows monthly; each refresh runs through the eval harness before going live. The moderation dashboard has seen precisely one guardrail escalation in three months.
Retrospective
What we'd do differently.
- We under-invested in the corpus-refresh UI in the first six weeks. The team who owns it now would have benefited from that work landing earlier.
- The voice work stayed with a specialist for longer than we planned. Next time we'd embed the specialist inside the delivery team from week one.
- We would spend more time with the figure's on-record archivist earlier — half of the interesting fixture-set questions came from a conversation we didn't have until week eight.
Services used
The moves in this engagement.
S2 · Build
Agentic Systems Engineering
Multi-step LLM workflows with clean topology, tool surfaces, memory, evaluation, and safety — built to land in production, not stall in a demo.
S3 · Build
LLM Integration
Chat, RAG, and the wider integration stack — measured, regression-tested, and cost-aware from the first prompt.
S4 · Build
Multi-Modal AI Integration
Video, speech, vision, and image capabilities — orchestrated across vendors, evaluated end-to-end, cost-controlled by design.
S8 · Build
Bespoke Software Development
Full production software delivery, usually with AI at the core — fixed team, named engagement lead, no staffing pyramid.
Have something with this shape?
A 30-minute discovery call. We'll say honestly whether the pattern fits.