§ Case study · Agentic systems
An agentic ops-triage copilot inside a UK retail bank.
Analyst-in-the-loop triage across three legacy ticketing systems — mean time to first response cut by more than half, without a single new ticketing surface.
a UK retail bank
Client
Enterprise
Tier
18 weeks
Duration
2026
Delivered
The context
What they came to us with.
The internal support desk was moving twelve-thousand tickets a month across three legacy systems that had never spoken to each other. Every ticket started life in one of the three, got a plain-text hand-off to the next, and ended up with an analyst who had to reconcile the story before doing anything useful.
Time-to-first-response had drifted past the internal SLA for the third consecutive quarter. Adding headcount had been considered and rejected on cost. The bank had run a proof-of-concept with a single-shot LLM classifier and pulled it after two weeks — the classifier was accurate on the training set and unreliable on anything that mattered.
They wanted a copilot pattern with an analyst at the centre — not a fire-and-forget agent, not a chatbot bolted onto a ticketing surface nobody enjoyed using.
The approach
What we did, in order.
-
Read the workflow like an ops person.
Four weeks of shadowing before a line of code. We wrote up the workflow the way a senior analyst would run it — decisions, escalations, the informal look-ups, the phone calls. The agent topology fell out of that document; the topology drove the tool surface, not the other way round.
-
Adapter, not replacement.
No new ticketing UI. We built read-and-write adapters over the three legacy systems and kept the analyst inside the ticketing surface they already knew. New buttons; same window.
-
One agent per verb.
Classifier, drafter, dispatcher, and escalator — each a small agent with a small tool surface. Every step logged, every draft attributed, every dispatch approvable in one keystroke.
-
Guardrail before generation.
A pre-flight step ran a policy classifier before the drafter got the ticket. Anything that touched an account balance, a PII field, or a complaint was routed to a human analyst without the model ever seeing the body. Documented, defensible, testable.
-
Evaluation on real traces.
Eval set drawn from six months of anonymised historical tickets with ground truth from analyst dispositions. Regression bars per class; merges blocked when quality dropped on any class over 2 percentage points.
How we measured
The bar we held ourselves to.
- Per-class classifier accuracy, refreshed weekly against a rolling window of dispositions.
- Draft-to-dispatch approval rate, per analyst and per ticket-type.
- Cost per handled ticket (LLM + tool calls), with a per-ticket cap that halted the agent loop.
- Guardrail effectiveness — the number of policy-relevant tickets routed away from the model before generation.
- Analyst-reported "would I have done this differently" rate, sampled on a rolling basis.
The outcome
Where it landed.
−58%
Mean time to first response
3.2×
Analyst throughput
99.4%
Human-approved dispatches
Six weeks after go-live, mean time-to-first-response was down 58% and analyst throughput was up more than 3×. The interesting metric turned out to be draft approval rate — analysts were dispatching the model's draft in more than 99% of cases where the classifier had picked a confident class, and always writing their own draft where the classifier had flagged low confidence.
The bank has since extended the pattern to two adjacent operations teams. The eval harness is theirs; the runbook is theirs; the internal library is theirs. We stayed on for a two-week hand-over and then stepped out.
Retrospective
What we'd do differently.
- We under-invested in the analyst dashboard in the first six weeks. It should have shipped with the classifier.
- The guardrail categories were designed by us and the compliance team; they should have been co-designed with analysts, who caught two categories we'd missed.
- We'd start the eval harness earlier — before the first agent ran end-to-end. Retro-fitting the fixture set cost us two weeks we didn't have.
Services used
The moves in this engagement.
S2 · Build
Agentic Systems Engineering
Multi-step LLM workflows with clean topology, tool surfaces, memory, evaluation, and safety — built to land in production, not stall in a demo.
S3 · Build
LLM Integration
Chat, RAG, and the wider integration stack — measured, regression-tested, and cost-aware from the first prompt.
S5 · Advise
Technical & Solutions Architecture
A senior architect embedded with your programme — end-to-end design, build-vs-buy calls, vendor evaluation, technical risk review.
Have something with this shape?
A 30-minute discovery call. We'll say honestly whether the pattern fits.