We are sometimes asked to review a system a previous firm built and a client stopped using. We have done five. In four, the model was fine: accuracy within a point of launch, latency acceptable, no incident. What had failed was everything around it.
What actually broke
- Nobody owned the evaluation threshold. When the model’s accuracy drifted two points, nobody knew whether that mattered, so somebody turned it off.
- The training data’s lineage lived with an engineer who left.
- The runbook was a slide deck. The first incident at 3am was handled by turning the system off, and it stayed off.
- The people who had built it with the consultants had been moved to the next project.
4 of 5
abandoned systems that were technically fine
How it was measured
Reviewed 2023–2026 across three sectors. The fifth had a genuine data problem, which the handover would also have caught.
What a real handover contains
An operating model with a named owner for the threshold and a named owner for the data. A runbook written by the client’s team, with us reading over their shoulder. A rehearsal: twice, and the second time the consultants say nothing. And a date after which the invoice stops, agreed before the build starts.
The measure of the handover is our own absence.
Tobias Lindqvist, partner