Production AI
The distance between a working demo and a system a business can put weight on, and the engineering that closes it.
Most enterprise AI does not fail at the model. It fails in the distance between a prototype that impresses a room and a system an organization can depend on. That distance is engineering: the data path, the permissions, the failure handling, the observability, and the ordinary software that puts a model in front of the people doing the work.
We write about that gap because it is where our engagements live. A pilot that never reaches production is not a cheap experiment. It spends the sponsor's credibility, and it makes the next proposal harder to get signed.
The failure has a pattern. A pilot is scoped as an experiment, so it is built like one: data pulled by hand, credentials borrowed, no owner for the systems it touches, correctness judged by whoever is watching the screen. It works, because the person demonstrating it knows which inputs to use. Then the request arrives to run it for real, and every question that was deferred arrives at once. Which system of record it writes to. Who is accountable when it is wrong. What happens when the model provider changes something underneath it. None of those are model problems, and none of them are solved by a better prompt.
We take the opposite approach and build the production version of the problem from the outset, in a deliberately narrow slice. The system runs inside your own cloud accounts and identity systems wherever the architecture allows, against real data with real permissions applied, and it is judged by an evaluation suite written in the language of the business rather than by a demonstration. The slice is small because the point is not to show breadth. The point is to prove that one path through the system can be trusted end to end, and to surface the integration and access problems while they can still be designed around rather than after the wider rollout has been committed to.
That is why production readiness is a design question here, not a hardening exercise once the demonstration has been approved. Observability, failure handling, rollback, permissions and the runbook belong in the first architecture. It makes the early work look less impressive than a prototype would, and it is the only version of the work that survives contact with an organization. A system your team can operate, change and defend to an auditor is the deliverable. Anything short of that is a demonstration with a maintenance obligation attached.
The articles
-
Getting an AI pilot into production in financial services
What it takes to move an AI pilot to production at a mid-market bank, lender, insurer or wealth firm now that SR 26-2 leaves generative AI to your own review.
-
Why most enterprise AI pilots never reach production
The demo was never the hard part. Pilots die in the gap between a working prototype and a system the enterprise can trust, and that gap is engineering.
-
Evaluation is the contract between an AI system and the business
An AI system without an evaluation suite is a system with no acceptance criteria. Evals are where the business writes down what it is actually buying.
Other topics
- Evaluation and assurance How a business knows an AI system is right, and keeps knowing it after the model underneath has changed.
- Technology strategy Deciding what to build, what to buy and what to leave alone, before the budget is committed.
- All articles Everything the founders have published, newest first.
Talk to a partner
If this is the problem in front of you, describe it and a founding partner will reply.