The seven controls an AI system needs before it goes live

Pilots stall in review, not at the model

Most AI pilots do not stall on the model. They stall in review, when security, risk and audit ask for evidence the system was never built to give: what it can reach, what it did, how it was tested, who approved the action and what happens when something underneath it changes. A system designed to answer those questions from its first architecture gets through review. A system that has to reconstruct the answers afterwards usually does not.

The gap between ambition and controls is wide. In BCG's Applied AI Index 2026, 42 percent of companies expect to grant agents genuine autonomy by 2030, but only 5 percent have the full set of controls in place today. The same research found that companies with all six AI controls in place across the enterprise generate three times as much agentic AI value as companies with just one control. Controls are not a tax on the value. They are what allows a system to be trusted with more of the work.

Leaving them out has a known cost. In June 2025 Gartner predicted that over 40 percent of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls. What follows are the seven controls a reviewer will ask to see before an AI system goes live, what each one ensures and the evidence to ask for. They apply whether you build the system, buy it, or have a firm like Caltor build it for you.

Who can do what, and a record of everything it did

The first control is access. Each person, and the system itself, sees only what its role allows. That is harder than it sounds, because an AI system inherits the reach of everything it is connected to: a search index can surface any document it was built from, and an agent that can call a tool can take any action the tool allows. The OWASP Top 10 for LLM applications calls this Excessive Agency, the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from a large language model, and traces it to excessive functionality, excessive permissions and excessive autonomy. Its advice is to limit the permissions the system is granted to the minimum necessary. In practice that means giving the system its own identity, scoping its credentials to the tools and data it needs, and making retrieval respect what the person asking is entitled to see.

Ask to see which accounts and data the system can reach, and what it can change. The answer should fit on one page, and someone in your security team should be able to read it.

The second control is the audit trail. Every step and every decision is recorded: the request, what the system retrieved, the model and prompt version that produced the answer, the action taken and who approved it. Regulators already expect it. The EU AI Act provides that high-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system, and under the amendment that entered into force on 27 July 2026, the obligations for high-risk systems start on 2 December 2027, or on 2 August 2028 for AI built into products such as lifts or toys. In the United States, FINRA's 2026 oversight report lists practices such as storing prompt and output logs for accountability and troubleshooting, and tracking which model version was used and when.

Ask to see the log of one real case, from the request to the decision. If nobody can produce it within minutes, the system cannot be reviewed.

Tested on your cases, and a person decides

The third control is evaluation. Before release, the system's answers are tested on your own cases, graded by the people accountable for the outcome, against a pass mark the business has signed. A public benchmark tells you how a model does on someone else's problem; an evaluation suite tells you how your system does on yours, and it runs again on every change after launch. That is why we treat evaluation as the contract between an AI system and the business. Ask to see the test results, and the pass mark the system had to reach.

The fourth control is the human decision. A person approves every action with real consequences: moving money, changing a customer's account, sending something in the company's name, or deciding something about a person. The EU AI Act requires high-risk systems to be designed so that they can be effectively overseen by natural persons during the period in which they are in use, and OWASP's guidance on Excessive Agency is to require a human to approve high-impact actions before they are taken.

The approval has to be enforced in code, not requested in a prompt. An instruction to always ask first is a suggestion the model can be talked out of; an action that cannot run without an approval record cannot be skipped. Ask to see which actions need approval, and who gives it.

It stays right after launch

The fifth control is exception handling. Cases the system is unsure about, or that fall outside what it was built for, go to a person with the reason attached, so the person does not start from nothing. The rules that send a case to a person are part of the design: a low confidence score, a missing document, an amount above a limit, a request the system was never meant to handle. The share of cases that reach a person is also one of the most useful numbers the system produces. When it moves, something has changed. Ask to see what sends a case to a person.

The sixth control is monitoring. Quality, cost and failures are visible as they happen, not reconstructed after a complaint. The NIST AI Risk Management Framework notes that AI systems may require more frequent maintenance because of data, model or concept drift, and drift only shows up if the live system is measured against the criteria it was released on. Cost belongs on the same screen, for the reasons we set out in what an AI agent costs to run. Ask to see the dashboard your team will watch, and who is called when a number moves.

The seventh control is change control. Model, prompt, retrieval and data changes go live only after the evaluation suite passes, through the same release process as code, with the previous version recoverable. AI needs this more than most software, because some changes arrive whether you plan for them or not: model providers retire models on published schedules, so almost every system will be moved to a new model during its life. Ask to see how a change goes live, and how it is undone.

Ask to see all seven before go-live

None of the seven is new. Access control, audit, testing, approval, exception handling, monitoring and change management are controls a regulated firm already applies to the software it runs. What is new is that an AI system needs all of them at once, and that each is far cheaper to design in than to add after a review has sent the system back. A pilot built without them is a demonstration. A system built with them reaches security, risk and audit with the evidence already in hand.

This is how Caltor builds. The controls are designed in from the first architecture, the threat model is written before the design is fixed, and the evidence for each control is a deliverable your own teams validate. We do not certify a system or replace your review; we build one your reviewers can inspect. If an AI system on your roadmap is waiting on a security or risk review, book a conversation with a founding partner and we will tell you which of the seven it is missing.