AI Agent Governance: Which Actions Need Human Approval? Ask What You Can Actually Undo
Sort every recurring AI agent action by what it can affect and whether you can restore the prior state before harm spreads. That gives you three classes: allow with limits, require review, prohibit.
The question is whether you can still pull the action back, and whether you have checked that you can.
Your support agent has drafted a $20 refund. It has the payment tool. Should it press the button?
Require human approval when an agent action can do meaningful harm and nothing contains it in advance. Contained means a hard limit bounds the loss, or a tested undo restores the prior state before the harm reaches anyone. OWASP's guidance on excessive agency states the core plainly: "require a human to approve high-impact actions before they are taken." [3] A contained action can run alone inside its limit, even when it is final. The harder part of AI agent governance is deciding which actions count as high-impact. Two questions do most of that work for anything an agent repeats: what can it affect, and can you restore the prior state before the harm spreads? The answers put the action into one of three classes. Allow it with limits that actually stop it. Require a human to review the exact action before it runs. Or prohibit it by taking the capability away. Having the tool connected settles none of this, because access is not authority. Then name one person who can grant exceptions, and decide in advance what will make you rewrite the rules.
That is the whole method. The rest of this piece is about where the lines fall, and where they quietly move.
Why the question is on the table now
On 18 November 2026 Microsoft announced a runtime policy service for agents. Its description is blunt about where the decision lives: "Each evaluation returns a decision for the agent host to enforce." [1] The announcement lists "transaction limits" and "human-approval steps" among the business rules organizations need, and its control table includes "Refunds and transfers stay within approved limits" and "No binding customer commitment without review." [1]
That is a vendor describing its own design, so it says nothing yet about whether the approach works. But it makes a useful point by its structure: the rule about what an agent may do sits outside the agent and outside the tool. Two days earlier, Axios's CEO newsletter relayed advice from Damon Kachur, a chief executive it quoted: "Clearly define what each agent can access, what actions it can take and when a human must step in." [2] It is one executive's advice, and Axios presents no survey behind it. Still, it is the right question, and each source reviewed for this piece answers only part of it.
Does technical access mean the agent has authority?
No. An agent's permissions, the tools it is connected to, tell you what it can call. They say nothing about what the agent may do with them, for which customer, up to what amount.
The same announcement draws that line at the protocol level: "HTTP 200 means the evaluation completed, not that the action is permitted." [1] OWASP's guidance on excessive agency goes further and tells builders to "implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not." [3]
The practical consequence for a small team: write the rule per action, and enforce it somewhere the agent cannot talk its way past. A sentence in the system prompt asking the agent to be careful with refunds is a preference. A payment integration that rejects the sixth refund of the day is a policy.
Two questions that sort every action
What can it affect? Name who or what is touched: money, a customer, a record other people rely on, a running service, someone's permissions. Then name the worst credible outcome and how far it travels. NIST's AI Risk Management Framework asks organizations to identify and document the "likelihood and magnitude of each identified impact." [4] You do not need a risk register to do that for ten actions. You need ten honest sentences.
Can you restore the prior state before the harm spreads? This is where teams fool themselves, so the definition has to be strict. Undo means you have tested putting things back the way they were, and you can do it before anyone downstream acts on the change. An apology is not an undo. A database rollback cannot pull back a refund that has already reached a bank. Restoring a customer row from version history leaves last hour's emailed report exactly as it was sent.
A "no" to the second question raises the bar: the action now needs either a hard cap that bounds the loss or a human who reviews it first. Irreversibility alone does not force review. A small refund cannot be recalled, but a hard cap keeps the exposure to an amount you have already decided you can live with, so it runs inside the limit. Larger irreversible actions, and anything past the cap, go to review. A "yes" counts only when the undo is tested and finishes before anyone is affected; a rollback that works after customers have already met the problem is a "no" for this purpose. Prohibition is for actions outside the agent's job, actions that widen its own authority, and actions whose worst case you would not accept even with a human signing off.
When either answer is "I don't know", treat it as the worse case. If the action belongs in the agent's job and a human can check it, it goes to review. If not, block it until someone can answer.
Two questions, three outcomes. "Unknown" is never treated as "low".
What can an AI agent do without asking, and what needs a human?
Only actions whose exposure is small or contained, by a hard cap or by a tested undo that finishes before anyone is affected. Everything else either waits for a person or should not be possible at all, and the three classes say which.
Allow with limits. The action is low in consequence, or its exposure is contained: a cap you set in advance bounds it, or a tested undo restores the prior state before anyone is affected. The agent does it without asking. The cap has to be mechanical: when it is reached, the next action fails or moves to review. A cap that exists only in the agent's instructions depends on the agent remembering it. OWASP suggests exactly this kind of damage limit for a mail-sending tool: "implementing rate limiting on the mail-sending interface." [3]
Require review. The action can do meaningful harm that neither a cap nor a fast, tested undo contains, but it is part of the agent's legitimate job and a reviewer can check it before it runs. This is human-in-the-loop review in its narrow sense, the OWASP mitigation quoted at the top: a human approves the specific action before it runs. [3] The reviewer sees the exact diff, recipient, amount and rollback status, and the approval covers that action only. If the agent changes the amount after approval, it needs a fresh one.
Prohibit. The action is out of scope, would let the agent widen its own authority, or has a worst case you would not accept even with a reviewer's sign-off. Here the right control is removal. OWASP's example is an extension that only needs to read documents but "also includes the ability to modify and delete documents." [3] Take the delete function away. A prohibited action does not become available through an exception. If a human needs it done, the human does it through their own access.
None of these buckets is new. Dataiku's governance guide, for one, describes policy boundaries as "what an agent is authorized to do, what it is prohibited from doing, and what requires human approval before proceeding." [5] What the published guidance tends to leave to you is the line itself: where a two-person team running two products should draw it on a Tuesday.
An AI agent governance matrix for a two-product team
The team below is illustrative, not a case study. A founder and one operator run a booking app and a reporting app, with agents doing support and data work across both. Every cap in the table is an example chosen to show the shape of a rule, not a recommendation and not a measured threshold.
| Action | What it can affect | Real undo? | Class and boundary | Basis |
|---|---|---|---|---|
| Draft replies or release notes (both apps) | Nothing until sent | Yes, discard the draft | Allow. Drafts only, no send or publish access | OWASP: minimise functionality [3] |
| Add private routing labels to tickets (booking) | Internal queue order | Yes, if labels are versioned | Allow, up to a daily cap, one queue. Raise the class if a label triggers a customer message | Editorial; OWASP rate limiting [3] |
| Issue a routine refund (booking) | Customer money | No, once it reaches the bank | Allow only matched, pre-authorised refunds to the original payment method, e.g. $20 each and $100 a day. The sixth $20 refund goes to review | Microsoft refund-limit control [1]; caps editorial |
| Pay a supplier (booking) | Company money | No | Review amount, payee, account and purpose. Prohibit any change to payout details | Editorial, from the refund logic |
| Send a customer a commitment, such as a promised date or credit (booking) | Customer reliance | No, a correction does not erase it | Review the exact text and recipient before send | Microsoft binding-commitment control [1] |
| Edit source-of-truth customer rows (reporting) | Every report built on them | Partly: the row restores, sent reports do not | Review the diff, scope and downstream effects | OWASP database permissions [3]; editorial |
| Deploy (reporting) | Running service | Staging: yes, nobody outside the team sees it. Production: partly, code rolls back but users meet the bad release first | Allow to staging with fixture data and no production secrets. Review production with release diff and tested rollback | Editorial |
| Bulk permanent deletion (reporting) | Data with no copy | No | Prohibit for the agent. A human runs maintenance deletes | OWASP delete example [3] |
| Change roles, tokens or its own permissions (both) | The agent's own authority | Technically, but trust is gone | Prohibit. A named admin handles it separately | OWASP least privilege [3] |
| Spend on metered compute or APIs (reporting) | Shared budget | No, spend is spent | Allow under a hard daily and per-job budget that actually stops spend. Review upgrades and commitments | Editorial; Microsoft transaction limits [1] |
Can a reversible action still need review?
Yes, and this is the case the matrix exists to catch. The customer-row edit is reversible in the narrow sense: version history will put the row back. But if a weekly report has already gone out built on the wrong number, restoring the row fixes the database and nothing else. The useful test is whether anything has already acted on the change.
The same applies to volume. One $20 refund inside the cap is a small, contained action. Five in a row against the same account is a pattern somebody should look at. That is why the cap routes the sixth refund to review instead of letting the agent retry in parallel. Small actions can add up to a large exposure, and a cap is how you notice.
Who grants exceptions, and when should the policy be revisited?
Pick one owner. In the example, the founder owns the policy and every exception. The operator reviews only the action classes the policy assigns to review, and cannot widen them. NIST's framework expects this kind of clarity: "organizational roles and responsibilities clearly defined, including determining the frequency of periodic review." [4] GAO's accountability framework makes governance and monitoring two of its "four complementary principles." [6] Neither tells a two-person team how to run it day to day, so here is one workable shape.
An exception record holds the action and its target or amount, the reason, who approved it, its scope, when it expires, and what happened after it ran. Two defaults keep it honest. If nobody with authority is available, the action does not run. If an approval has expired, it does not run either.
Then set the triggers for rewriting the rules. A fixed date works as a backstop; in the example it is the end of the first month. The better triggers are events: a near miss, a cap that keeps getting hit, a new integration, a new kind of customer data, a rollback that failed when you tried it, or a review queue that is growing faster than anyone clears it.
The strongest objection: this just rebuilds a human queue
It might. If every action needs approval, the agent works at human speed, and the reviewer learns to click approve without reading. No source reviewed here measures how often that happens, so treat it as a design risk to plan for rather than a known rate.
The matrix answers this in three ways. First, most of the volume should sit in "allow with limits", where caps stop the damage and a sample of actions gets checked afterwards. Second, every review is one concrete decision: this diff, this recipient, this amount, this rollback status. If the reviewer cannot verify it, the action waits. Third, review is for actions that are legitimately in scope. Anything that should never happen is removed, so it never reaches the queue.
The other objection is that least privilege and sandboxing make all this unnecessary. They remove capability the agent should not have. They cannot judge whether a refund the agent is allowed to issue should be issued to this customer, today, for this amount. OWASP lists minimising permissions and requiring human approval as separate mitigations for a reason. [3] You need both.
Where the matrix stops working: this is not compliance guidance, and regulated sectors have their own rules. Teams whose products share a budget or a database need caps that add up across products rather than per product. And if you have never tested a rollback, treat every "yes, we can undo it" as "unknown".
Sort your three most frequent actions this week
Take the three actions your agents run most often. Write down what each can affect and whether you have actually tested putting it back. Put each one in a class.
The action you cannot place is a rule your AI agent governance has not written yet. If one of yours resisted every class, reply with it. Those are the cases worth arguing about.
References
- Microsoft. "Introducing agent controls: An enterprise control plane for agent behavior." Published 2026-11-18. https://commandline.microsoft.com/introducing-agent-controls-an-enterprise-control-plane-for-agent-behavior/. Accessed 2026-11-19.
- VandeHei, Jim. "Axios CEO newsletter: Reining in AI agents." Axios. Published 2026-11-16. https://www.axios.com/2026/11/16/ai-agent-accountability-safety-workforce. Accessed 2026-11-19.
- OWASP Gen AI Security Project. "LLM06:2025 Excessive Agency." https://genai.owasp.org/llmrisk/llm062025-excessive-agency/. Accessed 2026-11-19.
- National Institute of Standards and Technology. "AI RMF Core," AI Risk Management Framework 1.0. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/. Accessed 2026-11-19.
- Dataiku. "AI agent governance guide." https://www.dataiku.com/blog/ai-agent-governance-guide. Accessed 2026-11-19.
- U.S. Government Accountability Office. "Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities," GAO-21-519SP, 2021-06-30. https://www.gao.gov/products/gao-21-519sp. Accessed 2026-11-19.
Related reading
Engineering
Credential Logs Are Not Enough to Prove Authority: Five Fields to CheckYour agent log can show which token was used and which tool ran. It cannot show who approved that specific change, what limit it drew on, or whether that permission still stands. Five dimensions to hold your log against, each with a formal precedent written for somebody else.
Engineering
After Six Months of Vibe Coding, Nobody Can Explain Your Repo AnymoreThe code keeps compiling. What breaks first is the shared map of what lives where and why. Why vibe coding erodes the system map long before it touches the syntax.
Engineering
You're Debugging the Wrong Thing: AI Agent Failures Start Before the First PromptDevelopers blame the model when their AI coding agent reverts fixes, hallucinates features, or loses context. The data says the bottleneck is further upstream than that.
Go deeper
- AI product strategy software
How Agimon turns weekly evidence into focus, budget, and metric decisions.
- Portfolio operating templates
What a portfolio review, a weekly evidence snapshot, and a typed decision should include.
- How Agimon works
The evidence, decision, and ledger model behind every call.