dooopSoftware · Quality · 12 min

How to Limit the Autonomy of AI Agents

Define autonomy by action, impact, and reversibility, with specific permissions, approval when necessary, and objective verifiers.

Published on September 6, 2026

CENTRAL THESIS

Autonomy without scope becomes an operational risk. The limit must fit the action the agent can perform.

Permission, reversal, and verification define the level of autonomy. General trust in the agent is not enough.

An artificial intelligence agent can comment on a ticket, prepare a code change, or execute a change in a real system. The autonomy limit of agents must arise from this difference: which action it performs, with what permission, and how easily the organization can undo the effect if something goes wrong.

The autonomy limit starts with the action, not the agent

Agent autonomy is the capacity granted to an artificial intelligence system to decide or execute steps of a task with less human intervention. In software, this can mean reading a repository, opening a change, commenting in a management tool, adjusting a configuration, triggering a script, or interacting with another system.

A common mistake is to discuss autonomy as a single characteristic of the agent: "it can act alone" or "it needs approval." This formulation is insufficient for engineering, quality, or product leadership. The same tool can have broad autonomy for one action and almost none for another.

Commenting on a ticket is different from changing code. Changing code on a branch is different from publishing a configuration. Publishing a configuration for a controlled group is different from activating it for all users. The agent does not change nature in each case. The risk of the action changes.

A more useful approach is to separate three levels:

Recommend: the agent analyzes the context and proposes an action but does not change the system state.
Prepare: the agent creates the change, organizes evidence, assembles a plan, or leaves an action ready for review.
Execute: the agent changes the state of a system, environment, configuration, data, or operational flow.

This separation helps avoid two bad extremes. The first is blocking any use because the agent "can make mistakes." The second is allowing broad execution because it succeeded in controlled demonstrations. Between these extremes lies process design.

In this article, we treat autonomy as an operational decision about speed, containment, and human judgment. It is not a modernization gesture. It is a choice about where the organization accepts automatic action, where it requires authorization, and where it needs to preserve a person before execution.

How to classify actions by impact and reversibility

Reversibility is a more operational criterion than a generic notion of risk. It asks: if the action goes wrong, can we undo it quickly, safely, and verifiably?

Low-impact actions tend to be easy to undo and are restricted to environments or artifacts without direct effect on users. Examples: suggesting a ticket description, generating a hypothesis for a failure cause, proposing test cases, or preparing a change in an isolated branch. Even so, low impact does not mean absence of rules. It means permission can be broader because the cost of reversal is lower.

Medium-impact actions change some internal state but still have reasonable containment. An agent that opens a pull request, changes a configuration in a test environment, or runs a test suite in staging is closer to real operation. Here, the question shifts from "can it do it?" to "what prevents this action from escaping the expected context?"

High-impact actions have external effects, difficult recovery costs, security risks, availability impact, persistent data changes, or communication with clients. In these cases, the agent may prepare the action, but execution requires explicit authorization, objective verification, and a reversal plan.

The practical rule is simple: the harder it is to undo, the more explicit the authorization must be before execution.

This classification does not replace security, privacy, or risk management policies. It organizes the conversation before these policies are applied. In particular, the existence of good tests does not eliminate the need for explicit permission when the action has high impact or low reversibility. DORA recommends tests throughout development, combining automation with manual activities such as exploration and usability. Based on this, for agents, tests should be treated as evidence of quality, not automatic authorization for any execution.

Permissions must be specific, temporary, and traceable

Permission needs to describe what the agent can do, where, for how long, and with what trace.

If an agent needs to change a configuration, it does not need generic permission to administer the entire system. If it needs to read a set of files, it does not need access to the entire repository. If it needs to perform a task in a test environment, it should not inherit production permission for convenience.

A well-designed permission answers some questions:

What exact action can the agent perform?
In which environment is this action allowed?
Which data, files, services, or configurations can it access?
How long does the permission remain active?
Who or what rule granted the permission?
How does the organization identify what was executed afterward?

Broad permissions are often justified by fluidity: "this way the agent does not get stuck." But operational fluidity without clear limits only shifts the cost to later. When something goes wrong, the team must reconstruct context, discover intention, map effects, and decide reversal under pressure.

The alternative is to grant autonomy by scope. The agent can prepare changes in any working branch but can only open pull requests with a specific label. It can run tests in a controlled environment but not change production variables. It can suggest routing adjustments but not activate the rule for clients without approval.

For this decision, it matters less whether the model is proprietary or third-party and more whether there is control of context, permission, evaluation, and consequence. The point is to limit what the tool can do when connected to real work.

When to require human approval before execution

Human approval should not be a ritual to relieve anxiety. It should appear where it changes decision quality.

There are situations where a human needs to be before the action, not after it. This happens when reversal is uncertain, when the effect reaches the user, when there is persistent data change, when a sensitive configuration changes product behavior, when the action affects security or availability, or when there is no reliable verifier of the result.

It is also worth requiring approval when the environment state is ambiguous. An agent may propose a correct action for an environment not in the expected state. In this case, the failure is not necessarily in the recommendation but in execution out of context.

The approval question should be concrete: "can this action, in this environment, with this reversal plan and this verifier, be executed now?" This differs from asking someone to "review the agent." Reviewing the agent is abstract. Authorizing a delimited action is governable.

This criterion protects the team from a false dichotomy. It is not necessary to choose between total autonomy and manual supervision for everything. The agent can recommend freely, prepare with some permission, and execute only when the action is reversible, observable, and compatible with the environment.

How to verify if the task really happened

Autonomy without verification becomes narrative trust. The agent says it finished, presents a log, describes what it did, and may seem convincing. But the relevant question is different: did the environment reach the expected state?

Anthropic distinguishes the agent's execution trajectory from the effective result in the environment. A message saying the task finished is not enough to prove the result. Agent evaluations use inputs, success criteria, and verifiers and may require multiple attempts.

This distinction is central to defining autonomy. If the organization cannot verify the result of an action, it should not increase the autonomy of that action. The agent can continue recommending or preparing, but independent execution requires objective evidence.

Verifiers can take different forms:

Observable final state: the expected configuration is active in the correct environment.
Test executed: the relevant suite ran and produced a consultable result.
Audit log: the action appears with author, time, scope, and identifier.
Before-and-after comparison: the applied change matches the approved plan.
Operational signal: the system exposes a verifiable indicator that the change was loaded.

Logs and justifications help investigation but do not replace a verifier. Leadership should be suspicious of any flow where the only evidence is the agent's own claim.

How to gradually release autonomy

Releasing autonomy does not have to be a leap. It can be a sequence of controlled exposure.

The first stage is observation. The agent follows tasks, suggests analyses, and allows comparison of its recommendations with human decisions without executing anything.

The second is recommendation. It proposes actions based on real context, but a person decides what to do.

The third is preparation with approval. The agent assembles the change, reversal plan, evidence, and verifiers. A person authorizes or rejects execution.

The fourth is execution in a controlled environment. The agent executes reversible actions in test, staging, or contexts without direct user impact.

The fifth is limited execution in production. Even here, autonomy must be restricted by action, environment, exposure, permission, and verifier.

The Google SRE chapter on gradual releases addresses evaluating a change on a portion of traffic before expanding exposure. It also distinguishes releasing code from activating features and discusses configurations to separate these decisions. Applied to agent operation, this distinction is useful: making the agent's capability available is not the same as activating its autonomy for all flows.

This difference matters for product and engineering. You can have the agent technically ready to perform an action and still activate that autonomy only for one team, one task type, one environment, or one user class. To plan adoption, this separation helps differentiate available functionality from effectively released operational autonomy.

Fictional example: agent that changes support configuration

Imagine a fictional example: a company uses an agent to support routing configuration for support requests. The rule defines which requests go to which internal queues. It is not a medical, legal, or financial case. Still, a bad change can worsen customer experience, overload a team, or hide requests that need response.

In recommendation mode, the agent analyzes queue descriptions, category history, and existing rules. It suggests a new routing rule and explains the hypothesis: certain requests would be directed to a more appropriate team. This effect is a hypothesis to be measured, not an assumed result.

In preparation mode, the agent creates the change in a test environment. It records the proposed rule, indicates which categories would be affected, prepares a reversal instruction, and defines the verifier: the rule must appear active only in the test environment, and a set of simulated requests must be routed to the expected queues.

In controlled execution mode, the organization could allow the agent to apply the configuration in test without prior approval, provided the action is traceable and reversible. In production, the rule changes. Since the action indirectly affects clients, the agent can prepare everything, but activation requires human authorization.

For production, the matrix would be as follows in operational language:

Action: activate new routing rule.
Permission: only for the prepared and reviewed rule, in the defined environment.
Reversibility: deactivate the rule and return to the previous configuration verifiably.
Verifier: confirm configuration state and test routing with controlled cases.
Exposure: activate in a limited way before expanding.
Trace: record who authorized, when executed, what changed, and how to revert.

The discussion shifts from whether the agent "is good enough" to whether that action has conditions for responsible execution.

Authorization checklist for agent actions

Before expanding agent permissions, validate authorization by action with product, engineering, quality, and operations:

Action: will the agent only recommend, prepare, or execute? If the action changes state in a real system, do not treat it as a simple recommendation.
Environment: does the action occur in draft, test, staging, or production? The closer to the end user, the lower the initial autonomy should be.
Reversibility: is it possible to undo the action quickly, safely, and verifiably? If reversal depends on manual investigation or data recovery, require approval before execution.
External impact: does the action affect clients, security, privacy, availability, or public communication? If it affects external parties or sensitive obligations, the agent may prepare but should not execute without defined authorization.
Permission: is the permission specific to that action, in that context, and time-limited? If it allows actions beyond what is necessary, reduce the scope.
Verifier: is there an objective way to confirm the expected result happened in the environment? If the only evidence is the agent's message saying it finished, autonomy should not increase.
Trace: is it possible to know who authorized, what was executed, when it happened, and how to revert? If there is insufficient traceability, keep the agent in recommendation or preparation mode.

Autonomy must be granted by action, not by general trust in the agent. If the team cannot delimit permission, reversibility, and verification, it is not yet facing a model problem. It is facing an operational design problem.

If you want to discuss this decision in your company's context, talk to dooop.

Sources

NEXT DECISION

Discuss application in the company

Discussion about the software company context

Content by dooop. Registration allows linking this topic to the reader's journey and tracking interest in the theme.