dooopSoftware · Quality · 12 min

How to Review AI-Generated Code with Less Risk

Review AI-generated pull requests by verifiable intention, evidence proportional to risk, and explicit decision before merging.

Published on September 6, 2026

CORE THESIS

AI-generated code does not require blind trust or automatic veto. It requires clear intention and proof proportional to risk.

The review ends when the PR explains what changed, how it was verified, and what decision the team made.

Reviewing a pull request generated with artificial intelligence starts before the merge: the reviewer needs to understand the intention of the change, read the diff against that intention, and decide if the evidence presented supports the assumed risk.

The focus is not on looking for “AI errors.” It is on connecting the intention of the change, evidence of correction, and explicit decision: approve, request adjustment, limit exposure, or reject.

Start with the Verifiable Intention of the Change

The description “generated with AI” says little. It indicates a probable origin, not a reason for change. To review well, the first shift is simple: treat the code as a proposed system change, not as a special artifact that must be accepted or rejected because of the tool used.

This changes the conversation in the pull request. Instead of asking if the AI wrote elegant code, the reviewer asks what behavior should change. If this intention is missing, the review starts in the dark. A plausible diff may hide scope change, unexplained rule alteration, new dependency, security shortcut, or a refactoring that trades clarity for an appearance of sophistication.

It is also not enough to say “the tests passed.” Existing tests help but may not cover the altered rule. DORA recommends testing throughout development, combining automation and manual activities such as exploration and usability, and also recommends maintaining and reviewing test suites instead of treating quality as a post-development step DORA. For review, this reinforces a practical point: good evidence needs to accompany the change, not just the ritual of opening the pull request.

Leadership’s decision here is to define what a relevant pull request needs to declare before approval. If the team accepts vague descriptions because the diff “looks good,” it defers judgment. If it requires verifiable intention, the team understands what changed, why it changed, and how they know it worked.

This logic aligns with broader AI adoption decisions. An organization may have a good business-connected AI strategy but still fail in routine if it does not translate that strategy into approval criteria in daily work.

How to Turn the Pull Request Description into a Verifiable Intention

A verifiable intention describes behavior. It answers at least four questions:

What should change in the system?
For which user, process, or integration does this change matter?
Under what condition should the change occur?
How can the reviewer observe that the change happened?

Compare two possible descriptions.

“Fix coupon” is not a verifiable intention. It could mean discount calculation, error message, eligibility rule, rounding, misuse in promotional product, or interaction with shipping.

“Prevents coupons from being applied to already promotional products, maintaining normal application on non-promotional products and displaying an invalid coupon message when the rule blocks the discount” already allows review. The expected behavior appears. The scope limit appears. The observable condition appears.

Fictional Example: Coupon Validation in a Cart

Imagine a pull request where an agent changes coupon validation to prevent discounts on already promotional products. A good review starts before line-by-line reading. It checks if the description declares the new rule, if the diff is limited to eligibility calculation, and if the evidence covers at least three conditions: promotional product, non-promotional product, and invalid coupon.

A weak review would accept the pull request because the agent summarized the task as completed and old tests passed. Another weak review would automatically reject because it was generated by AI. Both decisions avoid the central point: does the proposed change correspond to the intended behavior?

The concrete criterion is this: if the description does not allow writing or selecting a test, performing a controlled check, or inspecting a specific rule, it is not yet a reviewable intention.

What Evidence to Request Before Approving Code

Proportional evidence does not mean creating a heavy ceremony for every adjustment. A small text change without impact on rules or flow can have a light review. A change in authentication, authorization, billing, calculation, persistence, configuration, sensitive data, or dependencies requires stronger evidence.

The useful question is not “Is there a test?” but “Does the evidence cover the risk of the change?”

For business rule changes, look for evidence that observes the final behavior. Automated tests help when they express the new rule and its boundaries. Exploratory testing may be appropriate when the change affects experience, flow, or state combinations not yet covered by the suite. Code reading is indispensable when there is structural change but should not replace behavior observation when the risk lies in the result.

Some types of evidence fit different situations:

Automated test for stable rules, repeatable behavior, and likely regression.
Exploratory test for user flow, usability, state combinations, and unforeseen effects.
Code inspection for scope, dependencies, permissions, error handling, and readability.
Manual verification in a controlled environment when the effect depends on configuration, data, or integration.
Comparison with business rule when the risk is misinterpreting an internal policy or operational condition.

The caution is not to turn the checklist into theater. If the rule changed and the evidence is only “compiled,” the review did not demonstrate correctness. If existing tests passed but none exercise the new rule, the evidence may be insufficient. If the change alters a dependency without explanation, the problem may not be lack of test but lack of justifiable scope.

The productivity discussion also calls for caution. In the February 2026 update, METR considers new data an unreliable signal of AI’s current effect on productivity and points out difficulties measuring time with competing agents, participant and task selection METR. This does not decide if a specific pull request is good or bad. It only reminds that review should not be guided by a generic belief in acceleration. What matters in that change is the relationship between intention, evidence, and risk.

How to Review Execution Path Without Confusing Process with Result

Code agents often leave traces: execution plan, messages, comments, change summary, list of modified files, and explanations about tests run. These signals are useful. They help reconstruct the path. But the path is not the result.

Anthropic distinguishes an agent’s execution trajectory from the effective result in the environment. A message saying the task finished is not enough to prove the result; evaluations use inputs, success criteria, and checkers and may require multiple attempts Anthropic. In code review, the analogy is direct: the agent’s summary can guide reading but should not be the main proof.

Three questions help separate process from result:

Did the agent or developer describe steps or demonstrate the expected final state?
Does the evidence observe the running system or only the declared intention in the comment?
Is there an independent verifier apart from the agent’s own message, such as a test, local execution, validation environment, or inspection of the altered rule?

This does not mean distrusting everything. It means choosing where to place trust. Trusting the automatic summary to locate the change may be reasonable. Trusting the same summary to approve a permission, calculation, or persistence change is another matter.

In practice, agent comments serve as auxiliary review material. The reviewer can use them to ask: “If this was the path, where is the proof that the final result matches the intention?” This question avoids both generic skepticism and approval by fluency.

This discipline makes review more auditable. Instead of measuring only tool adoption, the company begins to observe if its teams can make correction criteria and residual risk explicit. For a broader diagnosis of this starting point, it is worth connecting code review to a reading of AI maturity, without confusing maturity with owning a model or automating everything.

When to Request More Tests, Limit Exposure, or Reject

Approval does not have to be binary. In AI-generated code, as in any relevant code, the decision can take four paths: approve, request adjustment, limit exposure, or reject.

Request more tests when the intention is clear, the scope seems coherent, but the evidence is weak. In the fictional coupon example, this would happen if the diff really changes eligibility but tests exist only for non-promotional products. The reviewer’s question is not “Why did the AI err?” but “What relevant limit has not yet been observed?”

Limit exposure when there is reasonable evidence but the impact is sensitive or depends on production variables. The Google SRE chapter on gradual rollouts addresses evaluating a change on a portion of traffic before expanding exposure and distinguishes deploying code from activating features, including using configurations to separate these decisions Google SRE. For review, this suggests a prudent path: approving the merge may differ from releasing the feature to all.

Reject when the intention is ambiguous, the scope changed without explanation, or the evidence does not cover the altered behavior. It also makes sense to reject when the diff introduces dependency, permission, configuration, or structural change unrelated to the declared intention. In these cases, requesting “one more test” may not solve the problem. The issue is that the change is not yet reviewable.

Approval is appropriate when the intention is verifiable, the scope is preserved, the evidence is proportional to risk, and the reviewer can record the reason for the decision in one sentence. This sentence is a good clarity test. If approval depends on “looks right,” “the agent explained well,” or “it always worked this way,” the decision is still fragile.

This reasoning should appear in the AI roadmap as a review policy, pull request checklist, or release criterion, not as an automatic gain promise. Teams adopting AI for development need to decide how they will review, observe, and learn from accepted changes.

Review Protocol: Intention, Evidence, and Decision

Use this checklist in the pull request itself. It does not validate quality by code origin but by the relationship between what was intended to change and what was demonstrated.

Verifiable Intention

Question: Does the change description say which behavior should change, for whom, and under what condition?

Approve if the intention can be checked by test, inspection, or controlled execution. Request adjustment or reject if the description only says “improve,” “fix,” or “refactor” without observable behavior.

Scope Preserved

Question: Does the diff alter only files, flows, and dependencies compatible with the declared intention?

Approve if side changes have clear justification and are small enough to review. Request adjustment or reject if there are unexplained changes in authentication, authorization, calculation, persistence, configuration, or dependencies.

Proportional Evidence

Question: Does the evidence cover the risk of the change, not just the happy path?

Approve if there is sufficient test or verification for the main behavior and at least one relevant boundary. Request adjustment or reject if the evidence is only “compiled,” “the agent said it finished,” or “existing tests passed” when the rule changed.

Result in the Environment

Question: Does the review confirm the expected final effect, not just the steps executed by the agent or developer?

Approve if the result can be observed by test, local execution, validation environment, or defined verifier. Request adjustment or reject if proof depends only on messages, comments, or automatic summary.

Explicit Decision

Question: Can the reviewer record why they approved, requested adjustment, limited exposure, or rejected?

Approve if the decision fits in one sentence linked to intention and evidence. Request adjustment or reject if approval depends on generic trust in the tool, author, or volume of tests.

How to Document the Review Decision Without Bureaucratizing the Team

The record does not need to become a new ceremony. A short comment in the pull request already creates a useful record for later learning.

A good record contains four parts:

Intention: what behavior should change.
Accepted evidence: which test, verification, or inspection supported the decision.
Known risk: which boundary still deserves attention.
Next observation: what will be monitored after merge or during controlled exposure.

In the fictional coupon example, an adequate record could say: “Approved because the change limits coupons on promotional products, the diff was restricted to eligibility, and tests cover promotional product, non-promotional product, and invalid coupon. Monitor error messages in the cart validation flow.”

This comment does not guarantee absence of defects. No review guarantees that. But it improves decision quality by clarifying what was accepted, based on what, and what residual risk remains. If regression occurs later, the team has a starting point to learn. If the change works as expected, the team also learns which evidence was sufficient for that type of risk.

The review ends with a clear sentence in the pull request: intention verified, evidence accepted, risk known, and decision made. When this sentence does not exist, the review is not yet finished.

If you want to discuss this decision in your company’s context, talk to dooop.

Sources

NEXT DECISION

Discuss Application in the Company

Conversation about the software company context

Content from dooop. Registration allows relating this topic to the reader’s journey and tracking interest in the subject.