dooopSoftware · Process · 13 min

AI Failures: How to Improve Context Without Creating Noise

Learn how to decide when an AI failure becomes context, where to record the correction, and how to avoid contradictory rules in the agent.

Published on September 6, 2026

CENTRAL THESIS

AI failures do not call for longer prompts. They require decisions about where the information should reside.

This article separates isolated errors from permanent context. Improvement only counts if it reduces verifiable ambiguity.

Artificial intelligence failures in development do not improve with an endless collection of new rules. To improve context for AI, the team needs to discover what information was missing, where it should live, and which old guidance needs to be removed.

A failure becomes learning when it reduces ambiguity for the next task. If it only adds another sentence to the prompt, it is probably transferring the problem to the next review.

Separate Isolated Failures from Permanent Context

Not every failure needs to become permanent context. The distinction matters because a poorly placed correction can turn a localized error into contradictory instructions for AI agents.

An isolated failure may belong only to the task. A requirement was poorly written, a file was outdated, someone forgot to mention an exception. Correcting this in the agent’s global prompt can contaminate future tasks with a rule that only made sense in that case.

A recurring failure reveals something else: there is information the team expects the agent to use, but it is unavailable, unclear, or in the wrong place. That is when it makes sense to treat it as a context improvement.

Anthropic defines context engineering as the selection and maintenance of information available to the model during inference, including instructions, tools, external data, and history, within a limited window Anthropic. This definition helps place the discussion correctly. Context is not synonymous with a long prompt. It is the set of information guiding an execution.

The first criterion is to ask: would this failure repeat in future tasks if nothing changed?

If the answer is no, treat it as an isolated correction. If yes, proceed to diagnosis. Improving context for AI is not about recording everything that went wrong. It is about selecting what needs to be reused.

This discipline also protects human review. GitHub’s documentation on Copilot agents describes distinct environments and permissions and emphasizes human supervision and output review GitHub. In practice, the more ambiguous the context, the more review time tends to be spent reconstructing intent rather than evaluating scope, risk, and quality of the change.

Locate the Source of the Failure Before Editing Instructions

The most common reaction after a failure is to open the agent’s instruction and write: “always do X” or “never do Y.” It seems prudent. Often it is noise.

Before editing any instruction, classify the probable source of the failure. The question is not “what did the agent get wrong?” The question is “which part of the context system should have prevented this error?”

Some common sources include:

Lack of objective: the task described a change but did not explain the expected outcome.
Lack of constraint: the agent knew what to change but did not know how far it could go.
Lack of technical standard: there was a team convention, but it was not documented in an accessible place.
Lack of example: the rule existed but was too abstract to guide a similar change.
Lack of test: the expectation was verifiable, but there was no automated test or acceptance criterion.
Lack of permission: the agent performed an action that should have depended on human authorization.

Each cause points to a different destination. A failure due to lack of objective should improve the task description. A failure due to lack of constraint may become a scope rule. A failure due to lack of technical standard might belong in the repository README or an architectural decision. A failure due to lack of verification probably belongs in tests, continuous integration, or review checklists.

This separation avoids a dangerous habit: turning every problem into operational instruction. Prompts accumulate exceptions quickly. After a few rounds, no one knows if it means “preserve the current pattern,” “refactor when duplication is found,” or “avoid changes outside the requested file.” All three phrases may be reasonable. Together, without conditions, they compete with each other.

Choose the Right Destination for the Correction

The destination of the improvement matters as much as the wording of the rule. The more stable and verifiable the guidance, the less it should depend on a loose prompt.

An agent instruction works well for operational behavior: how to request authorization, when to stop, what format to use when explaining risks, what limits to respect during execution.

Repository documentation is better for durable standards: module structure, naming conventions, local commands, test organization, recurring implementation choices.

An architectural decision records choices that should not be rediscovered with every task: why one module does not directly access another, why a certain dependency was avoided, which boundary must be preserved.

A review checklist helps when verification requires human judgment. For example: “confirm if the change altered public behavior,” “check if the requested permission matches the task,” “assess if the change exceeded the approved scope.”

An automated test is the best destination when the expectation can be expressed as verifiable behavior. If the rule can fail objectively, ask why it would only be written for the agent to read.

Continuous integration also enters this discussion. DORA describes continuous integration as frequent integration into the main codebase, accompanied by automated builds and tests, and states that fixing a broken build should have priority over new changes DORA. This point does not make testing the solution for every context problem but reminds that some expectations need to leave conversation and enter the verification flow.

There is a practical rule: if the guidance changes with every task, put it in the task description. If it applies to a repository, put it in repository documentation. If it expresses a design constraint, record it as an architectural decision. If it can be automatically verified, turn it into a test. If it depends on judgment, put it in a review checklist. If it guides the agent’s mode of action, keep it as operational instruction.

This reasoning relates to nearby topics but does not replace them. Well-written specifications help reduce ambiguity from the start. Repository preparation makes standards more accessible. Human review organizes the final decision. If you are structuring this foundation, it is worth connecting this article to the guide on how to create an AI strategy connected to business, because technical context without business decision tends to become elegant bureaucracy.

Replace Old Rules Instead of Stacking Exceptions

The most neglected context improvement is removal.

When a failure happens, the team looks for the missing rule. But rarely looks for the rule that became outdated, vague, or contradictory. This is where context begins to degrade.

Imagine an existing instruction: “prefer small and localized changes.” After a failure, someone adds: “always refactor when duplication is found.” The two guidelines can coexist but only if there is a condition. Without a condition, the agent may justify both a minimal change and a broad refactor. Human review ends up debating interpretation, not solution quality.

The review should answer: does the new rule replace, limit, or complement an old rule?

If it replaces, remove the previous one. If it limits, write the condition. If it complements, explain in which situation each applies.

A new rule that does not eliminate ambiguity is not an improvement. It is context debt.

This debt increases the time spent understanding why an instruction exists. The problem is not having documentation. The problem is having documentation that requires archaeology before any change.

To avoid this, each context update should carry five minimum pieces of information:

which failure it corrects;
where it was applied;
which previous rule it removes, replaces, or limits;
how it will be verified;
when it should be reviewed.

It does not need to become a heavy ritual. It can be a comment in a pull request, a short entry in an instruction file, an update in the decision log, or an item in a checklist. The important thing is to create enough traceability so the next person understands why that guidance exists.

A Scope Failure That Should Not Become a General Rule

Consider a fictional example. A team asks an agent to fix a simple bug on the project registration screen. The expectation was to adjust a validation. The agent changes the validation, modifies a shared component, and touches a permissions module. The change partially passes tests, but human review identifies that the scope became larger than the task justified.

A poor response would be to add to the prompt: “do not touch many files.”

This rule seems clear but is not. How many are many? One central file can be riskier than five test files. A legitimate fix may require crossing more than one module. The rule creates a sense of control without operational criteria.

A better response would be to turn the failure into a verifiable scope limit:

For fixes classified as simple, change only files directly related to the described defect.
If the fix requires changes in more than one functional module, stop execution.
When stopping, explain the dependency found, list affected modules, and request new authorization.
Do not modify permissions, authentication, or API contracts without explicit mention in the task.

This context reduces ambiguity because it defines a stopping condition. It also gives human review a clear authorization point before expanding the change.

But the destination still needs to be chosen. Part of this guidance may live in the agent’s operational instruction, especially the rule to stop and request authorization. Another part may go into the review checklist: “did the change cross functional modules without authorization?” If permissions and API contracts are sensitive areas in that product, this restriction may also belong in repository documentation.

The expected effect should not be treated as a guaranteed result. The hypothesis to measure is that the stop rule will make it more visible when a simple task ceases to be simple. If that happens, the team can better decide when to split the change, when to involve an architecture owner, or when to refuse automation in that segment.

Test the Improvement in a Small Batch Before Consolidating

After adjusting context, there is another temptation: apply the new rule to all agents, repositories, and flows. It is understandable. If the rule seems good, why not standardize?

Because broad context changes can also fail broadly.

DORA recommends small, independent, and testable work units to get feedback on changes and review hypotheses earlier. The guidance also warns about the difficulty of reviewing and integrating large AI-generated changes DORA. Applied to context, this suggests a cautious approach: test the improvement in a small unit before spreading it.

This test does not need to be sophisticated. Choose a task similar to the original failure, apply the new guidance, and observe if it helps make decisions earlier. The question is not only whether the agent “got it right.” The question is whether the new rule reduced unnecessary human interpretation.

Useful criteria for this evaluation:

Did the rule help the agent stop when it should?
Was human review more objective?
Did the rule conflict with any existing instruction?
Did the guidance generate excessive caution in simple tasks?
Was the chosen destination appropriate, or should the rule become a test, documentation, or checklist?

If the improvement only works when someone explains the context again for each task, it is not ready yet. If it avoids a class of ambiguity without blocking legitimate changes, it can be consolidated.

This care becomes more important when agents start acting in repositories, reviews, and development flow automations. An AI roadmap only gains quality when it separates adoption from operational capability. Likewise, an agent instruction only matures when it clarifies what it should decide, what it should verify, and what it should escalate to a person.

Criteria to Update Context Without Accumulating Contradictions

Use this checklist when a failure appears in review, testing, or integration. It does not replace technical judgment. It organizes the conversation so the team does not turn every exception into permanent prompt.

Would the failure repeat in future tasks? If not, treat as isolated correction. If yes, consider updating context.
Was the cause lack of information, lack of limit, or lack of verification? Lack of information tends to go to documentation or example. Lack of limit tends to go to scope instruction. Lack of verification tends to go to test, review, or continuous integration.
Does the new rule contradict any existing instruction? If it contradicts, replace the previous rule or record the exception with a clear condition.
Is the rule stable enough to become permanent context? If it depends only on one task, put it in the task description. If it applies to the repository or team, move it to a more durable artifact.
Is there an objective way to verify the improvement? Prefer automated test, review checklist, or acceptance criterion. If the rule cannot be verified, it probably needs rewriting.
Does the improvement reduce human interpretation work? If it requires a person to explain the exception every time, the context has not improved. It just moved.

The checklist is most useful when used near the failure, before the team forgets the reasoning. After a few days, the tendency is to remember only the visible error, not the condition that produced it.

It is also worth connecting this exercise to a broader maturity assessment. Not every organization needs the same degree of automation, nor is every repository ready for agents to act with the same level of autonomy. The article on AI maturity helps look at capabilities, limits, and governance before scaling practices.

Close the Failure with a Traceable Decision

A well-closed failure does not end with “prompt updated.” It ends with a decision someone can review later.

The decision should state whether the correction entered operational instruction, scope rule, architectural decision, automated test, reference example, review checklist, or task description. It should also say which guidance lost validity.

This point changes the quality of learning. Instead of creating an ever-growing file of advice to the agent, the team maintains a context system. Some information enters. Others leave. Some become tests. Others remain as human constraints. Some do not become permanent because they belonged only to that task.

At the next failure, do not write a new rule before choosing the correction’s destination. Classify the cause, remove previous ambiguity, test in a small slice, and record how the improvement will be verified.

If you want to discuss this decision in your company’s context, talk to dooop.

Sources

To Continue This Reading

How to Plan AI-Assisted Refactoring

NEXT DECISION

Discuss Application in Your Company

Conversation about the software company context

Content by dooop. Registration allows linking this topic to the reader’s journey and tracking interest in the subject.