dooopSoftware · Quality · 11 min

How to Record AI Failures to Prevent Recurrence

Separate event, cause hypothesis, and verifiable action to transform AI failures into operational learning, not disputes or noise.

Published on September 6, 2026

CENTRAL THESIS

Hasty recording mixes symptom, cause, and correction. Recurrence stems from this shortcut.

Separate event, hypothesis, and checkable action. The failure becomes a quality input, not fragile memory.

An AI failure record can start with a phrase too short: “the model was wrong” or “the prompt was bad.” The problem is that this kind of note already delivers a conclusion before preserving the fact. To prevent recurrence, the team needs to separate three layers: the observed event, the possible cause, and the action that will be checked later.

When an AI failure becomes learning, and when it becomes noise

An AI failure can appear in different places: an inadequate response in customer service, inconsistent classification in internal triage, a wrong action by an agent, or a recommendation that contradicts a company policy.

The initial reaction is usually quick. Someone changes the prompt. Another person asks to update the database. The team opens a task with a broad title. In more tense cases, the conversation becomes a blame dispute: was it the model, the data, the integration, the user, the evaluator, or the product?

This discussion can open tasks and quick changes but does not always leave enough evidence to prevent the same failure later.

The record of an AI failure needs to serve someone who was not in the meeting, did not see the original screen, and does not remember the context. If this person cannot reconstruct what happened, compare hypotheses, and verify the action later, the document has become fragile memory. It may calm the team for a few days but hardly reduces the chance of repetition.

The difference between learning and noise is not only in the apparent severity of the failure. It lies in the discipline of recording.

For broader topics of adoption and governance, it is worth connecting this care to the discussion about business-connected artificial intelligence strategy. AI failures are not just technical defects. They reveal how the organization decides, tests, accepts risk, and learns.

Record the event before discussing the cause

The event is the verifiable part of the AI failure. It must be written as something another person can verify without relying on the reporter’s memory.

A good event record includes:

input received by the AI;
output, recommendation, or action produced;
expected behavior;
operational context in which it occurred;
relevant version of the product, configuration, database, or flow, when available;
preserved evidence, such as transcript, capture, execution identifier, or internal log;
observed impact or perceived risk.

The order matters. Before asking “why did it happen?”, the team needs to answer “what happened?”.

This seems simple, but it is where many analyses start to lose quality. A record like “the AI hallucinated a refund policy” already mixes event and interpretation. Maybe there was hallucination. Maybe the system retrieved an old document. Maybe the acceptance criteria of the feature did not foresee conflict between versions. Maybe the user formulated the question in a way that exposed an instruction gap.

The event should be written more verifiably: “in a service simulation, after the question about the deadline to request a travel refund, the assistant answered that the deadline was 30 days; the current internal policy used as reference indicates 15 days.”

This formulation does not solve the problem. It improves the investigation.

DORA recommends that tests happen throughout development, combining automation with manual activities such as exploration and usability. It also recommends maintaining and reviewing test suites instead of treating quality as a post-development step. This point is useful here because a well-recorded failure can feed both automated tests and future human reviews.

Treat the cause as a hypothesis, not a conclusion

After the event comes the possible cause. The word “possible” is not a detail. It is protection against convenient conclusions.

In AI products, different causes can produce similar symptoms. A wrong answer can come from insufficient data, ambiguous instruction, inadequate context retrieval, model limit, integration failure, previous human action, configuration change, or incomplete acceptance criteria.

Therefore, the cause must be recorded as a testable hypothesis:

main hypothesis: what seems to best explain the event;
alternative hypotheses: other plausible explanations;
evidence supporting the hypothesis;
evidence still missing;
condition that would confirm or discard the hypothesis.

This structure changes the conversation. Instead of asking “what was the root cause?” too early, the team asks “what evidence separates one hypothesis from another?”.

The expression root cause can be useful in some incidents but can also create a false sense of closure. Not every AI failure has a unique, stable, and easy-to-remove cause. Some appear in the interaction between instruction, data, context, interface, user behavior, and system limit.

The care is not to turn the most plausible explanation into a definitive diagnosis.

Anthropic distinguishes the execution trajectory of an agent from the actual result in the environment. A message saying the task finished is not enough to prove the result was achieved. The same logic applies to failure analysis: a coherent narrative about the error is not enough to prove the cause.

If the hypothesis is “the database was outdated,” the team needs to check if the old version was actually available for retrieval, if it was retrieved in that case, if there was a newer competing document, and if the instruction guided the system to prioritize validity. Without this, updating the database may be a useful action but does not confirm that was the cause.

Turn the correction into a verifiable action

The third layer is the verifiable action. It answers a practical question: after the change, how will the team know if the risk of recurrence decreased?

Vague corrective action is one of the most common ways to close tasks without closing problems. “Improve the prompt,” “review the database,” “monitor closely,” and “strengthen testing” may be valid intentions but are not enough as quality records.

A verifiable action needs to say:

what will be changed;
who is responsible for the change;
when or under what condition the review will be done;
what behavior should change;
what verifier, evaluation case, or evidence will be used;
what happens if the failure reappears.

In AI features, the verifier can be an evaluation case, a human review, a regression test, a comparison of responses in a controlled context, or a check in the environment where the action happens. The format depends on the risk and product maturity.

The central point is that the correction cannot depend only on intention.

Anthropic describes evaluations with inputs, success criteria, and verifiers, possibly requiring multiple attempts. This idea helps differentiate “the system responded better once” from “we have a defined way to check if the expected behavior was achieved.”

For teams still structuring quality in AI products, the discussion also connects to the diagnosis of AI maturity. Maturity here does not require a proprietary model or sophisticated architecture. It requires the ability to observe, record, decide, and review consistently.

Test if the record allows reconstructing the failure

Fictional example: a company uses an internal assistant to answer questions about administrative policies. In a service simulation, a person asks: “What is the deadline to request a travel refund?” The assistant answers that the deadline is 30 days. The current internal policy used as reference says the deadline is 15 days.

A poor record would be:

“The assistant gave a wrong answer about refunds because the database was outdated. We will update the database and monitor.”

This record seems objective but mixes the three layers. It partially describes the event, fixes a cause without sufficient evidence, and proposes an action without verification criteria.

A better version separates the layers.

Event:

“In a service simulation, the assistant stated that travel refunds could be requested within 30 days. The current internal policy used as reference is 15 days. The answer appeared after the question: ‘What is the deadline to request a travel refund?’ The available evidence is the simulation transcript and the current policy document.”

Possible cause:

“Main hypothesis: the retrieval mechanism brought an old version of the policy. Alternative hypothesis: the assistant’s instruction did not prioritize newer documents when there was a conflict between versions. Necessary evidence: identify which documents were retrieved in that execution and if validity metadata was available.”

Verifiable action:

“Review the set of documents used in retrieval, remove obsolete versions or mark validity, and repeat an evaluation case with questions about refund deadlines. The action will only be considered effective if the assistant answers 15 days in the evaluated case and does not use old policies when a valid conflicting document exists.”

Note that the example does not claim the correction would eliminate future failures. It creates a way to measure if that hypothesis gained strength and if the action changed behavior in the relevant case.

This is the goal of AI failure analysis: reduce recurrence through better investigation and verification, not promise total prevention.

Define when the failure requires containment, correction, or just observation

Not every failure demands the same level of response. Some require immediate containment. Others require planned correction. Others can be just observed, provided they are recorded sufficiently to identify repetition.

The decision should consider concrete criteria:

impact on the user or operation;
possibility of repetition;
AI autonomy to act without human review;
reversibility of the action;
external exposure;
sensitivity of the context;
existence of a reliable verifier;
clarity about expected behavior.

If an AI only suggests a low-impact internal response, the team can record the failure, adjust the evaluation case, and review later. If an agent executes actions in systems, changes data, or sends external communications, containment may come before complete investigation.

Here it is useful to distinguish making code available from activating functionality. The Google SRE chapter on gradual releases addresses evaluating changes on a portion of traffic before expanding exposure and discusses using configurations to separate availability and activation. In AI products, this separation can help reduce exposure while the team investigates a failure.

Good documentation does not replace operational decision. If the risk is relevant, leadership must decide whether to pause, limit, revert, reduce autonomy, or keep the functionality under observation.

Without mechanisms to stop, verify, and learn, trust rests on reporting, not operational control.

Review the failure after correction, not just the task

A closed task does not prove the failure stopped recurring. It only proves someone marked an activity as completed.

After the correction, the team should return to the original record and ask:

was the event reconstructed with sufficient evidence?
was the main hypothesis confirmed, discarded, or still uncertain?
do alternative hypotheses remain plausible?
was the evaluation case repeated?
did the expected behavior appear in the defined verifier?
were there new similar occurrences?
did the documentation generate changes in acceptance criteria, testing, instruction, database, interface, or autonomy decision?

This review avoids a common problem: turning failure analysis into bureaucracy. The value is not in filling fields. It is in improving the organization’s ability to recognize patterns, test hypotheses, and adjust the system with less improvisation.

It is also worth separating isolated failures from recurring signals. A low-impact occurrence may not justify a long investigation. But several similar occurrences, even if small, may indicate a criteria gap, a weak verifier, or a poorly defined product decision.

Operational learning appears when the failure changes something verifiable in the quality system. It can be a new evaluation case, a more precise instruction, an autonomy limit, a document validity rule, an interface improvement, or a containment decision.

AI failure record in three layers:

Event: can another person reconstruct what happened without relying on the reporter’s memory? Does the record include input, output or action, expected behavior, context, evidence, and observed impact?
Possible cause: is the explanation written as a testable hypothesis? Are there main and alternative hypotheses, available evidence, missing evidence, and confirmation or discard conditions?
Verifiable action: does the change allow knowing later if the risk of recurrence decreased? Is there a responsible person, review timing, verification method, and decision if the failure reappears?

In each AI failure record, the three layers must appear without shortcuts: verifiable event, possible cause, and verifiable action. Without this, the organization may correct an occurrence but learns less than it should.

If you want to discuss this decision in your company’s context, talk to dooop.

Sources

To continue this reading

How to review a quality system of a team with AI

NEXT DECISION

Discuss application in the company

Conversation about the software company context

Content by dooop. Registration allows relating this topic to the reader’s journey and tracking interest in the subject.