dooopSoftware · Learning · 12 min

How to Turn AI Failures into Testable Hypotheses

Learn to separate fact, probable cause, intervention, and evidence to fix AI failures without losing interpretation or creating regressions.

Published on September 6, 2026

CENTRAL THESIS

A failure does not become an improvement through quick reaction. It becomes one when the cause can be tested without erasing interpretation.

The hypothesis connects context, probable cause, and minimal intervention. The decision depends on evidence and protection signals.

When an artificial intelligence feature fails, the first reaction is usually to fix it quickly. Change the prompt. Adjust a rule. Request more data. The risk is fixing the appearance of the problem without understanding its cause.

Turning a failure into a hypothesis for improvement requires formulating a verifiable causal relationship between the observed fact, a testable explanation, and a defined change. Before altering the product, the team needs to decide which cause to verify and which minimal intervention allows observing whether that explanation holds.

When a Failure Is Not Yet a Hypothesis

A failure in an AI product can be visible yet poorly formulated. "The AI is making mistakes" does not guide decision-making. "The assistant responded with the wrong context" helps a bit more but still mixes event, impact, and suspicion. To become a hypothesis, the failure needs to be described as something observable.

Consider a fictional example. A support assistant informs the user that a ticket was opened, but no ticket appears in the service system. The assistant's phrase says the task was completed. The real environment shows it was not.

In this case, there are at least three different layers:

Observed fact: the assistant confirmed the opening of a ticket without a corresponding record in the system.
Perceived impact: the user believes they will receive service, but the support team has no registered demand.
Initial assumption: perhaps the assistant treats the message generation as task completion.

The hypothesis arises when the assumption becomes an explanation that can be tested. Before that, it is only a possible interpretation.

This distinction is even more relevant in intelligent features because the textual response can seem convincing. Anthropic distinguishes the execution trajectory of an agent from the actual result in the environment: a message saying the task ended is not enough to prove it ended. This observation does not solve the case but helps formulate a better question.

The useful question is not "how do we make AI stop making mistakes?" It is: "under what condition does the assistant confirm an action without external evidence of completion?"

This care also avoids treating every failure as a model training problem. The cause may be in the context sent to the model, the instruction, the called tool, the confirmation flow, the interface, the available data, or the expectation created for the user. In the fictional example, the verification is not that the message seems correct but that there is a corresponding record in the system.

This logic connects to a broader discussion about AI strategy linked to business, but here the decision is narrower: turning a problematic event into a hypothesis that allows an interpretable improvement.

How to Write AI Improvement Hypotheses with Testable Explanation

A good formulation of AI improvement hypotheses does not start with the preferred solution. It starts with a verifiable relationship between context, probable cause, intervention, and expected effect.

A practical way to write it is: if the failure occurs in a certain context, then a probable cause is this, and a defined change will have the expected effect of reducing this type of failure without worsening this other signal.

In the fictional support assistant example, a hypothesis could be:

If the improper confirmation happens when the user requests ticket opening after a long response with multiple steps, then a probable cause is that the assistant interprets the answered intention as task completion without checking the ticket tool's return. Adding a mandatory identifier check before confirmation is expected to reduce confirmations without records without increasing abandonment due to excessive friction.

This sentence does more than describe the error. It creates a bet that can be discussed by product, data, support, and engineering. Each area sees a piece of the hypothesis.

Product evaluates whether the expected experience is clear. Engineering checks if there is a reliable return from the ticket tool. Data defines how to distinguish correct confirmation from confirmation without record. Support helps identify the operational consequence of the failure.

The hypothesis must be specific enough to be refuted. "The model does not understand the business" is a weak explanation because it allows any correction and almost no learning. "The assistant confirms completion before receiving a valid identifier from the tool" is better because it can be verified.

Microsoft describes its experimentation platform as a way to incorporate experimentation into the development cycle, validate hypotheses, measure impact, and iterate products. To apply this discipline, connect each hypothesis to the validation method and the product decision that depends on the result.

How to Define the Change Scope Without Trying to Fix Everything

After formulating the hypothesis, a known temptation arises: taking the opportunity to change everything. Change the model, rewrite the prompt, redesign the flow, alter the interface, and adjust the tool all at once.

This kind of package may even resolve the sense of urgency but reduces interpretability. If the failure decreases, the team does not know which change mattered. If it worsens, they also do not know where to look.

A defined change is small enough to be understood. It needs to preserve a clear relationship between intervention and expected effect, such as verifying the identifier before the completion message.

In the fictional example, changes that are too broad would be:

Change the model used by the assistant.
Rewrite the entire support prompt.
Redesign the support journey.
Change the ticket tool and interface at the same time.

More defined changes would be:

Require a valid ticket identifier before allowing the completion message.
Change only the task confirmation instruction.
Separate "I understood your request" from "ticket opened" in the assistant's response.
Record when the ticket tool does not return confirmation and treat this state as pending.

The choice depends on the hypothesis. If the probable cause is the absence of external verification, the most interpretable intervention tends to be at the confirmation point, not necessarily the entire model.

This does not mean larger changes are prohibited. In some cases, a poor architecture requires redesign. But when the question is to learn from a specific failure, the change should be narrow enough to show whether the chosen explanation holds.

This criterion also helps protect the AI roadmap from a sequence of reactive fixes. Without a clear hypothesis, the team accumulates patches. With hypothesis and defined change, each fix records a tested explanation about the product.

How to Define Evidence Before Implementing the Change

A hypothesis only deserves that name when there is some evidence capable of supporting or discarding it. In AI features, this requires looking at the actual task result, not just the apparent quality of the response.

In the fictional support assistant case, it is not enough to evaluate whether the message became more polite or confident. The evidence needs to connect the response to the environment:

Test input: ticket opening requests in contexts comparable to those that produced the failure.
Success criterion: the assistant only confirms opening when there is a valid identifier in the system.
Verifier: comparison between the message sent to the user and the actual record in the support system.
Error signal: textual confirmation without corresponding record.
Protection signal: increase in abandonment, excessive wait, or improper blocking of valid requests.

Anthropic describes agent evaluations with inputs, success criteria, and verifiers, noting that multiple attempts may be necessary. For this guide, the applied point is simple: if the task depends on an effect outside the message, the evaluation needs to verify that effect.

This care separates appearance evaluation from result evaluation. A response may seem correct and still fail operationally. The opposite can also occur: a less elegant response may be more reliable if it distinguishes received intention, ongoing action, and completed task.

Evidence also needs to be defined before the change. Otherwise, the team tends to choose signals after seeing the result, favoring the most comfortable narrative.

It is not necessary to turn every fix into a complex experiment. But it is necessary to know what would be a reasonable confirmation of the hypothesis, what would weaken it, and which data would be unreliable for decision-making.

Hypothesis Formulation Checklist from Failure

Use this checklist in a brief conversation among product, data, support, and engineering. It does not replace technical evaluation but forces the team to separate observation, explanation, and intervention.

Observable Fact

Is the failure described as a verifiable event without mixing explanation and judgment?

Good example: in test interactions, the assistant informed that the ticket was opened, but the support system did not record the ticket.

Bad example: the AI is inventing processes.

Defined Context

Does the hypothesis specify where the failure appears, for which users, tasks, or conditions?

Good example: the failure appears when the user requests ticket opening after a long response with multiple steps.

Bad example: the failure happens in support.

Testable Explanation

Can the proposed cause be confirmed or discarded by observation, evaluation, or experiment?

Good example: the assistant interprets the response generation as task completion because it does not check the ticket tool's return.

Bad example: the model does not understand the business.

Single or Predominant Change

Does the intervention allow interpreting what caused the observed effect?

Good example: add a mandatory check of the ticket identifier before confirming completion.

Bad example: change the model, rewrite the prompt, and redesign the flow all at once.

Success Criterion

Is there a clear condition to say the hypothesis was supported?

Good example: the assistant only confirms opening when there is a valid identifier in the system, and confirmations without records decrease in evaluated cases.

Bad example: the responses seem better.

Protection Metric

Does the change have at least one signal to detect collateral worsening?

Good example: besides correct confirmation, monitor abandonment, excessive time, or failures in other types of requests.

Bad example: measure only if the original failure decreased.

Subsequent Decision

Does the team know what to do if the hypothesis is confirmed, rejected, or inconclusive?

Good example: confirmed, incorporate into the flow. Rejected, test another cause. Inconclusive, review data and evaluation design.

Bad example: see after the test.

How to Protect Interpretation Against Regressions

A local improvement can create a worsening elsewhere in the product. An assistant that stops confirming tickets without records may become too cautious and start blocking valid requests. Or it may increase service time in simple situations. Or it may work well for one segment and poorly for another.

Therefore, the team needs to observe more than the original failure. Google SRE recommends choosing monitoring considering data speed, calculations, visualization, and alerts, and explains that averages can hide problematic behaviors. In this article, the applied inference to AI products is to avoid a single reading of average performance when the change affects different groups, tasks, or moments.

Microsoft also recommends observing a broad set of metrics and segments during experiments to identify regressions and avoid premature interpretations. The application here is prudential: improving a specific failure does not authorize ignoring side effects.

In the fictional example, the team could monitor three types of signals:

Hypothesis success: ticket confirmation only when there is a valid record.
Related error: cases where the assistant should open the ticket but does not complete the flow.
Protection: increase in abandonment, excessive wait, or worsening in requests that do not involve ticket opening.

The goal is not to create an infinite dashboard. It is to choose enough signals to avoid calling an improvement what only shifted the problem.

This care connects to the topic of AI maturity. Maturity does not appear only in model choice. It appears in the ability to observe, interpret, and decide without relying on isolated impressions.

How to Decide if the Hypothesis Deserves to Become a Product Change

After evaluation, the team needs to decide. The worst closure is to turn any positive signal into automatic authorization to publish. The second worst is to treat inconclusive results as political failure.

There are four useful outcomes:

Confirm: the evidence is compatible with the explanation, the change reduced the observed failure, and protection signals do not indicate relevant worsening.
Adjust: the direction seems promising, but the intervention needs refinement before becoming standard.
Abandon: the evidence weakens the explanation or shows side effects incompatible with the risk.
Investigate more: the data, sample, verifier, or evaluation design do not support a decision.

Microsoft recommends, in post-experiment analysis, verifying if metric changes are compatible with the test design and if data quality issues compromise interpretation before deciding on release. This caution is especially valuable in AI because the most seductive explanation is not always the most verifiable.

In the fictional case, the change should only advance if the team can answer concrete questions: did confirmation start depending on the real identifier? Do the evaluated cases represent the failure context? Was there improper blocking of valid requests? Were the support system data complete enough to verify the result?

If the answer is no, the mature decision may be to investigate more. The advancement criterion is to protect the change against a rushed fix that becomes operational debt.

Failures in AI products are not just incidents to erase. They are opportunities to understand where the system confuses intention, response, and result. The advancement criterion is to define which explanation will be tested and which minimal change will be authorized to verify that explanation.

If you want to discuss this decision in your company's context, talk to dooop.

Sources

NEXT DECISION

Discuss application in your company

Conversation about the software company's context

Content from dooop. Registration allows relating this topic to the reader's journey and tracking interest in the subject.