dooopSoftware · Process · 11 min

How to Use AI to Investigate Bugs Without Skipping Steps

Use AI to organize evidence, compare hypotheses, and define minimal reproduction before confirming the cause of a defect.

Published on September 6, 2026

MAIN THESIS

AI accelerates debugging when it organizes hypotheses. The risk begins when a hypothesis becomes a cause without reproduction.

Use AI to reduce uncertainty, not to close the diagnosis. Proof remains reproducing before and after.

In bug investigation, artificial intelligence works best when it supports the method: describing the symptom, raising competing hypotheses, choosing the most testable, reproducing the problem in a controlled condition, and only then confirming the cause.

AI helps when it improves the quality of questions. It hinders when it becomes a shortcut to skip reproduction.

The Role of AI Before a Hypothesis Becomes a Cause

A defect rarely arrives with the cause written on its forehead. It usually arrives as a symptom: a button that does not complete the purchase, a screen that loads old data, a charge with an unexpected amount, an integration that fails only in some cases.

For those leading product and engineering, the difficulty is not only technical. The problem is deciding where to focus attention without turning the investigation into a random hunt. A recent change can be a clue. A log can indicate a failure point. A user report can reveal a usage condition. But none of these pieces of evidence, alone, prove the cause.

This is where AI for bug investigation can be useful. It can organize scattered evidence, suggest alternative explanations, compare symptoms with code snippets, and propose reproduction paths. It can also help write better questions for logs, tests, and code review.

But there is a clear limit: AI does not replace logs, tests, technical inspection, or human judgment. The GitHub documentation on responsible use of agents with Copilot describes agents with distinct environments and permissions and emphasizes the need for human supervision and review of outputs. In defect investigation, this supervision is not a formality. It is part of risk control.

The useful question is not "what is the bug?" The useful question is: "which hypotheses explain this symptom and how can I refute or confirm each with the smallest possible test?"

Separate Symptom, Hypothesis, and Confirmed Cause

The most dangerous confusion in AI-assisted debugging is treating a plausible explanation as a diagnosis. Language models are good at producing coherent narratives. In software, coherence is not enough.

Three concepts need to remain separate:

Symptom is the observed behavior. For example: "customers with coupon and free shipping receive a discount different from expected."
Hypothesis is a possible explanation. For example: "the free shipping rule is applied before the coupon calculation."
Confirmed cause is the explanation that was reproduced, isolated, and tested. For example: "with this combination of cart, coupon, and free shipping, the error appears before the fix and disappears after the change in calculation."

This distinction seems basic but changes team behavior. When the team calls a hypothesis a cause, it tends to edit code too early. When it calls a cause a symptom, it can get stuck in observation without progressing. The investigation improves when each statement receives the correct label.

A good AI response, therefore, is not the most elegant. It is the one that allows progress from symptom to testable hypothesis and from hypothesis to confirmed cause.

This logic also aligns with broader decisions about AI adoption. In a business-connected artificial intelligence strategy, the gain is not in automating every possible step but in deciding where technology improves quality, speed, and control. In the case of bugs, quality means reducing premature explanations.

Assemble a Minimal Context Package Before Asking for Help

AI depends on the context it receives. Anthropic defines context engineering as the selection and maintenance of information available to the model during inference, including instructions, tools, external data, and history, within a limited window.

In defect investigation with AI, context is not dumping the entire repository into the prompt. Nor is it sending a loose phrase and expecting precision. The goal is to provide enough information to generate testable hypotheses without opening the scope too much.

A minimal context package may include:

expected behavior, described in product language or business rule;
observed behavior, without embedding probable cause;
known steps to reach the error, even if incomplete;
affected version, environment, or configuration;
recent possibly related change, treated as a clue and not as proof;
relevant logs, reduced to the necessary excerpt;
limited code snippet associated with the investigated flow;
known constraints, such as feature flag, external integration, cache, queue, or specific data.

It is also worth telling AI what it should not do. For example: do not suggest broad changes, do not assume cause without reproduction, do not ignore alternative hypotheses, do not use sensitive data, secrets, or credentials.

This care matters because context shapes responses. If you send only the final error, you tend to receive generic causes. If you send symptom, expected rule, reduced logs, and code snippet, you increase the chance of receiving hypotheses that can be tested.

It does not require a proprietary model or sophisticated architecture to start. It requires judgment about what enters the analysis, who reviews the output, and how learning returns to the process. This type of decision also appears in AI maturity diagnostics, when the organization separates tool, process, and responsibility.

Request Competing Hypotheses, Not a Single Answer

How you formulate the request changes the result. "What is the bug?" invites AI to choose a narrative. "List competing hypotheses, evidence for, evidence against, and the smallest reproduction test for each" forces an investigation structure.

A good request can follow this form:

"Based on the symptom, logs, and the code snippet below, list plausible hypotheses."
"For each hypothesis, indicate what evidence would support it."
"Also indicate what evidence would refute the hypothesis."
"Suggest the smallest manual or automated test to reproduce the defect."
"Clearly separate observed facts, inferences, and assumptions."

This format reduces a common bias: the team asking to confirm what it already believes. If the initial suspicion is a recent change, AI can help explore other possibilities: unusual data, execution order, cache, concurrency, rounding, environment difference, unstable integration, or ambiguous business rule.

Good hypotheses compete with each other. They are not for decorating the conversation. They serve to reduce uncertainty.

A strong hypothesis must allow a concrete action: seek a specific log, create a test entry, reproduce a sequence, isolate a dependency, compare two versions, or inspect a delimited snippet. If the hypothesis does not suggest any possible test, it may still be too vague to guide work.

Use Minimal Reproduction as a Truth Filter

Bug reproduction is the point where investigation stops being narrative and becomes operational evidence. A hypothesis should only advance if the team can make the defect appear in a controlled condition, remove irrelevant variables, and observe behavior change after the fix.

This criterion protects against two errors. The first is fixing something that seems related but does not trigger the problem. The second is celebrating a change that only masks the symptom in a specific scenario.

The DORA guidance on small batches recommends small, independent, and testable work units to get feedback on changes and review hypotheses earlier. The same page warns about the difficulty of reviewing and integrating large AI-generated changes. Applied to bug investigation, this points to a practical criterion: before requesting a broad change, reduce the defect to a minimal verifiable condition.

Minimal reproduction does not mean reproducing all error variations. It means finding a controlled scenario that demonstrates the main hypothesis. Then, if necessary, the team expands coverage.

A good confirmation filter asks:

Does the defect appear before the fix?
Is the condition that triggers the defect documented?
Have irrelevant variables been removed or controlled?
Does the fix change exactly the investigated behavior?
Is there an automated test when applicable to the failure type?
Has someone reviewed the explanation, code, and change risk?

The continuous integration described by DORA involves frequent integration into the main code, with automated build and tests. The same guidance states that fixing a broken build should have priority over new changes. In an investigation flow, this reinforces the discipline of taking the bug fix to testing, review, and integration.

How to Test an Intermittent Discount Calculation Error

Imagine a fictional digital store that receives reports of incorrect discounts, but only in some orders. Support identifies an initial pattern: customers with promotional coupons and free shipping seem to receive a final amount different from expected. The error does not appear in all carts.

The team could ask AI: "what is the problem in the discount calculation?" This question would likely generate a plausible but uncertain answer.

A better request would be:

"Based on this symptom, these business rules, these reduced logs, and this calculation snippet, list competing hypotheses."
"Separate observed facts from assumptions."
"For each hypothesis, suggest the smallest reproduction test."
"Do not propose a fix before indicating how to confirm the cause."

AI could suggest hypotheses such as:

the order of rule application is inverted, applying free shipping before the coupon;
there is different rounding between subtotal and final total;
the coupon may arrive as null data in a specific condition;
a cached promotion may be reused between similar carts.

None of these hypotheses is a confirmed cause. They are investigation paths.

The team chooses the most testable hypothesis: order of rule application. Then creates a minimal reproduction with a simple cart, a valid coupon, and free shipping. The goal is not to cover the entire promotional engine. It is to verify if that combination triggers the observed behavior.

If the test fails before the fix and passes after adjusting the calculation order, the hypothesis gains strength. Still, closure requires review: does the test represent the correct rule? Are other coupons affected? Does the change alter existing cases? Does the log confirm the calculation sequence? Was the AI explanation reviewed by someone responsible?

The final learning is not just "we fixed the discount." The useful learning is: "the combination of coupon and free shipping triggered an incorrect calculation order; hypotheses of rounding, null data, and cache were considered; a test was created to protect the rule."

This record improves the next investigation. It also helps distinguish a punctual fix from process improvement.

Criteria to Confirm a Hypothesis in AI-Assisted Investigation

Before closing an AI-assisted investigation, it is worth applying these confirmation criteria. They do not guarantee the absence of new defects but help reduce false confidence.

Was the symptom described without mixing probable cause?

Write the observed behavior, expected behavior, and known conditions. If the description already contains a cause, mark it as an assumption. "Error after the last change" can be a temporal fact. "Error caused by the last change" still needs proof.

Did AI receive sufficient and limited context?

Include logs, steps, version, recent change, and relevant code snippet. Avoid two extremes: dumping information without selection or omitting data that changes interpretation. Good context is one that allows testable hypotheses.

Are there at least two competing hypotheses?

Request alternative explanations and signals that would confirm or refute each. A single hypothesis tends to become a narrative. Investigation improves when the team compares possibilities before editing code.

Does the chosen hypothesis have a minimal reproduction test?

Before changing code, define which input, state, or action sequence should make the defect appear. If no one knows how the error should appear, the investigation may still be too early for a fix.

Does reproduction fail before the fix and pass after?

This is the strongest filter. If the team cannot demonstrate behavior change, the cause is not sufficiently confirmed. The honest result may be a prioritized hypothesis, not a closed cause.

Was the AI output reviewed by a responsible person?

Use AI to raise paths, compare hypotheses, and suggest tests. Keep the technical decision with someone able to assess code, risk, and impact. Human review is not a final stamp. It is part of process design.

Was the learning recorded for the next investigation?

Document the condition that triggered the defect, the confirmed hypothesis, discarded hypotheses, and test created or adjusted. This record helps the team not start from zero when a similar symptom appears.

The Practical Decision for the Next Investigation

In the next defect investigation, do not start by asking AI for a cause. Start by asking for competing hypotheses and the smallest path to reproduce each. Then choose the most testable hypothesis, reproduce the behavior, confirm or discard the cause, and record the learning.

Treat the investigation as closed only when the team demonstrates reproduction before the change and behavior alteration after it.

If you want to discuss this decision in your company context, talk to dooop.

Sources

To Continue This Reading

How to Turn a Failure into Context Improvement

NEXT DECISION

Discussing Application in the Company

Conversation about the software company context

Content by dooop. Registration allows linking this topic to the reader's journey and tracking interest in the subject.