dooopSoftware · Learning · 12 min

How to Turn Feedback into AI Evaluation

Learn how to separate useful reports, rewrite tasks, define criteria and verifiers to prevent AI failures from going unnoticed.

Published on September 6, 2026

MAIN THESIS

Feedback is not ready evidence. It only becomes evaluation when it describes task, context, expectation, and risk.

The focus is on transforming rich reports into verifiable cases. The goal is to prevent regression, not to diagnose cause.

Feedback for artificial intelligence evaluations gains value when it stops being just a complaint and starts describing an evaluable situation. The work here is not to diagnose the cause of the failure, but to select reports with task, context, expectation, and recurring risk, rewrite them as AI evaluation cases, and define how the failure will be verified in the next review.

Not all feedback deserves to become a test. But some examples reveal exactly the type of problem that cannot return unnoticed.

When Feedback Deserves to Become an Evaluation Case

An AI product usually generates many comments: "the answer was bad," "did not understand my request," "invented a rule," "did right for one case and wrong for another." Part of this is operational noise. Part is a signal of a failure that needs to enter the prevention system.

The difference lies in the quality of the report. Feedback deserves to become an evaluation case when it describes, or allows reconstructing, four elements:

the task the person was trying to complete;
the context that changed the expected response;
the correct or acceptable result;
the risk that the failure will recur with relevant impact.

If the comment only says "the AI was confusing," it might serve for discovery, support, or qualitative investigation. If it says "the AI recommended a condition that did not apply to the user's plan, even with the plan available in the context," there is raw material for evaluation.

This distinction avoids two common errors. The first is turning any complaint into a test, creating a bloated and hard-to-maintain suite. The second is treating rich reports as isolated cases, solved by a specific fix, without preventing the same pattern from returning in the next instruction, context, interface, or model change.

The criterion is simple: feedback useful for evaluation does not prove overall quality or failure. It identifies a situation that must be reproduced and verified in future changes.

How to Extract the Real Task Behind the Report

Raw reports almost always talk about the apparent response. The evaluation case needs to address the real task.

Imagine a user says: "the assistant gave a wrong answer about reimbursement." This is still insufficient. To turn it into an evaluable case, the team needs to rewrite the episode in operational terms:

what was the user's input;
what state the product was in;
what data was available to the AI;
what action the AI should perform;
what consequence could occur if the AI failed.

The question shifts from "does the answer seem good?" to "in this situation, with this context, does the functionality preserve the rule it should preserve?"

This change is especially relevant in features with AI agents, that is, systems that execute steps to try to complete a task. Anthropic distinguishes the agent's execution trajectory from the actual result in the environment: a message saying the task is finished is not enough to prove the result was produced. For evaluation, this means the test should not accept only a convincing answer. It must verify whether the task was completed or the constraint was respected.

In less autonomous products, the same logic applies. A summary, recommendation, classification, or support answer also needs to be evaluated against the real task. If the person needed to decide the next step, the evaluation should consider whether the output provided sufficient information, asked for clarification when necessary, or avoided inappropriate guidance.

This point connects this article to the debate on task success in AI products: before evaluating an answer, the team needs to know what work that answer was supposed to help complete.

How to Turn Examples into Success Criteria

The next step is to replace adjectives with observable criteria.

"Bad answer" is not a criterion. "Incomplete answer" can also be too vague if no one knows what counts as complete. A success criterion needs to specify what must happen for the case to pass.

Some examples of possible criteria:

minimum completeness: the answer covers the fields or conditions necessary for the task;
respect for restriction: the AI does not recommend an action prohibited by the available context;
correct use of context: the output considers plan, permission, history, language, category, or other relevant information;
request for clarification: when the input is ambiguous, the AI does not force a conclusion;
appropriate refusal: when the task is out of scope, the AI explains the limit without inventing alternatives;
consistency with expected base: the answer matches an internal source defined by the team.

The criterion should not be automatically inferred by the tool. It is a decision of product, data, support, and engineering. The tool can help execute the evaluation, but someone needs to decide what is acceptable in that context.

The comment points to a failure. The criterion defines what needs to be observed so that this failure does not go untested.

There is also an important limit: criteria do not replace human judgment in ambiguous, sensitive cases or those with relevant operational consequences. In these cases, the evaluation can indicate risk, organize evidence, and block automatic changes, but the decision may still require human review.

How to Choose Verifiers to Prevent Failure Recurrence

After the criterion comes the verifier. A verifier is the mechanism used to decide if the case passed or failed.

Not every verifier needs to be sophisticated. In some cases, a simple automatic check suffices: the answer contains mandatory information, does not contain a prohibited expression, preserves a category, or returns an expected format. In others, evaluation requires comparison with a reference base, human review, or execution in the environment.

The choice depends on the type of failure you want to prevent:

if the failure is structural, like missing a required field, an automatic check may be enough;
if the failure involves context interpretation, comparison with an expected answer may work better;
if the failure involves semantic quality, human review may be necessary;
if the failure involves action in the product, the verifier must look at the effect in the environment, not just the AI message.

This last distinction is decisive. In a feature that updates registration, opens a ticket, changes status, or queries a policy, the phrase "done" does not prove something was done. The verifier must observe the expected result when this is possible and safe.

Anthropic describes evaluations with inputs, success criteria, and verifiers, including the possibility of multiple attempts. The practical application here is not to confuse good appearance with good result. A well-written answer can fail the criterion. A less elegant answer can safely complete the task.

To deepen the prior collection before this conversion, it is worth connecting the process to the article on how to collect useful feedback on intelligent features. Without context captured at the source, evaluation is fragile from the start.

How to Define Acceptance Limits Without Hiding Regressions

An evaluation case needs an acceptance limit. Without a limit, the team just accumulates examples and opinions. With a limit, the evaluation guides a decision: publish, adjust, investigate, or stop the change.

The caution is not to let the average hide a relevant problem. Google SRE explains, in the context of monitoring, that averages can conceal problematic behaviors and that different views serve different audiences. The same caution applies to AI evaluations: an acceptable aggregate rate can hide failures in a type of input, user profile, plan, language, channel, or flow.

Therefore, acceptance limits should consider segments when there is a relevant difference. It is not necessary to create infinite slices. It is necessary to choose the slices that change the risk.

Some useful cuts can be:

task type: query, recommendation, classification, execution;
input complexity: simple request, ambiguous request, request with exception;
available context: with complete data, with partial data, without sufficient context;
operational profile: new user, recurring user, internal team, support;
risk category: low rework, user decision, possible trust breach.

Microsoft recommends observing metrics and segments during experiments to identify regressions and avoid premature interpretations. This recommendation does not turn evaluation into an experiment but reinforces a useful principle: decisions made solely by aggregate view may be premature.

The acceptance limit also needs to be defined before evaluation. If the team adjusts the limit after seeing the result, the case stops protecting the decision and starts justifying a preference.

Fictional Example: From Raw Feedback to Evaluation Case

Fictional example created only to demonstrate the method.

Raw feedback received by support: "The AI suggested a reimbursement policy that does not apply to my plan."

This report alone is not yet an evaluation case. It needs to be converted.

The real task: the user wanted to know if they could request reimbursement for a purchase made through a specific channel.

The necessary context: the user was on a specific plan with a different reimbursement rule. The plan was available in the product context. The general policy existed, but there was an exception applicable to that plan.

The case input: a user question about reimbursement, formulated naturally, without explicitly mentioning the plan.

The expected result: the AI must consult or use the plan context before answering. If the plan has a specific rule, the answer must inform the applicable rule. If the context is absent or contradictory, the AI must ask for clarification or guide checking in the appropriate channel, according to product design.

The success criterion: the answer cannot recommend the general policy when a specific rule is available in the plan context. It must make the applicable condition clear or ask for clarification when the necessary information is not available.

The verifier: comparison with an expected base of policies by plan, combined with human review for cases where the answer requests clarification. If the feature executes an action, such as opening a request, the verifier must check whether the request was created in the correct category or blocked when the rule did not allow progress.

The acceptance limit: the case should not be evaluated only in aggregate. The team must look separately at questions about the general policy and questions where there is a plan exception. If the failure appears precisely in the exceptions, the overall average should not authorize the change without investigation.

The preventive action: if the case fails, the next change should not be published as is. Alternatives may include adjusting the context delivered to the AI, reviewing the instruction, improving retrieval of the correct policy, expanding evaluation cases for other plans, or maintaining human review in the affected flow.

Nothing in this example measures a real result. It shows an operational hypothesis to evaluate: whether the AI respects a specific rule when the user's context changes the expected answer.

Feedback Screening Checklist for Evaluation

Use this checklist when reviewing examples from support, human review, research, analytics, or internal use:

Does the feedback describe a real task? Record what the user was trying to do, not just the opinion about the answer.
Is there enough context to reproduce the situation? Include plan, permissions, available data, relevant history, or restrictions that change the expected answer.
Is there a verifiable expectation? Replace generic adjectives with an observable result: respond, refuse, ask for clarification, preserve a rule, or execute an action.
Does the failure represent recurring or relevant risk? Prioritize cases that may repeat, affect user decision, generate rework, or compromise product trust.
Does the verifier measure the result, not just the appearance of the answer? Check if the task was completed in the environment or if the output respected the defined criterion.
Does the acceptance limit protect sensitive segments? Assess whether the overall average may hide problems in groups, input types, or specific flows.
Does the case guide a concrete decision? Define before evaluation what happens if the case fails: block publication, adjust context, review instruction, expand test data, or maintain human review.

This checklist does not turn all feedback into evaluation. It helps decide which reports have sufficient context, expectation, and risk to become evaluable cases.

How to Review the Set of Cases Before the Next Change

Evaluation cases age. The product changes, the context changes, user behavior changes, and some failures stop representing the main risk. Therefore, case set review should happen before relevant changes, not only after an incident.

The review can start with four questions:

Do the cases still represent real tasks of the current product?
Are there recurring failures in support that have not yet become cases?
Does any case test only the appearance of the answer without verifying the result?
Do acceptance limits still protect the segments where risk is higher?

This point connects to the guide on how to turn a failure into an improvement hypothesis. The failure helps formulate a hypothesis. The evaluation case helps prevent the hypothesis from being published without minimum protection against regression.

It is also worth maintaining a link with the core of learning cycles in AI products, because evaluation is not a dead archive. It is an operational learning mechanism: it transforms use, error, and review into criteria that improve the next decision.

In the next review, progress depends on separating available feedback and converting only examples with clear context, expectation, and risk into AI evaluation cases. The team needs to define success criteria, verifiers, and acceptance limits before using these cases to decide a change.

If you want to discuss this decision in your company context, talk to dooop.

Sources

NEXT DECISION

Discussing Application in Your Company

Conversation about the software company context

Content by dooop. Registration allows linking this topic to the reader's journey and tracking interest in the subject.