dooopSoftware · Product · 5 min

How to Version Prompts Within a Product

Connect prompt versions to context, evaluation, and feedback. Record the configuration that allows reconstructing changes in AI behavior.

Published on September 6, 2026

MAIN THESIS

A history of texts alone does not explain why behavior changed.

The version should allow comparing the change and recovering a known condition.

A prompt version needs to enable the team to discover what changed, assess the effect, and revert to a known configuration when necessary. Saving the text in a history is part of this work. The control unit should also include the information and conditions that influence execution: model, context, available tools, and response criteria. Without this linkage, two executions with the same prompt may be compared under different conditions.

Define What Belongs to the Version

Start with what the team needs to reconstruct to investigate a response. The instruction text may be the same, but the context retrieval policy may have changed. A tool may return a different format. A model may have been replaced. The record should allow locating these differences.

A proposed version unit includes:

Instruction identifier and corresponding content.
Model or model configuration used.
Context selection rule and its references.
Tools and input/output contracts.
Evaluation criteria and the set of cases used.
Responsible person, reason, and actual date of the change.

It is not necessary to duplicate all data in each record. Stable references for configurations and artifacts can fulfill this function. The important thing is that the team can recover the relevant condition, not just a version name without verifiable content.

Anthropic on context engineering includes instructions, tools, external data, and history among the information available to the model during inference. The application to versioning is to treat the instruction as part of an execution configuration.

Write the Reason Before Changing the Instruction

A prompt change should respond to an observed problem. "Improve the response" is too broad to guide an evaluation. Describe the behavior that needs to change and what should remain.

For example: the functionality summarizes requests but omits explicit constraints. The hypothesis may be that an instruction to identify and preserve request conditions reduces this omission. The evaluation needs to examine this behavior and verify if the summary remains useful and faithful to the content.

Microsoft ExP describes experimentation as hypothesis validation, impact measurement, and iteration in development. The proposal here is to maintain this link between reason, change, and decision. Not every small adjustment requires a controlled experiment but needs a form of verification proportional to the expected effect.

Compare Versions with Cases That Reveal the Change

Use cases linked to the problem that motivated the change and cases that represent already accepted behaviors. Evaluating only the examples that led to the change may fail to show regressions in other tasks.

Record the conditions of the comparison. If the team simultaneously changed prompt, model, and document retrieval, the evaluation may compare the new set with the old one. It should not attribute the entire effect to the prompt without other evidence.

Define beforehand what counts as improvement, regression, or inconclusive result. When the output allows variation, evaluate the relevant behavior instead of requiring an identical phrase. Preserve the results and the justification for the decision so the next review does not depend on the memory of those involved.

Fictional Example: Preserving Conditions in a Summary

Consider a product that summarizes change requests in projects. The team identifies examples where the generated text preserves the main request but fails to mention that execution depends on approval from another area.

A new version instructs the system to present conditions and pending items separately. The comparison includes requests with explicit conditions, requests without conditions, and ambiguous messages. The team observes whether the change preserves real constraints without inventing new dependencies.

If the version improves known cases, it will still be necessary to examine permitted use before expanding exposure. The example does not claim the change will solve the problem. It shows how to link an instruction adjustment to cases, criteria, and a release decision.

Prepare Reversion Without Ignoring Effects Already Produced

Reverting to the previous instruction version may stop new behavior but does not automatically undo actions already executed. Differentiate reverting configuration, correcting data, and communicating with affected users.

Before release, record who can restrict use, which version will be restored, and which signals justify this decision. Also check if the reversion remains compatible with current tools and context. An old version may depend on a contract that has already changed.

For functionalities that only produce drafts, reversion can be simple. For functionalities that trigger systems, the team needs to consider the task state and external effects. This care should be part of the product decision, involving those who know the operation.

Use a Short Release Record

A useful record allows answering: why did we change, what did we evaluate, which limits remain, and how will we monitor? Include the decision made, its evidence, and the person responsible for reviewing subsequent signals.

Avoid accumulating versions without indicating which is active in each context. If different groups use different configurations, this distinction needs to appear in observation and problem investigation.

Before the next prompt adjustment, establish a recoverable version of the current configuration and choose cases that can demonstrate the effect of the change. The history will have value when it helps decide and reconstruct the product behavior.

If you want to discuss this decision in your company context, talk to dooop.

Sources

To continue this reading

How to compare models for a product task

NEXT DECISION

Discussing the application in the company

Discussion about the software company context

Content from dooop. Registration allows linking this topic to the reader's journey and tracking interest in the subject.