dooopSoftware · Organization · 12 min

How to Train Developers to Use AI

AI training for developers should measure practice in real tasks, including hypothesis, validation, risk review, and reusable learning.

Published on September 6, 2026Updated on September 6, 2026

CENTRAL THESIS

Completing a course does not prove autonomy. Evidence appears when the person decides, validates, and records in a real task.

Evaluate AI by demonstrated practice. Use small tasks to separate fluency with the tool from technical judgment.

After an AI course, the most important question for technical leadership is simple: can this person use the tool in a real task without abandoning judgment, validation, and communication? A person trained to work with artificial intelligence needs to explain the task, formulate a hypothesis for AI use, produce a verifiable output, review risks, and record reusable learning. To declare autonomy, start with a short evaluation involving real, low-criticality work.

Why Completing a Course Does Not Prove Ability to Work with AI

A developer can complete a course, install a tool in the editor, generate code in seconds, and still not be ready to use AI autonomously in the engineering workflow.

This is not a contradiction. It is the difference between knowing a tool and demonstrating work capability.

The presence of AI changes parts of the process but does not remove the questions that have always supported good engineering: what problem is being solved, what constraints matter, what behavior must remain, what risk was introduced, and how the team will verify the change.

When leadership measures training only by hours of instruction, it risks approving a performative skill: the person knows how to ask AI for something but does not know how to evaluate if the response is useful. The result may seem fluid in a demonstration and fragile in a real codebase.

The DORA 2025 report describes AI as an amplifier of existing strengths and weaknesses in the organization. This perspective is useful for leadership because it shifts the question. The point is not whether the tool impresses. It is whether the work system allows the tool to be used with judgment.

This care also avoids another mistake: treating training as an isolated event. DORA relates learning culture to software delivery performance and proposes treating learning as an organizational investment. This does not mean every use of AI automatically generates learning. It means the team needs to design mechanisms to transform practice into repertoire.

If the company is still organizing its broader AI direction, it is worth connecting this article to the guide on how to create an AI strategy connected to business. But here the decision is smaller and more operational: how to know if a developer can work better with AI on a concrete task.

Which Real Tasks Serve to Evaluate Developers Using AI

A good practical AI evaluation does not need to be large. In fact, the larger and more ambiguous the task, the harder it is to distinguish competence, luck, domain familiarity, and external help.

The best starting point is to choose small, frequent, and relevant tasks for the team. They should represent work the person will actually encounter after training.

Suitable examples:

refactor a code snippet with known behavior;
write or adjust tests for an existing rule;
investigate a simple bug with a defined scope;
improve technical documentation used by the team;
compare two implementation alternatives for a low-criticality change;
review an AI-suggested change before opening a change request.

The criterion is not to choose a task that is too easy. It is to choose a task where technical judgment can be observed.

Tasks that are too artificial reward demonstration performance. A challenge created only to generate a nice prompt response may say little about the team’s daily work. On the other hand, critical tasks with sensitive data or high architectural impact are not good environments to evaluate initial autonomy.

A practical rule: the task must have enough context to require decision-making but low enough risk to allow observation and correction without undue pressure.

If the organization already has an AI roadmap, developer evaluation should not become a small version of the roadmap. It should select a concrete slice of work and verify if the person can use AI without outsourcing their own reasoning.

How to Observe Practice Before Measuring Productivity

Before talking about productivity, observe how the person works.

The most useful question is not “how long did it take?”. It is “what did they do to reach a reliable decision?”. In AI tasks, speed can hide rework, uncritical acceptance, or changes that seem correct outside the system context.

Technical leadership or an experienced peer should observe evidence such as:

clarity of the objective before activating AI;
quality of the context provided to the tool;
decomposition of the task into verifiable parts;
ability to question the received response;
identification of risks, side effects, and dependencies;
explicit decision about what to accept, adjust, or discard;
explanation of the result to another team member.

This type of observation protects the team from a common confusion: thinking a coherent response is a correct response. In software, textual coherence is not enough. The change must respect business rules, architecture, tests, security, maintenance, and team agreements.

Microsoft describes its ExP platform as a way to incorporate experimentation into the development cycle, validate hypotheses, measure impact, and iterate products. The parallel here is limited but useful: working with AI also requires hypothesis and verification. The hypothesis can be simple, such as “AI can help identify duplication in this function without changing expected behavior.” After that, the person needs to test if the hypothesis holds.

Do not call any attempt an experiment. An experiment needs a testable hypothesis, a way to check, and a decision after evidence.

Minimal Evidence of AI Training in a Real Task

The checklist below serves for a short practical AI training session using a real, low-criticality task. Observation can be done by technical leadership or an experienced peer. Each item requires concrete evidence, not a general impression.

Understanding the Task

The person explains the objective, expected behavior, and constraints before activating AI.

The passing criterion is simple: they can differentiate what needs to be solved from what would be only an optional improvement. If the person starts asking for a solution without understanding the problem, AI becomes a shortcut for a decision that has not yet been made.

Formulating the Context

The person provides AI with sufficient information about code, rule, test, dependencies, and change boundaries.

A good request reduces ambiguity without exposing unnecessary information. This includes knowing how to trim context. Dumping a large volume of information into the tool is not necessarily better than selecting the right snippet, explaining the intention, and delimiting what should not change.

Using AI as Support, Not a Substitute for Judgment

The person compares the response with the system context and questions the tool’s assumptions.

The passing criterion is to point out at least one suggestion accepted, one adjusted, and one rejected. This distinction shows the person did not treat the response as automatic authority.

Technical Validation

The person runs or proposes tests, static review, code diff reading, or another verifiable form of checking.

The decision to proceed cannot depend only on the appearance of the response. If the person cannot say how they would verify the change, they have not yet demonstrated sufficient autonomy.

Communication of the Decision

The person records what changed, why, what risks remain, and what needs human review.

Another team member should be able to understand the change without redoing the entire investigation. This record does not need to be long. It needs to be clear enough to support review.

Reusable Learning

The person identifies a pattern, prompt, caution, or example that can help the team in similar tasks.

The learning needs to be specific. “Using AI to refactor” is too generic. “When refactoring duplicated rules, first ask for a list of preserved behaviors and only then suggestions for extraction” is more useful.

A practical criterion is to consider someone ready to use AI with more autonomy when they demonstrate sufficient evidence in these six points in at least two different real tasks. This does not make the person an expert nor authorize critical decisions without review. It only reduces the gap between formal training and observable capability.

Practical Evaluation of AI-Assisted Refactoring

Imagine an internal product team with a function that calculates access permissions on two different screens. The code has duplication, but the expected behavior is already known by the team. The example is fictitious and serves only to illustrate the evaluation.

The developer receives the task of proposing an AI-assisted refactoring. Before asking the tool for anything, they describe the objective: reduce duplication without changing existing permissions. They also list constraints: maintain the public signature used by other modules, preserve current tests, and do not alter the audit rule.

Next, they formulate a request with the relevant code snippet, existing tests, and the instruction not to propose behavior changes. AI suggests extracting a common function and also suggests simplifying a condition.

The evaluation does not approve the person because the code became smaller. It approves, if applicable, because they realize the proposed simplification would change an exceptional access case. They accept the extraction of the common function, adjust the name to reflect product language, and reject the condition simplification.

Then, they run or describe the necessary tests, read code differences, and record the decision: what was extracted, what behavior was preserved, which AI suggestion was rejected and why. Finally, they note reusable learning for the team: in permission rule refactorings, ask AI to enumerate preserved behaviors before suggesting code reduction.

Nothing in this example proves productivity gain. That effect would be a hypothesis to measure in another evaluation design. What the session demonstrates is more specific: the person used AI without abandoning understanding, validation, and technical communication.

How to Record Evidence for Future Evaluations

Individual evaluation loses strength when it remains only in the observer’s memory. To become team capability, it needs to feed internal examples, review agreements, and easily found documentation.

DORA evaluates documentation quality by attributes such as clarity, ease of location, and reliability, and recommends active creation and maintenance. For a team using AI, this suggests a simple discipline: record good usage examples, recurring errors, and precautions that actually appeared in work.

This documentation does not need to become a heavy manual. It can start as a small library of examples:

prompts that helped delimit expected behavior;
situations where AI suggested a dangerous change;
criteria used to accept or reject a response;
examples of well-written decision records;
types of tasks where the team still requires close supervision.

The record should keep useful prompts, justified rejections, and review criteria that appeared in observed tasks. When practice is documented clearly, the team gains shared repertoire. This relates to broader discussions about AI maturity, but here the focus remains operational: turning practical evaluation into learning material.

It is also worth separating hypothesis discovery from experiment execution. A person may discover that AI helps explain a legacy snippet. Then, the team may formulate the hypothesis that this use improves refactoring review. Only then would it make sense to measure impact more structurally.

When Not to Declare Someone Trained Yet

Not declaring someone trained yet is a quality decision, not a punishment.

Leadership should withhold autonomy when the person accepts responses without verification, cannot explain technical choices, uses AI to circumvent review, exposes sensitive context without criteria, does not record learning, or depends on the tool for tasks they should understand.

It is also prudent to limit autonomy when the person only demonstrates competence in very artificial tasks. If the evaluation does not resemble real work, it measures something else: tool familiarity, improvisation, or presentation skill.

There is also a scope limit. Good execution in a small task does not authorize autonomous use in critical architectural changes, decisions with sensitive data, or high-impact product changes. AI training should increase responsibility, not suspend review.

For leaders connecting AI to strategic decisions, reflection on leadership and augmented human helps keep human judgment in work design, not as a late correction.

The next step is to create a short practical evaluation with a real, low-criticality task, minimal evidence, and later peer review. If the person demonstrates understanding, context, judgment, validation, communication, and reusable learning in more than one situation, leadership has a better basis to expand autonomy. If not, the next step is not another lecture. It is supervised practice in a better-chosen task.

If you want to discuss this decision in your company’s context, talk to dooop.

Sources

To Continue This Reading

NEXT DECISION

Discuss Application in the Company

Conversation about the software company context

Content from dooop. Registration allows linking this topic to the reader’s journey and tracking interest in the subject.