How to Evaluate the Experience of an AI Consulting Firm: Cases, References, and the Team That Will Work
By Pedro Sonda, dooop
To evaluate the experience of an AI consulting firm, verify three layers: the cases it presents (with problem, approach, result, and limitations explained), client references you can speak with, and the individual experience of the people who will work on your project. Logos, years in the market, and number of projects say little. What matters is whether the consulting firm has already solved a problem similar to yours and if the people who did so will be on your team.
Updated in October 2026.
Transparency note: dooop is an AI consulting firm. The criteria in this article apply to any provider, including dooop itself.
Why AI Experience Is Difficult to Evaluate
Much of the market experience with generative AI is recent, and many initiatives have not gone beyond the pilot phase. The MIT NANDA project report concluded that 95% of corporate generative AI pilots did not generate measurable financial returns (The National CIO Review on MIT NANDA, 2025). McKinsey recorded that only 6% of organizations are high performers in AI (McKinsey, 2026).
Implication: "Having done AI projects" is not enough. The question is whether those projects generated results and if the consulting firm can explain why, including when they did not.
dooop Framework: The 3 Layers of Experience
| Layer | What to Check | Key Question | Warning Sign |
|---|---|---|---|
| 1. Cases | Problem, approach, result, limitations | "What went wrong and what did you change?" | Cases that only describe success |
| 2. References | Direct conversation with clients | "What would you do differently if hiring again?" | No references available |
| 3. Assigned Team | Experience of those who will work on your project | "Will the people who did this case be on my project?" | Seniors only in sales |
Layer 1: The Depth Scale of the Case
Not every case has the same value as evidence. Use this scale:
| Level | What the provider shows | Value as Evidence |
|---|---|---|
| 1 | Client logo | Low: confirms relationship, not result |
| 2 | Project description | Low: shows activity |
| 3 | Problem, approach, and deliverables | Medium: shows method |
| 4 | Measured result, with baseline | High: shows impact |
| 5 | Result, limitations, and lessons learned, confirmed by reference | Very high: shows maturity |
Confidentiality is legitimate and many clients do not authorize disclosure. In these cases, request an anonymized presentation at levels 3 to 5 in a meeting, without written material.
Layer 2: How to Talk with References
- Request at least two references from projects similar to yours in topic or size.
- Ask what changed in the company after the project, not just if they liked the consulting firm.
- Ask how the relationship was when problems arose.
- Ask if the team presented during sales was the same that executed the project.
Layer 3: How to Evaluate the Team
- Request names, roles, percentage dedication, and summarized resumes of those who will work on the project.
- Verify if the team combines business knowledge, change management, and AI, not just technology.
- Ask who replaces whom, and with what approval, if someone leaves.
- Review what these people publish: articles, talks, and research show how they think, although they do not replace cases.
Relevant Experience Is Not Just AI Experience
Since most AI value depends on people and processes, experience in organizational transformation, change management, and your sector counts as much as technical experience. A consulting firm with few years in AI but extensive experience in organizational change in your sector may be a better choice than a technical specialist without that background.
How dooop Presents Its Experience
dooop lists on its website organizations it has worked with, such as Visate, Ipiranga, Grupo Ultra, CDL, Sebrae, NL, and Grupo IMEX (dooop). By the criteria of this article, a client list is level 1 evidence. To evaluate dooop or any other consulting firm, request levels 3 to 5 and speak with references.
What to Do Now
- Before meetings: define what type of case would be comparable to yours (topic, sector, size).
- During meetings: request cases at levels 3 to 5 and the names of the team.
- Before deciding: talk with references and record answers in the same format for all candidates; see AI Consulting: What to Evaluate Before Hiring?.
To learn about the organizations dooop has worked with, see dooop.com.br/clientes.
Frequently Asked Questions
Can a new AI consulting firm be a good choice? It can be, if the team members have relevant experience in transformation, your sector, or the type of problem, and if the consulting firm is transparent about what it has done and what it has not yet done.
Do awards and rankings serve as evidence of experience? They serve as complementary signals. Check who organizes them, what criteria they use, and if participation is paid.
Also Read
About dooop
dooop is a consulting firm based in Caxias do Sul (RS) that prepares leaders and organizations to transform artificial intelligence into business capability, with strategy, governance, and training.