The blocker is rarely the model. It is the absence of an evaluation discipline that can answer a
harder question: can this agent complete the right task, in the right context, with the right
controls, consistently?
⚠️
Leader Perspective: The evaluation gap is not a QA problem — it is a governance
risk. Without structured rubrics and release gates, agentic AI becomes a liability that finance,
legal, and the board will eventually call back.
Why AI Agents Need a Different Testing Strategy
Traditional software testing assumes a known input, a fixed expected output, and repeatable behaviour.
AI agents do not work that way. Outputs vary. Retrieval can return different chunks. Tool calls may
take different paths. The reasoning trace shifts from one run to the next.
Evaluating an agent requires checking whether it understood the user's intent, retrieved the right
context, called the right tool, avoided hallucination, followed enterprise policy, and escalated when
its confidence was low. No single generic metric covers all of this.
Existing Evaluation Frameworks Are Necessary, Not Sufficient
A strong ecosystem of evaluation tools already exists.
DeepEval,
Ragas, Promptfoo, LangSmith, Braintrust, TruLens, Phoenix, and OpenAI Evals each give teams real
leverage on prompts, RAG pipelines, model outputs, hallucination, retrieval quality, tool calls, traces,
and regression behaviour. They are essential building blocks.
But a customer-support agent, a banking-compliance agent, a test-case generation agent, and a Playwright
automation agent may all share an LLM core while having entirely different definitions of "good."
Generic accuracy and faithfulness scores cannot decide whether a generated test suite is release-ready.
The evaluation tool provides the engine; the organisation must define the quality model.
The Custom Evaluation Blueprint
A practical custom evaluation framework can be built in seven steps:
1
Define the agent's mission
Write down what the agent must do, what it must never do, and how much autonomy is allowed
before a human is required. This becomes the evaluation contract.
2
Build task-level evaluation datasets
Cover normal flows, edge cases, negative scenarios, ambiguous prompts, high-risk domain cases,
and historical production issues.
3
Create domain-specific rubrics
Score domain relevance, business accuracy, retrieval correctness, reasoning, tool correctness,
hallucination control, compliance, clarity, and escalation behaviour.
4
Apply weighted scorecards
A formatting slip is low severity; a wrong business recommendation is critical; a wrong tool
call may block release. Weight accordingly.
5
Combine automated evaluation with human calibration
Automated evaluators give scale; expert reviewers calibrate the rubric over time to account
for edge cases no automated scorer anticipated.
6
Run regression evaluation continuously
Re-score whenever the model, prompt, RAG corpus, tool definition, workflow, or enterprise
policy changes.
7
Convert scores into release gates
Pass · Conditional Pass · Human Review Required · Block — each gate tied to a clear
business risk threshold.
Custom Metrics Based on Business Context
Generic LLM benchmarks measure model capability in isolation. Enterprise AI agents operate in a
business context — with specific user personas, data governance requirements, integration constraints,
and financial consequences of failure. The metrics must reflect that context. Below is a framework
for selecting and weighting evaluation dimensions by deployment domain.
Insights
Metrics derive from business risk, not model architecture
Map each agent action to a business outcome before writing a single metric.
Ask: what is the cost of a wrong answer here — dollars, compliance, customer trust?
Let cost-of-failure drive the weight. High cost = high weight = tighter gate threshold.
Revisit weights every quarter as the business context and regulatory landscape shift.
Metric Clusters by Enterprise Domain
Each domain cluster below contains the metrics that carry the most signal for that type of agent.
Select the cluster that matches your deployment, then tune weights using your organisation's risk
tolerance and regulatory posture.
Customer Support & CX Agents
Contact Centre · Chatbot · Voice AI
Intent resolution accuracy
Tone & brand voice compliance
Escalation precision (TP / FP rate)
Policy adherence per interaction
First-contact resolution rate
Hallucination rate on product facts
Compliance & Legal Agents
Banking · Insurance · Healthcare
Regulatory rule coverage (%)
False negative rate on risk flags
Citation accuracy to source documents
Auditability of reasoning trace
Data residency & PII non-disclosure
Structured output schema conformance
Test-Case Generation Agents
QA · Test Engineering · Delivery
Acceptance-criteria coverage (%)
Scenario diversity score
Defect-pattern recall from history
Edge & negative case density
Automation feasibility rating
Business rule fidelity
Playwright / UI Test Agents
Test Automation · MCP · CI/CD
Locator stability score (brittle index)
Page Object Model compliance
Wait strategy quality (no sleep abuse)
Assertion business-outcome alignment
MCP tool-call correctness rate
Cross-browser flakiness rate
Code Gen & DevOps Agents
Engineering · Platform · Security
OWASP vulnerability introduction rate
Coding standard conformance
Test coverage of generated code
Dependency hygiene score
Idiomatic pattern adherence
Build success rate on first run
Analytics & Insight Agents
BI · Reporting · Decision Support
Numerical accuracy vs. source data
Confidence interval reporting rate
Unsupported claim rate (hallucination)
Metric definition consistency
Time-period disambiguation score
Drill-down traceability to raw data
Weighted Scorecard: Enterprise AI Agent Release Template
The table below shows how to structure a weighted scorecard across core evaluation dimensions.
Adjust weights to match your domain cluster above and your organisation's risk posture.
Right tool called with right parameters at right time
15%
Critical
Retrieval Correctness
Retrieved context relevant and sufficient for task
15%
High
Compliance & Policy
Adherence to regulatory and enterprise policies
10%
High
Escalation Behaviour
Agent escalates when confidence is low or scope exceeded
8%
Medium
Reasoning Quality
Logic chain is coherent and traceable
4%
Medium
Domain Relevance
Response scoped appropriately to domain context
2%
Low
Output Clarity
Response understandable to intended user persona
1%
Low
Business-Context Metric Matrix
The following matrix maps enterprise agent types to their primary KPI, the hardest-to-catch failure
mode, and the metric that most reliably surfaces it.
Agent Type
Primary Business KPI
Worst Failure Mode
Most Diagnostic Metric
Gate Threshold Guidance
Customer Support
First-contact resolution rate
Confident wrong answer on returns / billing
Policy-fact hallucination rate < 0.5%
Block if hallucination rate > 1%
Compliance / Legal
Regulatory breach rate = 0
Missed risk flag (false negative)
Risk-flag recall ≥ 99%
Block if recall < 98%
Test-Case Generation
Defect escape rate post-release
Generic cases, no edge/negative coverage
Acceptance-criteria coverage ≥ 90%
Human review if AC coverage < 85%
UI / Playwright Automation
Pipeline flakiness rate
Brittle locators causing false failures
Locator stability score ≥ 85/100
Block if sleep-based waits > 5%
Code / DevOps Generation
MTTR on AI-introduced defects
Security vulnerability (OWASP Top 10)
SAST critical finding rate = 0
Block on any critical SAST finding
Analytics / BI
Decision accuracy from AI insights
Incorrect aggregate with false confidence
Numerical accuracy vs. source ≥ 99.5%
Block if unsupported claims > 0.5%
Back-Office / Process
Straight-through processing rate
Wrong data written to ERP / CRM
Data integrity score post-action ≥ 99.9%
Block if data mutation errors > 0.1%
Release Gates — Translating Scores into Decisions
Every weighted scorecard must terminate in a binary business decision. The four-gate model below maps
score ranges to actions and assigns responsibility for each outcome.
✅
PASS
Score ≥ 90% No critical failures
🔵
CONDITIONAL
Score 80–89% Non-critical gaps only
👁
HUMAN REVIEW
Score 65–79% Any high-severity gap
🚫
BLOCK
Score < 65% Any critical failure
💡
Threshold calibration: Starting thresholds should be conservative (block at <80%).
Track false-block rates over 90 days and adjust. A gate that blocks too often erodes trust; one
that lets failures through erodes safety. Calibration is itself a continuous process.
"Plausible" outputs failed a domain rubric before they reached a human reviewer.
In this engagement, a RAG-based test-case generation agent produced outputs that looked complete
at first read. Deeper review surfaced familiar issues: scenarios were too generic, business rules
were missed, and edge and negative cases were thin. Standard RAG metrics confirmed the answers
were grounded, yet they could not tell us whether the suite covered every acceptance criterion,
whether historical defect patterns were reflected, or whether the cases were genuinely automatable.
A custom rubric covering acceptance-criteria coverage, scenario diversity, defect relevance, and
automation feasibility decided release readiness.
RAG metrics are necessary, but QA-domain evaluation is what makes the output trustworthy.
Case Study 02 · Playwright Automation via MCP
Code that compiled and passed lint still failed a production-readiness rubric on 4 of 5 dimensions.
The agent generated Playwright automation through Playwright MCP. The code compiled and often
passed lint. But locators were brittle, Page Object Model conventions were inconsistent, wait
handling leaned on sleeps, and assertions validated syntax rather than business outcomes.
A rubric scoring locator quality, POM compliance, wait strategy, assertion quality, and MCP
tool-call correctness exposed the gap — before any code reached the CI pipeline.
Code that compiles is not code that ships. Production-readiness is a rubric, not a build status.
What This Means for QA Teams
Agentic AI is reshaping the QA mandate. Test execution is no longer the centre of gravity;
evaluation design is. The QA function becomes the quality gatekeeper for enterprise
AI agents — owning rubrics, scorecards, regression datasets, and human-in-the-loop calibration.
The skills that compound from here are domain-aware evaluation design, structured human review, and
translating business risk into release gates that engineering and the business both trust.
Strategic Imperatives for Leaders
Build the evaluation discipline before the agent scales
Fund a dedicated evaluation engineering function alongside agent development — not after.
Require every AI agent to ship with a scorecard, a dataset, and a documented release gate before production.
Mandate human-in-the-loop review for any agent whose failure has financial, regulatory, or reputational consequences.
Treat the rubric as a living artefact: review and version-control it alongside the agent prompt and model.
Connect evaluation outcomes directly to board-level risk reporting on AI governance.
Closing
The future of testing is not just more automation. It is trusted AI-agent evaluation: a clear
mission, a custom rubric tuned to business context, a weighted scorecard, calibrated human review,
and a release gate that reflects business risk.
Teams that build this discipline now will be the ones who put agents into production with
confidence — and who earn the trust of the board, the regulator, and the customer.
References
Gartner: Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 —
gartner.com
Over 27 years of experience in software testing, test automation, performance engineering,
DevOps, and AI-led quality engineering. He has trained more than 50,000 QA professionals and
works with enterprises to implement modern testing practices across automation, Generative AI,
and agentic quality engineering. An active speaker and community contributor in the software
testing ecosystem, with a strong focus on helping QA professionals transition into
AI-augmented engineering roles.
