# AI Agent Reliability Is Not One Score: A Practical Scorecard

Source: https://converra.ai/blog/ai-agent-reliability-metrics
Published: 2026-09-02. Updated: 2026-09-02. Sources reviewed: 2026-09-02.
Publisher: Converra (https://converra.ai), the autonomous improvement loop for production AI agents.

AI agent reliability is not one score. It is a scorecard of outcome, behavior, safety, latency, and recovery, each with a declared denominator and release gate. A strong average cannot rescue a safety failure, missing cohort, or agent that completes the task too slowly.

The useful question is not, ‘What is our agent's reliability score?’ It is, ‘Which evidence would let us ship, stop, investigate, or roll back this version?’

> Keep outcome, behavior, safety, latency, and recovery visible. Define every denominator before collecting results, preserve missing data as missing, and apply hard gates before any composite score.

## One reliability score hides the failure you need to see

An agent can improve average task success while creating a rare safety failure. It can answer correctly but violate a routing rule, call the wrong tool, or take long enough that the user abandons the workflow. Combining those outcomes too early makes a clean dashboard and a weak decision.

Treat reliability as a vector: R = [outcome, behavior, safety, latency, recovery]. Add tool and evaluator metrics when they affect the workflow. NIST's 2026 post-deployment monitoring report separates functionality, operations, human factors, security, compliance, and broader impacts, while noting that monitoring methods remain use-case-specific and immature. There is no official universal agent score waiting to be adopted.

## The AI agent reliability metrics that matter

Start with the failure costs and decisions for one workflow. The formulas below are a practical operator template, not an industry standard. A metric is useful only when its event, denominator, cohort, window, version, and gate are explicit.

*AI agent reliability formula scorecard*

| Metric | Formula | Required denominator | Example gate |
| --- | --- | --- | --- |
| Outcome success | successful eligible attempts / eligible attempts | Attempts with a defined, observable outcome | Minimum rate or positive paired lift |
| Behavior compliance | compliant opportunities / evaluable opportunities | Cases where the named rule applied and evidence was captured | Per-rule floor; critical rules may block |
| Safety incident rate | confirmed incidents / safety-evaluable attempts | Attempts reviewed by a validated detector or policy process | Maximum rate; critical incident may block |
| Tool success | correct completed actions / required tool actions | Tool actions the workflow actually required | Minimum rate by tool and failure type |
| End-to-end latency | p50, p95, and p99 eligible-attempt duration | Attempts with consistent start and terminal markers | Maximum percentile by workflow |
| Recovery rate | successful recoveries / recoverable failure episodes | Episodes classified as recoverable before the outcome | Minimum rate plus retry-cost ceiling |
| Regression rate | regressed protected scenarios / protected scenarios evaluated | Versioned scenarios with valid baseline and challenger results | Zero critical regressions; bounded standard tolerance |
| Evaluator precision | TP / (TP + FP) | All evaluator alerts in a reviewed cohort | Set from false-review cost |
| Evaluator recall | TP / (TP + FN) | All real failures in a reviewed cohort | Set from escape cost |

## Define the denominator before collecting the numerator

Most reliability arguments are denominator arguments in disguise. Suppose an agent receives 1,000 conversations. Of those, 120 are spam, 80 lose an upstream tool before the agent can act, and 100 have no observable business outcome. The agent succeeds in 640 of the 700 attempts your policy calls eligible and observable.

Reporting 640 successes as 64 percent of all received traffic or 91.4 percent of eligible observable attempts can both be defensible if the policy is declared. Reporting 100 percent by retaining only the 640 observed successes cannot. Every exclusion needs a reason and a count.

*The same numerator under three denominator policies*

| Reported rate | Calculation | What it means |
| --- | --- | --- |
| 64 percent | 640 / 1,000 received | Success across all received conversations, including excluded and unknown cases |
| 91.4 percent | 640 / 700 eligible and observable | Success under a declared eligibility and observability policy |
| 100 percent | 640 / 640 retained successes | Invalid survivorship framing that silently drops every non-success |

## Missing data is a result, not a zero

A missing metric must stay missing. Turning it into zero can manufacture either a pass or a failure, depending on whether lower or higher is better. Use explicit states in the scorecard and block any claim that requires evidence you do not have.

The labels below are an operator convention, not a formal standard. What matters is that the system, reviewer, and decision rule can distinguish no event from no measurement.

*A practical missing-data contract*

| State | Use it when | Do not conclude |
| --- | --- | --- |
| UNOBSERVED | The metric or required cohort was not captured | That the value was zero or the gate passed |
| UNAVAILABLE | The source could not be accessed or retained data cannot compute it | That missing access is evidence about performance |
| CAPTURED_ZERO | The denominator was observed and the numerator was truly zero | That the true rate is universally zero outside this cohort |

## Measure the evaluator separately from the agent

Precision and recall describe the measurement instrument, not the agent. Precision asks how often a flagged failure is real. Recall asks how many real failures the evaluator catches. A judge with weak recall can make an unreliable agent look clean; weak precision can flood reviewers with false alarms.

Google's classification guidance shows why accuracy can be misleading on imbalanced data and why metric choice should follow the cost of false positives and false negatives. Calibrate each material detector against a reviewed cohort, preserve its threshold and version, and report unresolved denominator-zero cases as missing rather than silently assigning a perfect value.

## Compare versions on matched evidence

Absolute scores show how one version performed on one cohort. A release decision usually asks a different question: did the challenger improve the same cases? Run the baseline and challenger against the same persona, scenario, starting state, and tool conditions, then calculate d_i = score(challenger_i) − score(baseline_i) for each valid pair.

Report the valid-pair count, mean paired lift, wins, ties, losses, uncertainty, and protected-scenario regressions. Do not subtract averages from unrelated test mixes. Converra's testing workflow uses the same personas and scenarios for baseline-versus-challenger comparison; the companion sample-size guide explains why pair count, variance, effect size, and coverage belong in one plan.

## Infrastructure reliability is not behavior reliability

OpenTelemetry recommends operation-duration metrics with contextual error types so teams can derive throughput and error rates. Its current GenAI semantic conventions also define operation-duration and token-usage metrics. Those are valuable operational signals, but low latency and zero technical errors do not establish that the agent achieved the user's goal or followed the right rule.

Keep operational telemetry beside behavioral evidence. Neither layer replaces the other.

*Two layers of AI agent reliability evidence*

| Operational telemetry | Behavioral evidence |
| --- | --- |
| Request and model latency | User goal achieved |
| Exceptions and error types | Required policy followed |
| Token and resource usage | Correct tool and arguments used |
| Retries and timeouts | Recovery preserved the intended outcome |
| Availability and throughput | Handoff retained context and authority |

## Recovery deserves its own metric

A tool timeout, ambiguous request, missing permission, or downstream rejection does not have to become a failed user outcome. Define recoverable episodes before looking at their outcomes. Then record detection, bounded retry, clarification or escalation, final outcome, added latency and cost, and any new safety issue.

Recovery rate = successful recoveries / recoverable failure episodes. Pair it with a retry-cost ceiling and a loop detector. An agent that eventually succeeds after ten uncontrolled retries did not recover reliably.

## Use composites for triage, never to erase a gate

If an executive view needs one composite, define composite = sum(weight_k × normalized_metric_k). Publish who owns the weights, why they exist, how lower-is-better metrics are normalized, and which versions and cohorts they apply to. Keep every component visible next to the total.

Apply safety, policy, and other hard gates before the composite. Never impute a favorable value for an unobserved dimension. Strong latency, sentiment, or task completion must not average away a critical failure.

## A scorecard selects a candidate; production closes the change

Pre-deployment metrics can select and protect a candidate under tested conditions. After deployment, measure the same defined production outcome on fresh traffic with preserved version identity. Account for prompt, model, routing, traffic, tool, and evaluator changes that can confound the comparison.

The production result may be verified, not fixed, or confounded. A green simulation suite or approved pull request proves only that the candidate cleared those gates. It does not prove that real behavior improved.

*Copyable AI agent release scorecard*

| Metric | Definition and cohort | Evidence | Gate | Decision |
| --- | --- | --- | --- | --- |
| Task success | Eligible attempts for the target workflow | Current, baseline, valid pairs | Minimum rate and positive lift | Pass / fail / inconclusive |
| Policy compliance | Cases where the named rule applies | Rate, evaluator precision and recall | Per-rule floor | Pass / block / review |
| Critical safety | Safety-evaluable attempts | Incident count and denominator | Declared hard gate | Pass / block |
| Latency | End-to-end eligible attempts | p50, p95, p99 | Workflow-specific ceiling | Pass / fail |
| Recovery | Preclassified recoverable episodes | Rate, retry cost, loop count | Minimum rate and cost ceiling | Pass / fail / investigate |

## Sources

- Challenges to the monitoring of deployed AI systems (NIST): https://www.nist.gov/publications/challenges-monitoring-deployed-ai-systems-center-ai-standards-and-innovation
- Accuracy, recall, precision, and related metrics (Google for Developers): https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall
- Recording errors (OpenTelemetry): https://opentelemetry.io/docs/specs/semconv/general/recording-errors/
- Semantic conventions for generative AI metrics (OpenTelemetry): https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-metrics.md
- AI agent testing for production teams (Converra): https://converra.ai/ai-agent-testing

## Frequently asked questions

### What is AI agent reliability?

AI agent reliability is the repeated ability to achieve the intended outcome while following required behavior, avoiding unacceptable failures, meeting operational limits, and recovering safely within a defined workflow, cohort, environment, and time window.

### Which AI agent reliability metrics should I track?

Start with outcome success, behavior compliance, safety incidents, latency, and recovery, then add tool execution and evaluator precision or recall when they materially affect the workflow.

### What is a good AI agent reliability score?

There is no universal good AI agent reliability score; thresholds should come from the workflow, failure severity, baseline, user expectation, and measurement quality.

### How should I combine AI agent reliability metrics?

Keep component metrics visible and apply hard gates first; if you need a composite, publish its weights, normalization, denominators, versions, and missing-data policy.

### How should I treat missing AI agent metric data?

Keep missing metric data explicit: use UNOBSERVED when it was not captured, UNAVAILABLE when the source or computation cannot be accessed, and CAPTURED_ZERO only when the denominator was observed and the numerator was truly zero.

### Do low error rates prove an AI agent is reliable?

No. Low infrastructure error rates show that operations completed technically, but they do not prove that the agent achieved the correct outcome, followed policy, chose the right tool, or recovered safely.

## Related reading

- [How many simulated conversations do you need?](https://converra.ai/blog/how-many-simulated-conversations-ai-agent-testing): Plan matched-pair volume, rare-event coverage, and a defensible stop rule.
- [Your LLM judge is not ground truth](https://converra.ai/blog/llm-as-a-judge-pitfalls): Calibrate the evaluator before treating its labels as reliability evidence.
- [Agent Improvement Rate](https://converra.ai/agent-improvement-rate): See how verified production changes can be counted without collapsing component metrics.

Markdown edition of https://converra.ai/blog/ai-agent-reliability-metrics. The HTML page is canonical; this file is for agents and answer engines.