Evals, traces, and manual analysis can surface and sometimes diagnose failures. Turn-level attribution separates where the failure began, why it propagated, and what can safely change.
It may surface a bad result, but a defensible origin still needs turn, span, and trace evidence.
Production traces add real combinations and edge cases. Use both instead of treating either as complete on its own.
Step and span evals can narrow the search, while causal attribution separates the origin from downstream symptoms.
When evidence or ownership is missing, the result should remain partial, unresolved, or abstained rather than guessed.
A defensible diagnosis separates the observed failure from its category, fix type, causal owner, and evidence state. Treating those as one universal layer hides uncertainty and can point at the wrong fix.
The first turn and step where the conversation diverged, labeled with an observed failure type such as context loss, a wrong tool call, or a missed escalation.
Where the issue lives: prompt instructions, tool configuration, retrieval, orchestration, model limits, data quality, or unclear when evidence does not support a category.
How the evidence says to intervene: prompt, code, configuration, or data change — or unclear when no safe change is supported.
An evidence-backed chain connecting the failure origin to downstream effects and, when possible, the prompt, producer run, or orchestration event that owns the fix.
A diagnosis can remain partial, leave the owner unresolved, or abstain when required prompt, runtime, or causal evidence is missing.
Every turn is recorded - system prompts, user inputs, agent responses, tool calls, retrieved documents. Nothing is summarized or dropped.
Each agent turn is evaluated against intent alignment, context usage, relevance, and tool appropriateness. This identifies the exact turn where the conversation diverged from the expected path.
The failure origin is analyzed against the active instructions, context, tool calls, and runtime evidence. Category, fix type, and causal ownership remain separate, and unsupported ownership stays unresolved or abstained.
A fix specific to the identified layer is generated, tested in simulation against the failure scenario and similar cases, regression-checked, and deployed with production verification.
Each conversation turn is evaluated against the agent's goals and behavioral expectations. A grounded diagnosis separates the first failure origin and observed failure type from the likely category, fix type, downstream causal effects, and owner evidence. Those are related decisions, not one five-layer label.
Output, step, and trace evals can score behavior and sometimes surface likely causes. Turn-level attribution adds an evidence-backed failure origin, separates category from fix type, and traces causal effects to a resolved owner when the captured evidence supports one.
Yes. In multi-agent architectures, causal attribution can connect an upstream agent's output or orchestration transition to a downstream symptom. If the captured evidence does not identify a unique prompt or producer owner, the attribution remains unresolved instead of assigning blame to the last agent in the chain.
The fix follows the diagnosed fix type: a prompt, code, configuration, or data change when supported. Prompt changes can be generated and tested in simulation against the failure and regression scenarios, then deployed under governance with production verification.
The contract does not force every case into a confident classification. A diagnosis can be grounded, partial, or abstained; causal ownership can remain unresolved when required evidence is missing. That uncertainty is explicit so review can focus on the missing prompt, runtime, or causal evidence.
Connect your agent and see exactly where issues originate - then watch Converra fix them automatically.
Start for free