Analyzing Insights

Understand patterns and issues from your logged runs.

What Are Insights?

Insights are AI-generated analysis of your runs:

• Performance metrics - Task completion, sentiment, quality
• Common topics - What users ask about most
• Issues - Recurring problems or confusion points
• Recommendations - Suggested improvements

Viewing Insights

Dashboard

1. Go to converra.ai/agents
2. Select an agent
3. Click the Insights tab

You'll see aggregated insights from all logged runs.

Via MCP

How is my support agent performing?

Show insights for agent_123 over the last 30 days

Via SDK

const insights = await converra.insights.forAgent('agent_123', {
  days: 30
});

console.log(`Task completion: ${insights.metrics.taskCompletionRate}%`);
console.log(`Avg sentiment: ${insights.metrics.avgSentiment}`);
console.log(`Total runs: ${insights.metrics.conversationCount}`);

Key Metrics

Task Completion Rate

Percentage of runs where users achieved their goal.

Rate	Interpretation
90%+	Excellent - agent is working well
70-90%	Good - room for improvement
<70%	Needs attention - review common failures

Sentiment Distribution

How users feel about interactions:

Positive: ████████████░░░░░░ 68%
Neutral:  ████░░░░░░░░░░░░░░ 22%
Negative: ██░░░░░░░░░░░░░░░░ 10%

Common Topics

What users ask about most:

1. Order status (34%)
2. Returns/refunds (28%)
3. Account issues (18%)
4. Product questions (12%)
5. Other (8%)

Issue Detection

Converra identifies common problems:

Recurring Confusion

Issue: Users frequently ask "what do you mean by..."
       after the AI's first response.

Suggestion: Add clearer explanations or examples.

Abandoned Runs

Issue: 15% of runs end abruptly after
       technical terms are used.

Suggestion: Simplify language or define terms.

Unresolved Requests

Issue: "I still don't understand" appears in 8%
       of runs.

Suggestion: Add step-by-step breakdowns for
            complex topics.

Run-Level Insights

View insights for individual runs:

const conversation = await converra.runs.get('conv_456');
const insights = await converra.runs.getInsights('conv_456');

console.log(`Sentiment: ${insights.sentiment}`);
console.log(`Topics: ${insights.topics.join(', ')}`);
console.log(`Task completed: ${insights.taskCompleted}`);
console.log(`Summary: ${insights.summary}`);

Filtering Insights

Focus on specific subsets:

// Last 7 days only
const recent = await converra.insights.forAgent('agent_123', {
  days: 7
});

// Low sentiment runs
const { data: negative } = await converra.runs.list({
  agentId: 'agent_123',
  sentiment: 'negative'
});

Acting on Insights

High Task Completion, Low Sentiment

Users succeed but aren't happy. Check:

• Tone - too robotic or formal?
• Response length - too long or short?
• Personalization - too generic?

Low Task Completion, High Sentiment

Users like the AI but don't get results. Check:

• Accuracy - is information correct?
• Completeness - all cases covered?
• Follow-through - does AI confirm resolution?

Recurring Topics

If certain topics dominate:

• Add specific handling for them
• Consider creating specialized agents
• Update documentation/FAQ

Negative Sentiment Spikes

Investigate recent runs:

• New edge cases appeared?
• Agent change caused issues?
• External factor (product issue, etc.)?

Insights-Driven Optimization

Use insights to guide optimization:

// Get insights first
const insights = await converra.insights.forAgent('agent_123');

// Use findings to guide optimization
const optimization = await converra.optimizations.trigger({
  agentId: 'agent_123',
  intent: {
    targetImprovements: ['task completion'],
    hypothesis: `Users struggle with "${insights.topConfusionPoints[0]}"`
  }
});

Exporting Insights

For reporting or further analysis:

const insights = await converra.insights.forAgent('agent_123', {
  days: 30
});

// Export to your reporting system
await sendToAnalytics({
  agentId: 'agent_123',
  period: '30d',
  taskCompletion: insights.metrics.taskCompletionRate,
  sentiment: insights.metrics.avgSentiment,
  conversationCount: insights.metrics.conversationCount
});

Fleet Intelligence

Fleet Intelligence gives you a business-level view of how your agents are performing across your entire fleet. Instead of reviewing runs one by one, you get aggregated patterns and impact analysis.

What You Get

• Issue pattern clustering - Similar failures are grouped by failure type and agent type, so you see "47 runs hit eligibility misparse" instead of 47 individual alerts
• Business impact analysis - Each issue pattern is scored by estimated cost and improvement upside, so you can prioritize fixes by business value
• Age-based refresh - Fleet Intelligence data refreshes automatically as new runs arrive, with a "last updated" indicator so you know how current the data is

Viewing Fleet Intelligence

1. Go to converra.ai/agents
2. The Fleet section on the homepage shows aggregated patterns
3. Click any issue pattern to see the filtered runs that match

Via MCP

Show fleet intelligence for my support agent

What are the top failure patterns across my agents?

Unified Issue Intelligence

Converra automatically identifies and tracks issues across your runs using LLM-powered analysis with evidence enrichment.

How It Works

Each issue includes:

• Issue type and description - What's going wrong (e.g., "eligibility misparse", "context loss after turn 3")
• Evidence - Links to specific runs that demonstrate the issue
• Severity - Critical, major, or minor based on frequency and impact
• Affected agents - Which agents exhibit this issue

Issues are generated from run insights and aggregated at the agent and fleet level. They replace the older primaryIssueLabels system with richer, evidence-backed intelligence.

Business Impact Analysis

Available at both the run and agent level, business impact analysis quantifies the real-world cost of agent failures and the upside of fixing them.

Run-Level

Each run's insights include a business impact assessment — what went wrong and what it cost (e.g., "customer likely churned due to unresolved billing issue").

Agent-Level

Aggregated across all runs for an agent:

• Cost of failures - Estimated business cost of recurring issues
• Improvement upside - Expected gains from fixing top issues
• Priority ranking - Issues ranked by business impact, not just frequency

Via SDK

const insights = await converra.insights.forAgent('agent_123', {
  days: 30
});

// Business impact is included in the insights response
console.log(insights.businessImpact);

Step Failure Aggregation

For agents with step-level diagnosis data, Converra aggregates failure patterns across runs to show which agent steps fail most often.

• 30-day rolling window - Aggregation covers the last 30 days of runs
• Per-step breakdown - See failure rates for each step in your agent's execution flow
• Clickable patterns - Each failure pattern links to the runs that exhibit it

Via MCP

Show step failure patterns for my support agent

Best Practices

1. Review weekly - Check insights at least weekly
2. Track trends - Look for changes over time
3. Investigate outliers - Unusually good or bad runs teach the most
4. Act on findings - Insights without action are wasted
5. Close the loop - After changes, verify improvements
6. Use Fleet Intelligence - Start with the fleet view to spot systemic issues before diving into individual runs

Next Steps

• Running Optimizations - Act on insights
• Best Practices - Prevent common issues