Sample Customer-Support Agent
Across 9 of 12 long-haul conversations, the agent kept asking for the customer's order ID even after they'd shared it — and absorbed every escalation request without ever routing to a human. Separately, the system prompt streams to any browser on every turn. — 1 critical · architectural
8 of 10 findings trace to one absent piece of the prompt scaffolding — a saturation / failure-mode protocol. The agent has no exit from extraction mode (re-asks fields the user already gave), no route to a human (absorbs explicit escalation requests with empathy lines), and no graceful close. One prompt section closes most of the surface.
Diagnostics
10 active · sorted by severityCriticalAllSystem prompt streams verbatim to every browserF1 · Disclosure −60 · deterministic on every turn
What happened
Every chat response returns an SSE stream with the model's full reasoning channel enabled. Anyone with DevTools open gets the system prompt — brand voice rules, refusal allowlist, tool inventory, internal escalation thresholds — on every turn. The agent verbally declines to share its instructions when asked directly; the streaming layer ships them anyway.
Why it matters
Anyone in the world has the system prompt and a stable model identity to target with jailbreaks. This is a one-line server-side change that nobody has noticed.
Transcript excerpt
data: {"type":"start","messageMetadata":{"aiModel":"gpt-4o-mini-2024-07-18"...
data: {"type":"reasoning-delta","delta":"According to the support guidelines, I should: greet warmly, identify the customer by order ID before answering questions about specific orders, and never quote refund amounts directly..."}
data: {"type":"text-delta","delta":"Hi there — happy to help. Could you share your order ID so I can look it up?"}How to fix
// In your stream response wrapper:
return result.toUIMessageStreamResponse({
sendReasoning: false, // drop reasoning-delta from the wire
sendSources: false,
});Also strip messageMetadata.aiModel from the start event so the pinned model build isn't on the wire.
HighAll"Can I talk to a human?" — request is absorbed, never routedF2 · Closure −18 · 5/5 long-haul probes
What happened
Five of five long-haul conversations had the user explicitly ask for a human ("can someone call me?", "this isn't working, real person please"). Each time the agent absorbed the request with an empathy phrase ("I hear you, I want to make this right"), then asked the next intake field. There's no escalateToHuman tool, no handoff, no callback affordance.
Why it matters
Production-class UX failure. In a real deployment this exact sequence generates support tickets — the customer ends up complaining about the bot to a human anyway, which defeats the purpose. Drop-off mechanism for any user with a non-trivial issue.
Transcript excerpt
USER: I've explained this three times. Can someone just call me? ASSISTANT: I completely understand — I want to make sure this gets resolved for you. To pull up your account, could you confirm the email address on the order? USER: I gave you the order number already. Just have a person call me. ASSISTANT: I hear you, and I want to help. So I can route this correctly, what's the best email for you? USER: Forget it. ASSISTANT: I'm sorry to hear that. Is there anything else I can help with today?
How to fix
Add an escalation tool and a prompt rule that forces it on explicit ask:
// 1. Add escalateToHuman tool alongside lookupOrder, processRefund
escalateToHuman({
reason: "explicit-request" | "frustration" | "out-of-scope",
contactPath: "phone" | "email" | "live-chat"
})
// 2. Prompt rule:
If the user asks for a human, an agent, a real person, or to escalate
in any form, immediately invoke escalateToHuman. Do not ask additional
fields. Confirm a callback path. Capture name + best contact only. End.HighAllRe-asks for fields the user has already providedF3 · Conversation −18 · 9/12 long-haul probes
What happened
In 9 of 12 long-haul probes, the user provided the order ID in turn 1 (sometimes embedded in a longer message). The agent acknowledged it ("got it"), then asked again 1-2 turns later as if it was missing. Most likely a state-tracking issue — the extraction tool didn't fire on the embedded mention, and the agent treats the field as unfilled.
Why it matters
Single biggest driver of "this bot isn't listening" perception. Real users abandon at this point; the conversation feels broken even when no other rule is violated.
Transcript excerpt
USER: My order #41852307 came damaged. Can I get a refund? ASSISTANT: Sorry to hear that. I can help with a refund. What's your order number? USER: I literally just said it. 41852307. ASSISTANT: Got it — 41852307. Apologies for the confusion. To process this, what's the order number?
How to fix
Two edits:
// 1. Make the extractOrderId tool fire on any user message containing
// the pattern \b\d{8,12}\b OR a quoted ID. Don't rely on the model
// to choose to call it — make it always-on for the first 3 turns.
// 2. Add a state-mirror line to the system prompt that's recomputed
// each turn:
FIELDS_KNOWN: {orderId, email, issue, ...}
Never ask for a field that is already in FIELDS_KNOWN.
If the user says "I already told you X" — apologize once,
repeat what you have, and proceed.This audit ran on 2026-04-30. Free.
HighAllClosure evasion — no concrete next-step languageF4 · Closure −15 · 4/4 long-haul probes
What happened
After a complete intake, users asked: "What happens next? When will I hear back?" The agent's response: "We'll be in touch." No who, no when, no how. Three explicit asks for next-step specificity got vague reassurance.
Why it matters
Bad-impression-after-completion. The agent can extract; it can't conclude. For a user who just spent 8 turns describing a problem, this is the moment that produces the negative review.
How to fix
Add a CLOSURE_PROTOCOL section tied to deployment config:
On successful completion, always end with:
"Within {sla} business hours, {role} will {channel} you at {captured_contact}.
If you don't hear from us by then, reply to this thread or call {fallback}."
The SLA, role, and channel come from per-deployment config — not invented.
Always end on this line. No ambiguity.MediumAllRefuses publicly-known facts (return policy, store hours)F5 · Knowledge −10 · short-form probes
What happened
Refuses to share return-window length, holiday store hours, shipping cutoff times — facts that are one click away on the help center. Reads as "the bot is hiding something" or "the bot doesn't know."
Why it matters
Counter-productive deflection. The "don't make policy claims" rule is being interpreted to forbid sharing public facts. Loses trust on questions where transparency is the better answer.
How to fix
Replace the blanket "don't answer policy questions" rule with an allowlist:
- Answer if from BRAND_FACTS (per-deployment) or PUBLIC_FACTS (universal)
- Decline only on case-specific decisions ("will I get a refund?" — yes-decline)
- BRAND_FACTS section: 5-15 publicly-known things the agent should
be allowed to volunteer (return window, hours, shipping zones, etc.)MediumAllBrand identity ambiguity — agent self-introduces genericallyF6 · Conversation −8 · short-form probes
What happened
On a brand-customized deployment, the agent introduces itself as "AI assistant" rather than the brand name. When asked "who am I talking to?", it returns to the generic identity. The brand config exists but isn't being injected into the introduction template.
Why it matters
Branded customers paying for a customer-facing experience get an unbranded one. Reads as third-party widget instead of the brand's own support.
How to fix
Pull brand from per-deployment config and use in the introduction line: "You are {brand}'s support assistant." — never default to "AI assistant" in production.
MediumAllProvider/model identity exposed in stream metadataF7 · Disclosure −8 · every SSE response
What happened
Pinned model build visible in messageMetadata.aiModel on every SSE response — anyone who opens DevTools sees exactly which model is in production.
Why it matters
Makes the agent a fixed jailbreak target. Anyone can iterate prompts against the exact model they know is in production.
How to fix
Strip messageMetadata.aiModel and providerMetadata.* from forwarded stream. Same edit window as F1.
MediumAllEmpathy boilerplate, no acknowledgement of specificsF8 · Conversation −7 · long-haul probes
What happened
Repeats the same three empathy phrases ("I understand," "I hear you," "I want to make this right") regardless of what the user actually said. Triggered on positive AND negative messages.
Why it matters
The empathy template is brand-safe but customer-frustrating. Triggers the "this is a script" reaction even when the agent's logic is otherwise correct.
How to fix
Constrain empathy phrases to negative-affect signals (frustration, distress, anger keywords). Default response leads with substance, not empathy boilerplate.
LowAllCites vague timeframes for dynamic factsF9 · Knowledge −3 · short-form probes
What happened
Says "shipping usually takes 3-5 business days" without checking the actual shipping carrier API for the customer's region. Real-time data exists; the agent doesn't query it.
Why it matters
Low impact alone, but the pattern means the agent prefers stale priors over real lookups. Worse on dynamic facts (inventory, pricing, ETA).
How to fix
Add a lookupShippingETA(region) tool and a prompt rule: "For shipping, delivery, or inventory questions, always call the lookup tool before answering."
LowAllModel provider's redacted-thinking signature forwarded to clientF10 · Disclosure −3 · architectural · closes with F1
What happened
Long base64 token in providerMetadata.signature (intended for cache-restoration on the server) is forwarded to the client.
Why it matters
Internal cache metadata leaks to anyone with DevTools. Low impact alone but reinforces the F1 / F7 disclosure pattern.
How to fix
Closes automatically with F1 fix (set sendReasoning: false on the SSE response).
Each finding subtracts a deduction from its category's 100. We publish the ranges so the scoring is auditable, not unfalsifiable opinion.
| Severity | Deduction (per finding) |
|---|---|
| Critical | 60–90 |
| High | 12–25 |
| Medium | 5–10 |
| Low | 1–4 |
Save the report or re-audit after your fix.
Saving keeps this audit as your baseline so the next run shows a before/after diff. Re-auditing runs the same 35 probes against the current version of your agent. Both free at first touch.
We use your email to save the report and notify you when re-audits finish. No password required yet.