Start with the dated change and the affected run. If behavior has worsened gradually rather than suddenly, use the agent drift guide for the definition and detection workflow.
This page covers a sudden stopped-working incident. Gradual behavioral decline belongs to the agent-drift guide.
A provider update can change instruction following, output format, or tool-use behavior even when your prompt file is unchanged.
A prompt patch, tool configuration, policy, or integration change can alter the inputs and constraints the agent sees.
A new customer cohort or traffic source can introduce requests the agent was not tuned to handle.
Products change, pricing updates, policies shift. The agent's instructions reference a world that no longer exists. Users get outdated or incorrect information delivered confidently.
Do not start by rewriting the prompt. Use the first evidence that distinguishes a sudden incident from a broad behavioral trend.
| Symptom | First evidence to inspect |
|---|---|
| The agent stops responding | The affected run, its last successful tool call, and the provider or tool error in the same change window. |
| Answers change after an update | A before-and-after pair of runs plus the exact model, prompt, tool, or policy configuration used for each. |
| Only a new request cohort fails | The failing requests alongside the cohort, referrer, or context change that made them different. |
| Answers become stale or wrong | The affected answer and the product, policy, or knowledge source that changed since it was last correct. |
If a model, prompt, plugin, policy, or product change lines up with the failure, keep the incident scoped to that run window and inspect the affected evidence before you change anything else.
If the behavior declined slowly with no clear change window, follow the agent drift guide instead of treating a broad pattern as one incident.
| First check | Next check | |
|---|---|---|
| Change window | List model, prompt, tool, policy, and product changes | Compare the affected runs before and after the change |
| Affected behavior | Keep examples of the failing runs and expected behavior | Group the failure by task, tool, output, or context |
| Candidate fix | Test the smallest change that addresses the evidence | Compare the candidate against the affected and preserved scenarios |
| After approval | Measure production evidence and record the outcome | Keep incident evidence distinct from gradual drift |
Keep the failing runs, expected behavior, and relevant change window together before editing the prompt.
Group the failure by task, tool, output, or context. That gives a candidate fix a specific job to do.
Compare a candidate against the affected scenarios and the behavior you need to preserve before approval.
After an approved deployment, the change can be verified from production data instead of assuming the incident is closed.
Start with the dated change and the affected run. A provider update can shift instruction following or output format. A prompt, tool, policy, or integration change can alter inputs and constraints. Stale product context can also make once-correct answers wrong.
Model updates can change how the same prompt is interpreted, including instruction following, output formatting, tone, and tool use. Compare affected runs before and after the provider release before assuming the prompt itself is the cause.
A sudden failure after a dated change is an incident to investigate from the affected runs. A gradual, broad behavioral decline needs the separate agent-drift definition and detection workflow. Do not collapse the two before you have the evidence.
Connect your agent to inspect the evidence around a production failure and decide what to test next.
Start for free