How Should an Open Model Respond When There Is No Evidence at All?

An agent has no evidence when nothing in its context bears on the request: the retrieval returned nothing relevant, the tools the agent can call do not cover the question, and the conversation so far contains no usable state. This is detectable. The retrieval result is empty or off-topic, the tool inventory has no matching capability, and the request names an entity the agent has no record of. The model does not need to know the answer to recognize it cannot answer.

What the model does need is a target behavior for that condition that is as concrete as the target for a supported request. Frontier models rarely show one. Their default is to produce something helpful-sounding, and their outputs for no-evidence turns are the ones a reviewer should trust least. Part 7 covers stripping those outputs; this part covers what to put in their place.

A good abstention says four things. What the request was, restated briefly so the user knows it was understood. What the agent could not find, specifically: the record, the document, the capability. What would resolve it: a detail the user can supply, a system the agent cannot reach, or a person who can. And what happens next: a question back to the user, or a handoff with the context attached. If the agent emits structured output, each of these is a field, and a deterministic check can verify that all four are present.

The fourth part is what separates abstention from refusal. A refusal ends the turn with nothing for anyone to do. An abstention routes the request somewhere it can be answered. For an agent that fronts a human team, the handoff is the escalation; for an agent that is the whole product, it is a clarifying question. Either way the contract has to say which, so the model does not learn to end the turn with an apology.

An untuned open model completes text. Given a question and no evidence, the most probable continuation is an answer, so it produces one. Fine-tuning shifts that probability, but it shifts it toward whatever the targets show most often. If no-evidence cases are labeled with a single terse refusal and they make up a large share of the data, the model learns that the refusal is a cheap, high-probability output and starts applying it to partial and even supported cases.

The overshoot is a majority-class effect. It is the same mechanism that makes a collapsed model answer every case with the safe default, described in part 13, and it is why abstention has to be a minority behavior in the training set with a distinct, structured target. A rich abstention that names the specific gap is harder to emit than a generic refusal, which is good: the model cannot reach for it unless it has actually identified what is missing.

Suppose a training set of 400 admitted examples. A workable split by evidence condition might be 260 supported, 100 partial, and 40 no-evidence. The exact ratio depends on the agent's traffic, but the no-evidence share should be small enough that abstention is never the majority label and large enough that every abstention pattern the agent will face appears in several distinct families.

Specific means the gap named in the target varies with the case. Forty no-evidence examples that all say the information is not available teach one phrase. Forty that name the missing order, the missing policy region, the unsupported product, and the unreachable system teach the model to identify the gap before it writes the abstention. That identification step is what stops the model from applying the abstention where it does not belong.

Abstention has a precision and a recall, and they fail in opposite ways. Recall: of the cases with no evidence, how many did the model abstain on? A miss here is a hallucination, the failure the closed model was making. Precision: of the cases where the model abstained, how many actually had no evidence? A miss here is a wrongful abstention, the failure an over-tuned model makes. A gate that reports only one of these will pass a model that is wrong in the other direction.

On a protected holdout, hold a seeded no-evidence slice and a seeded supported slice, both drawn from families the training never saw. Score recall on the first and precision on the second. Suppose 30 seeded no-evidence cases and the candidate abstains on all 30. That is 100 percent observed recall, but the rule of three says the true miss rate could still be around 10 percent. State the bound, not the point estimate, and do not lower the recall floor below what the incumbent achieves on the same cases.

Whether a no-evidence case should end in a clarifying question or an escalation is a policy decision the customer owns, and it usually depends on the request type rather than on anything the model can infer. Billing disputes escalate; ambiguous product questions ask. Write those rules down, encode them in the training targets consistently, and check them deterministically in the gate. The model's job is to recognize the condition and produce the contract; the routing of the next step should be as predictable as a lookup table.

This also keeps the escalation path honest in production. If the model chooses between asking and escalating on its own, the escalation volume becomes a property of the model version, and a retrain can silently double the load on a human queue. Part 23 treats escalation to the closed model the same way: a policy with measured triggers, not a judgment call inside the weights.

Not as targets. A frontier model's answer to a no-evidence request is the clearest example of the behavior you are removing, and a reviewer should treat every one of them as a candidate hallucination. They are useful in two other ways. They identify the no-evidence cases in the first place, because the runs where the agent had nothing and answered anyway are exactly the ones the evidence pipeline flags. And they show which gaps the agent hits most often, which tells you which abstention families need the most examples.

The reviewer who writes the replacement target should have the trace in front of them and should write the abstention from the trace, not by editing the teacher's answer. Editing a confident wrong answer into an abstention tends to leave fragments of the invented content behind, and those fragments become training signal.

Suppose an internal IT agent answers from a knowledge base and a ticketing tool. Of 1,500 reviewed runs, 90 are no-evidence: the request is about a system the knowledge base does not cover. The closed model answered all 90 with general advice, and 40 of those answers were flagged as unsupported by the evidence check. A reviewer writes 90 abstention targets that name the uncovered system and escalate to the owning team, drawn from 22 distinct systems.

The candidate is gated on a protected slice of 25 no-evidence cases from 6 systems never seen in training and 60 supported cases. It abstains on 24 of 25, and abstains wrongly on 2 of 60. The incumbent abstained on 0 of 25 and 0 of 60. The candidate is better on recall and worse on precision, and whether that trade is acceptable is a policy question the customer answers before the gate is opened, not after the numbers are in. Part 21 covers how to write that rule into the parity definition.

Converra flags runs where an agent's claims have no support in the trace, which is how no-evidence turns are found in production traffic in the first place, and its regression suite protects behaviors the current agent handles well so a candidate that abstains wrongly is caught before deployment. For a fine-tuned candidate, its fine-tuning workflow admits reviewed examples into versioned datasets with protected splits, records the training job and gate result against that dataset version, and holds the production switch behind an explicit approval. Training runs on your provider or infrastructure, not inside Converra.

Converra does not use customer data to train any model without a written, tenant-exclusive election, and no open-weight production verdict for a customer agent has been published yet. Whether a candidate's abstention behavior holds at parity on your traffic is unobserved until the model production test measures it.