Why Can a Fine-Tuned Model Produce Perfect JSON but Worse Answers?

Look at a supervised fine-tuning dataset for a structured-output agent from the model's point of view. Every target begins with the same opening brace. Every target has the same key names in the same order. Most targets share the same closing pattern. Those tokens are nearly deterministic given the preceding tokens, so the loss on them drops fast. After a single epoch the model reproduces the envelope reliably, and after a few more it is close to perfect.

The content of each field is a different kind of token. A risk tier, a routing decision, a cited evidence id, a free-text finding: each depends on the input, and the correct value changes from example to example. The model has to learn a mapping from evidence to decision, which is a much harder function than reproducing a template. Cross-entropy on those tokens falls slowly, and with too few distinct examples it does not fall at all in any useful direction.

The loss you watch during training is one number averaged over every target token. Envelope tokens are the majority of the tokens in a short structured target, so the average is dominated by tokens the model has already learned. A loss that drops from 6 to 0.6 in a few steps looks like rapid learning. Most of that drop is the model memorizing braces and key names.

The content tokens can sit at a high loss while the average looks healthy. Nothing in the aggregate exposes this. Providers that let you inspect per-token or per-message loss make the split visible; if yours does not, the only way to see it is to evaluate the content decision directly on a held-out set. Treat a fast-falling training loss as evidence that the format converged, and nothing more.

A gate that scores only whether the output parses is measuring the part of the task the model was always going to learn. Every candidate trained for more than a couple of epochs on consistent targets passes it. That gate cannot distinguish a model that reasons over the evidence from a model that emits the majority class in a well-formed wrapper.

The right structure is two checks with different roles. A structure check is a precondition: schema-invalid, empty, truncated, or refused outputs score zero for that case and the case still counts. A content check is the decision rule: the value in each field is compared with an answer key or a reviewed target, and the candidate passes or fails on that comparison. Reporting 100 percent schema validity as if it were a quality result is the single most common way a bad fine-tune gets described as a good one.

The signature is consistent across tasks. Every output parses. The distribution of decisions collapses toward one or two values, usually the majority class in training. Confidence, if you can read it from token probabilities, is flat and high across cases the model should find easy and cases it should find hard. Free-text fields are fluent, generic, and reuse phrases from the training targets rather than details from the input.

Compare the candidate against its untuned base on the same cases. The base often does worse on structure, with missing keys or trailing prose, and better or equal on the decision, because it is still reading the input rather than pattern-matching a template. If the base decides as well as the candidate, the fine-tune taught format only. That is a data finding, and part 15 covers what to do when the two tie.

Report the two measurements in separate rows for every model in the comparison, with the incumbent included. A pre-registered composite works well for the decision: for example, exact match on the primary classification, recall of required findings against the answer key, coverage of required evidence citations, and a penalty for unsupported entities or numbers. Structure is reported beside it, never folded into it.

The table below is a minimal layout. Each row is a distinct claim about the model, and a reader can see immediately when a candidate is winning on the row that does not matter.

Many serving stacks repair malformed output before it reaches the application: they strip prose around the JSON, coerce types, fill missing keys with defaults, and retry on parse errors. In production that is reasonable. In a model comparison it destroys the measurement, because a candidate whose raw outputs are worse can score identically to the incumbent after repair.

Capture the raw completion for every case in every arm and score it before any repair runs. If the pipeline must repair, record the repair as structured metadata on the case, count repaired cases separately per arm, and never let a default value inserted by repair count as a correct decision. Part 20 goes through this comparison discipline in full.

A model that learned format and not judgment did so because format was the only consistent signal in the training set. The usual reasons: too few examples for the decision to generalize, examples drawn from a handful of templates so the decision is constant within each template, targets that all share the same decision, or targets produced by a teacher that itself defaulted to one answer.

The remedies are also data-side. Add distinct examples at the complexity of the evaluation distribution, not more copies of the same shapes. Balance the decisions that appear in targets, and include reviewed counterexamples where the same surface form leads to a different decision. Check that the teacher's decisions were verified before they became targets. More epochs on the same data will make format even more perfect and the decision no better, which is the subject of part 13.

Suppose an agent classifies supplier documents into four risk tiers and lists the findings that justify the tier. Suppose the training set holds 240 examples generated from 12 scenario templates, and 70 percent of them are tier one with no findings. A candidate trained for eight epochs parses every output, assigns tier one to 92 percent of held-out cases, and lists findings only when the input matches a training template almost verbatim.

On the structure row the candidate beats the incumbent. On every content row it loses. The incumbent, a closed model with a good prompt, produces the occasional malformed output and reads the document. The right conclusion is not that the candidate needs another epoch or a larger base. It is that 240 rows from 12 templates with a 70 percent majority class is not a dataset that can teach the decision, and the next step is generating and reviewing more distinct families.

Converra's model benchmarks run every candidate and the incumbent on the same scenarios built from the agent's instruction, three runs per scenario with median scoring, and report quality, cost, and latency per model before opening a pull request for a winning switch. For a fine-tuned candidate, its workflow curates reviewed production runs into versioned datasets with protected splits, records the training job and the gate result against that dataset version, and keeps the production switch behind an explicit approval. Training runs on your provider or infrastructure; Converra records and gates it.

Converra does not use customer data to train any model without a written, tenant-exclusive election. No open-weight production verdict for a customer agent has been published yet, so whether a given candidate holds parity on your traffic is unobserved until the model production test measures it. The discipline in this article, structure and content scored apart, applies whether or not Converra runs the comparison.