What Is the Cost of Producing Teacher-Generated Training Data?

Cost per accepted example equals the sum of four terms divided by yield. Term one is teacher inference: input tokens plus output tokens at the closed model's price, times the number of generation attempts per input. Term two is verification: any judge call, deterministic check, or second-model pass used to decide whether the output is a candidate. Term three is human review: minutes per example times the reviewer's loaded hourly rate. Term four is tooling and storage, usually small. Yield is the fraction of generated examples that end up admitted to the dataset.

Two things follow immediately. Dividing by yield means a pipeline that admits one example in four pays four times its per-example cost for every example it keeps. And because review is priced per example reviewed, not per example kept, a rejected example costs the same review minutes as an accepted one. Cutting the token price by half changes the smallest term; raising yield changes all of them.

Suppose an agent's inputs average 3,000 tokens with retrieved context and the teacher's output averages 400 tokens. At illustrative frontier prices of a few dollars per million input tokens and several times that per million output tokens, one generation costs in the region of a cent or two. Add a verification call at a similar cost. Then a reviewer spends four minutes on the example at a loaded rate of ninety dollars an hour, which is six dollars. The token terms are rounding error next to the review term.

Now apply yield. If half the generated examples are rejected, the cost per accepted example is roughly twelve dollars, almost all of it review. If the pipeline can raise yield to eighty percent by filtering before review, the same accepted example costs about seven and a half dollars. If the reviewer's time drops to two minutes because the review interface shows the evidence and the proposed target side by side, it falls to under four. The numbers are hypothetical; the ordering of the terms is not.

It is tempting to start by generating once with no retry, or by choosing a cheaper teacher. Generate once with no retry is a good discipline for a different reason: it prevents the pipeline from selecting the teacher's luckiest output and calling it typical. Choosing a cheaper teacher is usually a mistake, because a weaker teacher produces lower-yield examples and yield is the expensive term.

The one place teacher cost matters is at scale: tens of thousands of examples across several agents. There, reusing production runs as the inputs rather than synthesizing new ones removes the input-generation cost and keeps the inputs at the real distribution's difficulty, which raises yield as a side effect. Part 6 covers building examples from real conversations; the economic argument for it is the same as the quality argument.

Every example a reviewer sees costs review minutes whether or not it is kept. So the cheapest improvement is to reject bad examples before a human sees them. Deterministic checks do this well: schema conformance, required fields, argument values that appear in the source conversation, tool citations that match the tool results, length within range. Each check costs nothing per example and removes a class of failure from the review queue entirely.

A judge model can do a second layer, flagging outputs that contradict the evidence or that assert facts the context did not contain. That layer is not free, and it is not ground truth; a judge that rejects good examples lowers yield the same way a bad teacher does. Measure the judge's agreement with reviewers on a sample before trusting it to filter, and keep the cases it abstains on in the review queue rather than dropping them.

Many generated examples share the same underlying behavior: the same rule applied under the same conditions with different surface wording. Reviewing them one at a time pays full review cost for each. Reviewing them as a cluster, with a representative example, the evidence, the proposed target, and access to the members, lets one decision cover the group. A final per-example check then confirms the decision applies to each member before it is admitted.

The per-example check is not optional, and it is where the savings can be lost if it is skipped. A cluster decision that is propagated without checking applicability admits examples the decision did not fit, and those become label errors that a later evaluation will pay for. The honest accounting counts the propagation check as review time, at a lower rate per example than a full review, and reports direct and propagated examples separately. Part 12 covers keeping that distinction in the lineage.

A rejected example still carries information: what the teacher got wrong, under what input, and why the reviewer rejected it. Recorded with a reason code, rejections become the seed list for adverse cases in the evaluation set, the evidence for which failure classes need a deterministic check, and the counterexamples that part 11 argues belong in training data as well. A pipeline that discards rejections throws away the second-most-useful thing it produced.

Recording rejections also keeps the yield number honest. Yield reported against generated examples, with rejections listed by reason, is a measurement. Yield reported against examples that reached review, after silent filtering upstream, is a flattering number that hides the cost of the filter.

Teacher-generated data means closed-model outputs used to train another model. The closed providers' terms speak to that directly. Anthropic's commercial terms, as reviewed on 2026-09-23, prohibit using the services to build a competing product or service, including to train competing AI models. Google's Gemini API terms prohibit using the services to develop models that compete with them. Whether a customer-specific agent model falls inside or outside those clauses is a question for counsel, and the answer changes the cost of the whole pipeline from affordable to unavailable.

Treat the terms review as a line item with a real cost and a real schedule, and do it before generating at scale rather than after. Record the terms version and date against each dataset version, because a dataset built under one version of the terms may not be usable under the next. Part 42 works through the licensing layers in detail.

Before generating, estimate the four terms and the expected yield from a pilot of a few dozen examples, and set a stop rule: if cost per accepted example exceeds a stated figure after the pilot, redesign before scaling. After generating, report the measured values: tokens and spend by term, review minutes per example, yield by rejection reason, direct versus propagated admissions. That report is what makes the next dataset cheaper, because it shows which lever to pull.

The number to carry forward is cost per accepted example at the measured yield. It feeds part 27's break-even calculation, where the data cost is amortized over the inference savings the fine-tuned model produces. A dataset that costs more than the savings it enables is not a bargain at any token price.

Converra's fine-tuning workflow starts from reviewed production runs rather than synthesized inputs, which removes the input-generation term and keeps examples at real difficulty. It groups findings into clusters with a representative, the source evidence, and a proposed target, records a scoped decision, and admits each member only after a per-example check, with direct and propagated admissions kept distinct in the dataset version. Every admitted, rejected, or unresolved candidate is accounted for, which is what makes yield measurable.

Converra does not run training and does not use customer data to train any model without a written, tenant-exclusive election. The review minutes, the teacher spend, and the terms review are the customer's costs, and no open-weight production verdict for a customer agent has been published. The workflow makes the accounting in this article visible; it does not make the costs disappear.