At What Traffic Volume Does Fine-Tuning an Open Model Break Even?

The one-time cost is everything spent before the first production request: curating and reviewing examples, generating teacher outputs where needed, the training run, gate evaluation runs on both models, and the integration work to serve the adapter behind the agent. The recurring fixed cost is capacity that bills whether or not it is busy. Savings per request are what pays both back.

The training run itself is usually the cheapest line. Managed LoRA supervised fine-tuning on models up to 16B is listed at $0.50 per million training tokens on the Fireworks pricing page as of 2026-09-23. A dataset of a few thousand examples costs single-digit dollars to train. Reviewer time for those examples costs hundreds of times more. Teams that budget the transfer from the training invoice miss almost all of the cost.

Savings per request equals the closed-model cost of a request minus the open-model variable cost of the same request, minus the expected cost of fallback on that request. Daily savings equals that number times requests per day, minus the daily fixed cost of capacity. Break-even volume is the requests per day at which daily savings turn positive, and payback time is one-time cost divided by daily savings above that volume.

Write it down with your own inputs before you believe anyone's estimate. Tokens per request, split by input and output, come from the gateway logs. The closed per-token price comes from your invoice, not the list price. The open variable cost comes from a load test at production concurrency, which part 28 covers. The fallback share comes from the routing policy or, before one exists, from the gate's escalation rate.

Suppose requests average 1,800 tokens and the closed model costs $0.0072 per request at the invoiced rate. Suppose one open replica costs $8.00 per hour, must stay up around the clock for latency, and sustains 14,400 requests per hour at full load. Suppose 10 percent of requests fall back to the closed model. Suppose one-time setup, mostly reviewer and engineering time, comes to $18,000. All of these are illustrative.

Fixed capacity is $192 per day, and because the replica is already paid for, the variable cost of an open-served request is close to zero, so each one saves about $0.0072. With 10 percent fallback still billed at the closed rate, the effective saving per incoming request is about $0.0065. Daily savings turn positive above roughly 29,600 requests per day. At 60,000 requests per day, daily savings are about $197, and the $18,000 setup pays back in roughly 91 days. At 20,000 requests per day, the transfer loses about $62 a day and never pays back.

Now suppose the fine-tuned adapter is served on a hosted per-token endpoint at a blended $0.40 per million tokens, so a request costs about $0.00072 with no fixed capacity. Savings per open-served request are about $0.0065; with 10 percent fallback, about $0.0058 per incoming request. There is no fixed daily cost, so break-even on operating cost is any volume above zero, and the $18,000 setup pays back after about 3.1 million requests.

At 20,000 requests per day, the per-token deployment reaches payback in about 156 days, where the dedicated deployment never does. At around 270,000 requests per day in this example, one fully used dedicated replica costs less per request than the per-token rate, and the ranking flips. The deployment shape is part of the break-even question, not a detail after it.

The minimum replica count is not chosen by cost; it is forced by the latency the agent promises at peak concurrency. A strict latency target with a spiky traffic profile can require two or three replicas of headroom that sit mostly idle, which multiplies the fixed term and pushes break-even up. A relaxed target on a background workload can run one replica near full utilization or scale to zero overnight.

Measure peak concurrency, not average. Suppose average load is 20 requests per second but the busiest ten minutes reach 60. The replica count that meets latency at 60 is what you pay for all day unless the autoscaler can add capacity faster than the spike builds. Part 40 covers how to measure whether the open model meets the latency target at all; here the point is that the answer sets the fixed cost.

A fine-tuned model is not a one-time purchase. Behavior changes, the teacher changes, the customer's rules change, and each retraining generation costs reviewer time for new examples, a training run, and a full gate run on both models. Part 33 covers when retraining is warranted; for break-even, treat expected retraining as a recurring cost spread over the interval between generations.

Suppose the team expects to retrain quarterly at $6,000 in review and evaluation time per generation. That is about $66 per day added to the fixed term, which raises the dedicated break-even in the example above from roughly 29,600 to roughly 39,800 requests per day. A transfer that only pays back if it never needs retraining is not a transfer that pays back.

Break-even assumes the open model meets the quality bar on the traffic it serves. If it does not, the calculation is measuring savings on a worse product, and the cost of that shows up somewhere else: in escalations to humans, in lost conversions, in support load. The gate in part 17 and the parity definition in part 21 exist so that the savings number is computed only over traffic where quality held.

This is also why fallback share belongs in the arithmetic. A router that keeps quality by escalating aggressively lowers savings per request; a router that keeps savings by escalating rarely may be lowering quality. The two dials are connected, and the break-even volume moves when either one does.

Set the break-even volume as an explicit go or no-go threshold before curating any data. If sustained traffic is a fraction of that volume and unlikely to grow, the transfer should stop at the calculation. If traffic is well above it, the setup cost is small relative to what it returns, and the remaining question is whether the model can pass the gate.

Revisit the number after the load test and after the gate, because both change inputs: the load test replaces the estimated variable cost with a measured one, and the gate's escalation rate replaces the assumed fallback share. A break-even that survives both measurements is one you can present to a customer.

Converra measures cost and latency per model from real calls when it benchmarks candidate models on scenarios built from the agent's instruction, so the closed and open per-request costs in the calculation above can start from observed values on your agent rather than from a rate card. When a candidate reaches production, the model production test requires non-inferiority plus strictly lower cost before it promotes, and the model-swap verdict reports the cost delta per arm alongside parity verified, regressed, confounded, or insufficient data.

Capacity, headroom, retraining, and engineering time are your inputs; Converra does not estimate them for you and does not run the training itself. No open-weight production verdict for a customer agent has been published yet, so a break-even computed today is a plan until the cost delta is measured on the traffic it was planned for.