When Is Moving to Open Weights Not Worth the Operational Complexity?

A closed model arrives as an API with an SLA, a bill per token, and no on-call. An open model on your own endpoint arrives as a service you run: capacity planning, autoscaling, security patching, driver and serving-stack upgrades, quantization decisions, fallback logic, drift monitoring, and periodic retraining with its own gate. The per-token price is lower. The operating cost is new, recurring, and mostly people.

That trade only pays when the traffic is large enough, the workload narrow enough, and the ownership clear enough that the operating cost is small relative to the saving. The rest of this part goes through each of those conditions as a stop signal. Any one of them is enough to change the recommendation; two or more make the answer clear.

A dedicated endpoint bills by the hour whether or not requests arrive. On-demand H100 capacity was listed at eight dollars per hour on the Fireworks pricing page on 2026-09-23, as one illustrative reference. At a few thousand requests a day, that endpoint spends most of its hours idle, and the effective cost per request can exceed the closed model's per-token bill. Part 27 works through where the break-even sits; below it, the transfer costs money in exchange for complexity.

Serverless open endpoints that bill per token change this arithmetic and remove most of the operations, at the cost of less control over latency, version pinning, and data handling. If per-token open serving meets the workload's needs, it is usually the right first move at low volume, and it can be tested without any of the infrastructure a dedicated endpoint requires.

Fine-tuning transfers behavior: format, conventions, tool choice, abstention. It does not transfer knowledge the smaller model lacks, and it does not extend reasoning to shapes the training data never showed. An agent whose value comes from the closed model knowing things, or from following a long chain of reasoning across a varied long tail, keeps that value only on the closed model. Part 3 lays out the distinction request type by request type.

The tell is in the gate design. If the team cannot describe a bounded set of request classes with a checkable outcome for each, the workload is probably open-ended, and an open model will pass the cases it was shown and fail the ones it was not. Routing can rescue part of such a workload; full replacement cannot.

An open-weight deployment needs a person who owns the endpoint, the fallback policy, the drift thresholds, and the retraining decision, with time allocated every week. Not a team in the abstract, and not the engineer who happened to run the first experiment. When that person does not exist, the model is deployed once and never gated again, the fallback path rots, and the first serious incident becomes a scramble to remember how the thing was built.

Ask who will be paged when the endpoint saturates at two in the morning and who decides whether a drift alert justifies a retrain. If the answers are vague, the recommendation is a hosted endpoint or a cheaper closed tier, where the provider owns the first question and the second one does not arise.

Every parity claim in this series rests on a frozen evaluation set with enough distinct cases, seeded hard cases, all three evidence conditions, and a pre-registered margin. Building that set takes reviewed production runs, an outcome definition the customer agrees to, and the discipline to keep the holdout protected. Parts 17 through 19 describe the work. When a team cannot or will not do it, the transfer has no way to know whether it succeeded.

Shipping an open model on a benchmark score and a sample of good-looking outputs is a bet with the customer's users as the stake. If the gate cannot be built, the honest position is that the transfer is not ready to be evaluated, and a decision that cannot be evaluated should default to the incumbent.

Training a customer-facing model on a closed provider's outputs runs into that provider's terms; Anthropic's and Google's terms reviewed for part 42 restrict using their services to build competing models, and OpenAI's contain a comparable restriction. The base model's license carries its own obligations. The customer's own data rights may not allow their runs to train anything, and standard vendor terms, including Converra's, forbid training on customer data without a written election.

These are resolvable, and part 35 and part 42 describe how. They take time and counsel. When the timeline for the transfer is shorter than the timeline for the written review, the transfer should wait, and the interim saving should come from an option that needs no training data at all.

Suppose the full cost stack from part 26 shows the open model saving a modest share of the agent's spend after fallbacks and hosting. Set that against the cost of one bad month: a quality regression caught late, a rollback, a confounded verdict, the engineering hours, and the customer conversation. If the expected saving over a year does not comfortably exceed that, the transfer is a poor bet even if it would technically work.

The table below collects the stop signals from this part. Each one is checkable in a few days from data a team already has, which is the point: the decision not to transfer should be cheap and early, not discovered after training.

A cheaper closed tier from the same provider needs no training data, no endpoint, and no new fallback logic; a benchmark on the agent's own scenarios can show in an afternoon whether it holds quality. A targeted prompt change can cut tokens per request more than a model switch. Prompt caching, shorter tool results, and trimming retrieval context all lower spend without changing the model at all.

A hosted open endpoint billed per token sits between these and full self-hosting. It captures most of the price advantage, keeps version pinning and adapter ownership, and leaves serving operations to the provider. For many teams it is the right long-term home for a fine-tuned model, and the decision to run dedicated capacity can wait until volume makes it obviously worthwhile.

Day one: pull the agent's request volume, tokens per request, and latency budget, and compute the break-even against dedicated and serverless open serving. Day two: classify a sample of requests by whether they need closed-model knowledge or a bounded, checkable behavior. Day three: name the owner and confirm their weekly time, or fail this step. Day four: benchmark two or three cheaper catalog models and one untuned open model on the agent's own scenarios. Day five: check the provider terms and the customer's data rights against the training plan.

If the week ends with a stop signal, write down which one and what would change it, then take the alternative. If it ends clean, the transfer proceeds to part 5's intake and part 17's gate design with a decision record that says why. Either way, the expensive parts of the transfer have not started, and the reasoning is on file for the next time volume or the workload changes.

Converra's model benchmarks run candidate models, including cheaper closed tiers, against scenarios generated from the agent's own instruction, measure quality, cost, and latency from real calls, and open a pull request with the winning switch. That is the day-four check in this part, and for many agents it is the whole answer: a catalog model that holds quality at lower cost needs no training data, no endpoint, and no new operations. Converra's prompt optimization loop covers the prompt-change alternative, with fixes tested in simulation and verified on production traffic.

For the transfers that do proceed, Converra's fine-tuning workflow curates reviewed runs into versioned datasets, records the training job and gate result, holds the switch behind approval, and issues a parity verdict on real traffic with the cost delta per arm; training and serving run on the customer's provider or infrastructure. Converra does not train on customer data without a written, tenant-exclusive election, and no open-weight production verdict for a customer agent has been published. Whether a given agent should make this move is a measurement, and this part is the list of reasons the measurement may say no.