Vendor Lock-In Math: The Real Cost of Switching LLMs

"Just swap the API endpoint" understates every LLM migration. The model is the most visible dependency, but the evaluation suite, the prompts, the output limits and the operational runbook were all tuned to it. This guide lists what a switch touches, shows how to put a number on it with your own inputs, and sets out when the move pays.

1. The five cost buckets of an LLM switch

• API integration. SDK calls, environment variables, retry logic, response parsing. Usually the smallest bucket.
• Evaluation suite. Thresholds calibrated on the old model's outputs need recalibrating; model-graded scorers may need their prompts retuned.
• Prompt rewriting. Each production prompt needs to be re-tested on the new model and, where it regresses, rewritten.
• Output and latency budgets. Output-length caps, timeouts and streaming behaviour set for one model often need retuning for another.
• The parallel-running window. Running both vendors while the new one is validated costs engineering attention, on-call load and, on shadowed traffic, double the model bill.

Estimates that count only the first bucket understate the switch. Count your production prompts and evaluation cases before quoting a number to anyone.

2. Pricing a switch with your own numbers

The LLM vendor lock-in calculator turns that inventory into hours and dollars, then compares the switching cost with the monthly saving the new vendor offers. Its example: an $8,000 monthly model bill, a prompt-complexity score of 6, 80 evaluation cases, 40 hours of other retraining work at $150 an hour, two days of degraded service against $8,000 of monthly revenue at risk, and a new vendor 30% cheaper. The calculator's own assumptions convert prompt complexity and evaluation count into hours; replace them with your estimates where you have better ones.

The result: 104 engineering hours ($15,600) plus about $533 of revenue at risk, $16,133 in total, about two months of the current model bill. A 30% discount saves $2,400 a month, so the switch pays back in about 6.7 months. Halve the discount and the payback doubles; that sensitivity is why the saving has to be checked against list prices first.

3. Eval-suite redesign

An evaluation suite has three parts: a fixture set of inputs, a scoring function (rule-based or model-graded) and a threshold below which a change counts as a regression. All three were calibrated on the current model.

• The fixture set survives a switch intact.
• Rule-based scorers (schema validation, exact match, regular expressions) survive if the outputs stay structured.
• Model-graded scorers often need retuning, because their prompts were written against the old model's output style.
• Thresholds and cost and latency budgets have to be recalibrated on the new model.

Skipping the evaluation work during a switch trades a faster migration for regressions you will not see until customers do.

4. Prompt and API differences

Some differences are documented by the vendors themselves. Anthropic's prompting guide recommends structuring prompts with XML tags^[1]. Both Anthropic and OpenAI now offer structured outputs that follow a JSON schema you supply, through different request parameters^[2][3], so schema-based code ports more easily than free-text parsing but still has to be rewired.

Other differences have to be measured, not assumed: output length, refusal behaviour and how literally a model follows examples differ between vendors and between versions of the same vendor's models. Run your fixtures on the candidate model and compare before rewriting anything; some prompts will need no change and others a full rewrite.

5. Downtime and rollback risk

Both vendors publish status pages with incident histories^[5][6]; check them for the routes and models you depend on. A migration adds its own risk: the new vendor's failure modes are unfamiliar, and rollback is slow if the old integration has been removed. Common mitigations:

• Shadow traffic. Send a small share of requests to the new vendor while the old one stays primary, and compare outputs offline. It costs double on the shadowed slice.
• Route-by-route rollout. Move the lowest-stakes route first, so a regression is confined to one feature.
• Keep the old integration live. Leave the previous vendor's keys and code path in place until the new one has run cleanly for a few weeks.
• A fallback path. Automatic failover to the other vendor on errors turns a vendor incident into degraded service rather than an outage.

6. When to switch, when to stay

Switch when:

• The calculator's payback is well inside your planning horizon, on a saving you have confirmed against the vendors' list prices or a written quote.
• The new model matches or beats the current one on your own evaluation suite, route by route.
• The current vendor lacks a capability customers need, or its incidents have repeatedly hit your users.

Stay when:

• The saving is small or uncertain: at list price, the mid tiers of the two largest vendors currently cost the same.
• You have no working evaluation suite for the current vendor. Switching without one means you cannot tell whether quality held.
• Your outputs are hard to evaluate (long-form generation, analysis), so regressions are slow to detect and expensive to recover from.

Check the downside of staying too: the model price drop stress test shows what a competitor's price cut does to your margin if you cannot move, and the AI stack cost calculator puts the model line in context.

7. The dual-vendor architecture

The middle path: one primary vendor for most traffic and a second vendor on a small sample of the same routes, used as a measurement instrument rather than a fallback.

• A thin internal wrapper around the vendor SDKs, just enough to choose the vendor per call site.
• Per-route configuration of the primary vendor.
• A small sampled share of requests sent to the secondary vendor for comparison.
• The evaluation suite run on both vendors on a schedule, with the same fixtures.
• A dashboard of quality, cost and latency per vendor and route.

The point is that when a switch becomes attractive, much of the evaluation and tuning work is already done on the secondary vendor. Vercel's AI SDK is one starting point for the wrapper: it gives a common interface across Anthropic, OpenAI, Google and others without hiding the prompt-level differences^[4]. Pick well once, measure continuously, and switch when the saving and your evaluation results both clear the bar.

Frequently asked questions

How much does it cost to switch LLM providers?

It depends almost entirely on how many prompts and evaluation cases you have to rebuild, so estimate it from your own inventory rather than from a published range. The API code is usually the small part; recalibrating the evaluation suite, rewriting and re-testing prompts, retuning output limits and running both vendors in parallel during the cut-over are the large parts. In the LLM vendor lock-in calculator's example (an $8,000 monthly model bill, 80 evaluation cases, 40 hours of retraining work at $150 an hour and two days of degraded service), switching costs about $16,100, two months of spend, and a 30% cheaper vendor pays it back in about 6.7 months.

Why isn't switching LLM providers just changing the API endpoint?

Because the rest of the system is tuned to the old model. Evaluation thresholds were calibrated on its outputs, prompts were refined against its behaviour, output-length caps and streaming UX were set for its responses, and the request formats differ between vendors (for example, both Anthropic and OpenAI now support JSON-schema structured outputs, through different request parameters). Each of those has to be re-checked on the new model before customers see it.