Vendor Lock-In Math: The Real Cost of Switching LLMs

"Just swap the API endpoint" is the most expensive sentence in production AI engineering. The five cost buckets below are taken from public engineering posts and pricing analysis. The dollar ranges are calibrated for solo to small-team operations (1-10 engineers); larger teams scale these up roughly linearly with prompt-suite size.

1. The five cost buckets of an LLM switch

An LLM switch in production touches five separable cost areas. Most cost estimates count only the first one and miss the next four.

• Bucket 1 — API integration. Replace SDK calls, environment variables, retry logic, response parsing. 1-3 days of engineering work for a typical product.
• Bucket 2 — Eval-suite redesign. Rebuild the evaluation harness for the new model. Existing eval thresholds are calibrated against the old model's behavior; a new model with different verbosity, refusal patterns, and reasoning style requires a new threshold per metric. 1-3 weeks of engineering work.
• Bucket 3 — Prompt rewriting. Anthropic and OpenAI respond differently to the same prompt. Claude responds well to detailed task framing and XML structure; GPT-4o responds well to bullet-style instructions and JSON schemas. A prompt suite of 20-50 routes requires 5-15 days of rewriting and re-eval.
• Bucket 4 — Latency and token-budget retuning. Output verbosity, time-to-first-token, and structured-output reliability differ. UX behaviors (streaming cadence, max output length, timeout limits) often require rebalancing. 3-7 days.
• Bucket 5 — Rollback risk window. 1-3 weeks of operating both vendors in parallel with traffic shadowing while the new vendor is validated. This is not a development cost, but it is a real cost in customer-impact risk and engineering attention.

Total range: $15,000 (small product, single primary route) to $200,000 (multi-route product with custom evals and high-stakes outputs). Most solo founders underestimate this cost by 5-10x because they only count Bucket 1.

2. Eval-suite redesign

An eval suite for a production LLM feature has three components: a fixture set (input examples), a scoring function (LLM-as-judge or rule-based), and a threshold (the score below which a regression is flagged). All three are calibrated against the current model's behavior.

Switching models breaks the calibration. A scoring function that returns 0.85 average score on Claude Sonnet may return 0.78 average on GPT-4o for the same fixtures, not because GPT-4o is worse but because it produces shorter, more direct outputs that the scorer rates as less complete. The threshold of 0.80 that was a safe regression bar on Claude is now a false-positive generator on GPT.

What survives a switch:

• The fixture set (input examples) — survives intact.
• Rule-based scorers (regex, JSON-schema validation, exact-match) — survives if outputs are structured.
• LLM-as-judge scorers — typically need re-tuning because judge prompts often reflect the original model's output style.
• Pass/fail thresholds — must be recalibrated, full stop.
• Cost and latency budgets — must be recalibrated.

For a solo founder running 5-10 evaluation routes with 50-200 fixtures each, the eval rebuild typically takes 1-2 weeks of engineering attention. For a team running 30+ routes, it takes 4-8 weeks. The cost is high enough that some teams choose to defer evals entirely during a switch, which trades short-term speed for long-term regressions.

3. Prompt rewriting and tuning

Anthropic and OpenAI documents prompt engineering differently because the models respond differently^[3][4]. The same prompt sent to Claude and GPT-4o produces measurably different outputs along three axes: structure, verbosity, and refusal behavior.

Concrete differences that drive rewrite work:

• Output structure. Claude responds well to XML-tagged output requests (<answer>...</answer>). GPT-4o responds well to JSON-schema structured outputs via the response_format parameter. A prompt suite using one convention requires translation to the other.
• System prompt placement. Anthropic puts the system prompt in a separate parameter and treats it with high salience. OpenAI puts the system prompt in the messages array. Long system prompts that work well on Claude often need restructuring to be honored by GPT-4o.
• Refusal behavior. The two models refuse different content. A prompt suite tuned to avoid Claude refusals may trigger GPT refusals on different content; the false-positive set is a different shape on each model.
• Few-shot sensitivity. Claude tends to follow few-shot examples literally; GPT-4o tends to generalize from them. A prompt with 5 examples tuned to Claude's literal-following often produces overgeneralized output on GPT-4o.

For a typical solo product with 10-30 prompts in production, a careful rewrite takes 5-15 engineering days. A naive rewrite (find-and-replace the API call) takes 1 day but produces measurable quality regressions on most routes.

4. Latency and token-budget retuning

Latency budgets and token budgets are calibrated against a specific model's characteristics. A switch breaks both.

Output verbosity. Claude tends to produce longer, more thorough outputs by default. GPT-4o tends to produce shorter, more direct outputs. A product with a 500-token output cap calibrated to Claude's verbosity may truncate every Claude response but leave GPT-4o responses with 200 tokens of headroom. The cap needs to be retuned per model.

Time-to-first-token (TTFT). Streaming UX is calibrated against a specific TTFT range. Claude Sonnet's median TTFT is around 200-400ms; GPT-4o's median is similar but with a heavier tail. Products with a "spinner-to-text" transition tuned to one model's distribution show visibly different UX on the other.

Per-token output rate. Streaming display speed is tuned to per-token rate. The rates differ enough between providers that hand-tuned typing animations look natural on one and jittery on the other.

Cost-per-output-token differences. At Anthropic's 5x input-to-output ratio (Claude Sonnet at $3 input / $15 output) versus OpenAI's 3x ratio (GPT-4o at $5 input / $15 output), the same prompt produces different cost profiles depending on whether the workload is input-heavy or output-heavy. A product that is profitable on Claude with high context and short outputs may be unprofitable on GPT-4o with the same shape because the input savings (Claude is cheaper) compound.

5. Downtime and rollback risk

Both Anthropic and OpenAI operate public status pages with historical incident data^[6][7]. Both providers see meaningful incidents per month — partial outages, elevated error rates, model rollouts that introduce regressions. A vendor switch is itself an incident class: the new vendor's failure modes are unfamiliar, and the rollback path takes longer than usual because the old vendor's keys may have been deactivated.

Mitigation patterns observed in production:

• Shadow traffic for 1-2 weeks. Route 5-10% of production traffic to the new vendor while keeping the old vendor as primary. Compare outputs offline. This catches behavior regressions before customer impact but doubles cost on the shadowed slice.
• Per-route progressive rollout. Switch one route at a time, lowest-stakes first. This isolates regressions to one feature. Full rollout takes longer but reduces blast radius.
• Keep the old vendor's API key active for 30 days. The rollback path remains warm. Cost is one extra month of vendor-relationship management.
• Pre-build a fallback router. If the new vendor fails, automatic failover to the old vendor on next request. Adds 1-2 weeks of engineering work to the switch but reduces incident severity by 5-10x.

The full rollback risk window (parallel running, monitoring overhead, on-call burden) typically lasts 2-4 weeks after the switch and costs the equivalent of one engineer's part-time attention.

6. Real production cases

LangChain published an engineering retrospective on switching LLM providers in production^[1]. Key findings: the team's initial estimate of "two days of work" expanded to roughly six weeks of engineering attention when the eval rebuild and prompt-rewrite costs were counted. The team reported a 12% accuracy regression on the highest-stakes route during the first week of the switch, recovered within four weeks of prompt tuning.

Replit publicly documented their decision to use Anthropic Claude for code-generation features^[2]. The post explicitly cites long-context performance (Claude's 200k context handling) and instruction-following on multi-file edit tasks as the reasons for not switching to alternatives that were cheaper per token. The implicit argument: the per-token cost difference (3-5x) was less than the engineering cost of switching plus the regression risk on a high-stakes coding workflow.

Stripe's engineering blog has discussed building reliable abstractions for LLM provider calls^[8]. The pattern they describe: a thin internal abstraction over provider-specific calls, used for telemetry and retries, but no full abstraction layer that smooths over prompt or output-schema differences. The reasoning: "leaky" abstractions that hide provider differences cost more in production debugging than they save in switch optionality.

7. When to switch, when to stay

Switch when:

• Per-token cost difference is more than 3x AND your monthly model spend is more than $5,000. Below those thresholds, the switch cost is larger than the savings over a 12-month window.
• The current vendor has a measured quality gap on your specific workload (run the eval suite on both vendors monthly to know).
• The current vendor has a feature gap that blocks a customer-requested feature (e.g., 1M-context window for a specific use case).
• The current vendor has had two or more incidents in 90 days that materially affected your customers, and the alternative has a better track record on the same routes.

Stay when:

• Per-token cost difference is under 2x. The savings will not pay back the switch cost.
• You operate fewer than 5 production routes. The switch cost is fixed; the per-route savings scale with route count. Below 5 routes, the math rarely works.
• You have not yet built a working eval suite for the current vendor. Switching before you can measure quality means you are switching blind.
• Your customers are paying for outputs whose quality is hard to measure (long-form generative content, creative writing, high-context analysis). The regression risk is higher and the recovery path is slower.

8. The dual-vendor architecture

The middle path that most production teams converge on: one primary vendor for 90-95% of traffic, one secondary vendor for 5-10% of traffic on the same routes for ongoing comparability. The secondary vendor is not a fallback (those add operational complexity for failure modes that rarely materialize); it is a measurement instrument.

The architecture:

• One thin internal abstraction (3-5 functions) wrapping the vendor SDKs. Not a LangChain-scale provider abstraction. Just enough to swap vendors at the call site.
• Per-route configuration of which vendor handles primary traffic.
• Sampling rate (5-10%) of routed-to-secondary requests for comparison.
• Eval suite that runs on both vendors weekly with the same fixtures.
• A dashboard that shows quality, cost, and latency on both vendors per route.

The cost of this architecture is roughly 1-2 weeks of engineering work upfront and 1-2 hours per week of ongoing monitoring. The benefit: when a switch becomes attractive, you have already paid most of Bucket 2 (eval) and Bucket 4 (latency tuning) on the secondary vendor. The remaining switch cost drops from $50k-$200k to $10k-$30k.

Vercel's AI SDK ships a useful starting point for this abstraction^[5]. It exposes a common interface across Anthropic, OpenAI, Google, and others without trying to abstract away prompt-engineering differences. It is the closest thing to a "thin enough" abstraction that pays back at solo scale.

Vendor lock-in to a single LLM provider is real, costs more to undo than to acknowledge, and is best managed by keeping one primary vendor with a small, measured secondary stream rather than by chasing perfect abstraction. The math says: the cheapest path is to pick well once, measure ongoing, and switch only when both the cost-per-token differential and the eval-quality differential cross the threshold simultaneously.