Model Agnostic, on Purpose

On Friday, Anthropic disabled Fable 5 and Mythos 5 for every customer overnight after a government export-control order citing national security. Whatever you think of the merits, the operational story is the same, a model you depend on can vanish overnight by forces you don't control. A government order. A price change. A rate limit. A vendor deciding to go a different direction.

Everyone is suddenly talking about vendor lock-in this week like it's a fresh idea. I've been working on being vendor agnostic for quite some time. Anyone can do this if they want.

test against your own work, not the leaderboard

I have a job that runs every night and reads my meeting transcripts, then pulls out who decided what and who owns which follow-up. The model behind it is not sacred. It's whoever wins in an A/B test I run myself.

My own metrics, not a benchmark that tells you how a model does on someone else's homework. I want to know how it does on mine. So I score candidates against my actual transcripts, with my actual messy cross-talk and half-finished sentences. Last round, a local model called Qwen3-30B pulled about 49 relationships per transcript against the old one's 23 which happened to be a Gemma3-27B. Roughly double, and three to five times faster. I went through the extra ones by hand to make sure they were real and not invented. They held up. So I swapped it in.

That swap was one line of config. The engine behind a core part of how I keep track of my work changed, and nothing else had to. That's how you avoid lock-in. The model is a part of the system, not the foundation.

be willing to say no, including to free

People hear "vendor-agnostic" and think it means use whatever's local, or whatever's cheapest. It doesn't. It means test, then keep what actually wins, and be honest when the answer is no.

I've said no to free. I was able to get Google's Gemma 4 31B for free, which is exactly the kind of thing that's tempting to just adopt. I tested it anyway. It ran five to nine times slower than what I already had, and it leaked its own scratch-pad thinking into the output, which broke the clean format my pipeline needs. Free, and it still failed the actual job. Easy no.

I've said no to genuinely good, too. I tried Cohere's North Mini Code as a possible brain for some of my pipelines. Capable model, no complaints about the quality. But it was five times more wordy for the same answer, and it leaked stray formatting bits that broke my code. Great model, wrong job for me. I kept what I had.

The reason saying no works is that I had a real test. Does it do my job, on my data, cleanly? That question beats hype, beats free, and beats whatever's at the top of the leaderboard this month.

open is not the same as safe

Now that everyone's suddenly shopping for open-weight models, local on your own hardware is one way to stay free of any single vendor. It is not the only way. The one thing you can do is just refuse to source every model from the same place.

So I test widely. I've pulled weights straight off Hugging Face, run models through a router like OpenRouter, and gone to the labs themselves such as Nous and Arcee. I don't have a favorite. I have a shortlist I keep poking and testing.

Because "open" doesn't mean "safe." An open-weight model sitting behind one API I don't own is still a single point of failure. The price can move, the endpoint can go quiet, the terms can change on a Tuesday. Openness isn't the protection. The protection is that I've already run three or four candidates against my own work, so on any given Friday I'm holding a bench, not a bet.

build so no single model is load-bearing

My family health agents run on a local model on my own hardware by default, with cloud models like Claude as a fallback rather than the foundation. Same for the nightly transcript job. I have a simple rule for myself. Reach for local first, and only spend on a paid API when the task actually needs it.

The point isn't local for its own sake. The point is that every model, local or cloud, sits behind a layer I can swap. So when one fails, and one always eventually fails, the work doesn't stop. A rate limit, a price hike, a deprecation, or this week, a literal government recall. The people scrambling on Friday were mostly the ones who'd welded their whole workflow to one model from one vendor. My nightly job didn't notice anything, because to it, the model is a single line I can change in about ten seconds.

You don't need a home lab in your office to do any of this. I happen to have one because I'm a giant nerd about it. Treat any model, even the best one you've ever used, as a part you can replace. And actually test the replacement against your own work before you need it, not the morning a directive comes down. Lock-in feels like convenience right up until the day the thing you locked into walks out the door.