The best local coding setup isn't one model: how I route across Claude, Kimi and my own rig

Everyone asks which local model is best for coding. It's the wrong question. The one that got me a setup I actually trust was: which model should do which part of the job?

Because the answer isn't a model, it's a routing rule. The expensive frontier model scopes the work and assembles the result. A cheap, huge-context model builds the modules. And a coder model on my own hardware does the bulk volume - the boilerplate, the tests, the reviews - for the price of the electricity. Three tiers, one pipeline, and the token bill lands on judgement rather than typing.

This is the whole thing, wired up: what each tier is, how they're plumbed together, and the rule that decides where a job goes.

The three tiers

Think of it as a workshop, not a single tool. Three people, different rates, different jobs.

Frontier (Claude) - the expensive one

This is where I spend the money, and only here. Claude does the reasoning: scoping the work, holding the architecture in its head, assembling the pieces, checking the result. It's the tier where being wrong is costly, so it's the tier worth paying for. It also does the thing none of the others can - reaches for tools, reasons across a dozen files at once.

Kimi (Moonshot) - the build tier

A strong coding model with a one-million-token context window, at API prices a fraction of frontier. I use it for build units: a whole module, a long refactor, reasoning over a corpus too big to hold in my head. I paste everything - the entire file tree, the full doc - because the context is enormous and the tokens are cheap. Stinginess here is a false economy.

The local coder - the free one

A Qwen3-Coder-Next 80B model, quantised to AWQ-8bit, running on two modded 48GB RTX 4090s in my office through vLLM. It decodes at 118 tokens per second measured, about 315 across four concurrent requests. Because it's my hardware, its output costs nothing but power, so I put the volume work here. Boilerplate, format conversion, extraction, commit messages, a first-pass code review, tests generated from a spec. Anything bounded and bulky, it does for free, all day, and I let it.

The trade the whole setup is built on: expensive tokens go on judgement, not typing. Every bounded, well-described piece of work looks for an off-ramp to a cheaper tier before it touches the frontier bill.

The routing rule

One line governs the whole thing:

Frontier scopes. Kimi builds. The local coder evaluates. Frontier assembles.

In practice, a real job flows like this. I hand Claude a task - say, add a feature across a few files. It reasons about the shape of it, decides what needs to exist, and writes the spec. That spec, plus the whole relevant file tree, goes to Kimi to build the module, because Kimi can hold all of it at once and it's cheap to let it. The result comes back and the local coder reviews it against the spec and writes the tests - volume work, free. Claude takes the reviewed pieces, assembles them into the codebase, and does the final verification with its own eyes and its own tools.

Nobody works above their pay grade, and nobody works below it. The frontier model isn't writing boilerplate; the local model isn't making architecture calls. That division is the entire point.

The plumbing

Here's the part people assume is complicated. It isn't - I wired mine up in an afternoon. The three tiers sit behind one endpoint, and the model is chosen per call.

The connective tissue is houtini-lm - an MCP server I built so Claude can delegate work off its own token bill - pointed not at a model but at a LiteLLM router. The router is the switchboard: one OpenAI-compatible address, several models behind it, selected by name on each request.

# the router's model list (LiteLLM)
- model_name: local              # -> my vLLM rig, Qwen3-Coder-Next 80B
    api_base: http://host.docker.internal:8000/v1
- model_name: kimi-k2.7-code     # -> Moonshot, 1M context
    api_base: https://api.moonshot.ai/v1
- model_name: kimi-k3
- model_name: deepseek-v4-flash  # -> DeepSeek, cheapest API tier
- model_name: deepseek-v4-pro

houtini-lm's endpoint points at the router (http://127.0.0.1:4000), and a call names the model it wants - local for the free rig, kimi-k2.7-code for a build unit, deepseek-v4-pro for a cheap-but-capable middle. Swap which model local resolves to by swapping the vLLM preset; the router doesn't need to know. Adding a hosted model is a few lines and its API key.

That's the whole architecture. One MCP server, one router, three tiers of model reachable by name. No orchestration framework, no agent mesh - just a switchboard and a naming convention.

The local tier in detail

The local coder is the piece that makes the economics work, so it's worth being specific about what it takes.

The rig is two RTX 4090s modded to 48GB each - 96GB of VRAM total - serving Qwen3-Coder-Next 80B through vLLM in Docker. The vLLM setup and the tuning that got it from 18 to 133 tokens per second are their own articles; the short version is that an 80B mixture-of-experts model at 4-bit fits across the two cards and serves fast enough that it doesn't feel like waiting.

The numbers, measured on my own bench rather than quoted from anyone: 118 tokens per second on a single request, 315 across four concurrent, first token in around a third of a second. That concurrency figure is why it handles a fan-out of review-and-test jobs without queueing. And because the marginal cost of a token is zero, I let it run freely - if a job is bounded and I can describe it in a sentence, it goes local first without much thought.

The catch is that this tier costs money up front and quiet in the room. Two modded cards, a workstation to hold them, and the patience to tune a vLLM config. If you don't want the hardware, the same routing works with the local tier pointed at a cheap hosted coder instead - you lose the free-volume economics but keep the shape.

Running the experiment

I'd been telling people this setup works, and telling myself the hosted tiers earn their place, on numbers I hadn't taken myself. So I took them. One question: on a bounded coding task, does the tier you pick change whether the code is right, or only what it costs and how long you wait?

The method

I wrote one small task - a parse_duration function that turns "1h30m" into 5400 seconds and refuses junk like "abc" or "1x" - and fired the identical prompt at all five tiers through the router, one call each. Then I scored every answer against a dozen cases: the valid conversions, and the invalid strings it had to reject.

Nothing elaborate. Same prompt, same temperature where the model allowed it - Moonshot's Kimi locks temperature to 1, which is its own small reminder that "OpenAI-compatible" is a family of dialects, not a standard. One sample per model, so this is a smoke test of the wiring and the shape, not a benchmark suite. I wanted the felt difference, not a leaderboard.

The results

Here's what came back:

Two things jump out.

First, they all passed. Every model, every case, edge rejections included. On a task you can describe in a sentence, the "which model is best" question buys you nothing - they all write correct code. That is the whole justification for routing instead of hunting for one best model: pick the cheapest tier that clears the bar, because they all clear it.

Second, the local rig was fastest, by a distance, for free. 1.9 seconds against 11 to 26 for the hosted models. The reason is the bit worth knowing. Every hosted model here is a reasoner - it spends a few hundred to 1,500 tokens thinking to itself (reasoning tokens, the model's private working-out) before it writes a single visible character. My local coder doesn't. It just answers. So the tier that costs nothing also came back first, which is a good day.

Watch the decode column as well. DeepSeek V4 Flash decodes quickest of the hosted lot at 97 tok/s and still finished among the slowest, because it thought the longest. Tokens per second isn't time-to-answer once a model reasons first. Across a fleet doing high-volume bounded work that wait compounds on every call - one more reason to keep the bulk on hardware that answers straight away.

What I haven't proven yet

Two edges I'll own, now the experiment's done.

The first is Kimi's build role. The race above proved all five tiers write correct code, and it told me what each one costs in time. But a six-line parser doesn't go near a million-token context window, and that context is the whole reason I send Kimi entire modules. So that leg still runs on its published scores, not my stopwatch. Reported, not measured.

There's a second edge, and it's honesty of a different kind: the pipeline is a rule I follow, not a fully automatic loop. The tiers are wired and reachable by name; the decision about which tier a job goes to is still mine, made per task. Automating that routing cleanly - so the pipeline picks the tier itself and it holds up on real work - is the next thing, and I don't think it's a solved problem yet.

Where to start

You don't need three tiers on day one. You need the habit that makes them worth having.

Start by noticing which of your coding tasks are bounded - the ones you could describe to a competent junior in a sentence. Those are the ones that shouldn't be costing you frontier tokens. Point houtini-lm at a single cheap model - hosted is fine, a local rig is better if you've got one - and send those tasks there. Watch the token counter in the footer; the offloaded number climbs faster than you'd expect.

Add the router when one model stops being enough - when you want a huge-context build tier and a free-volume tier and the expensive brain, each for what it's best at. At that point the setup here is a config file and a naming convention away.

The best local coding setup was never going to be one model. It's a workshop where the right worker does each job, and the expensive one only does the work that's actually expensive to get wrong.