New experiments by email, as I publish them. No spam, unsubscribe anytime.
What I've tested, and what it found
These are the experiments I've run and written up. The local models are measured on my own hardware, the job market is read from live data, and the frontier models are priced live from OpenRouter's public list. Each section links through to the full write-up, and where a number is reported rather than measured by me, I've said so.
Local models on two modded RTX 4090Ds
On 26 Sep 2026 I put 21 setups of 13 open-weight models through one fixed protocol, three runs each, on two modded RTX 4090D 48GB cards running vLLM 0.26.0. When I'm working with clients, I don't tell them they need to start by investing in an array of NVIDIA Blackwell RTX Pro 6000s, because the investment's huge. From an engineering point of view, the discipline is making the most out of the equipment you have. The method and the test harness, vllm-bench, are public on GitHub.
ConclusionThe build and the quant (how far the weights are compressed) moved speed nearly threefold on the same model: on one card, Qwen3.6-27B ran at 16.6 tokens per second (tok/s) in its official block-FP8 build and 47.4 in the AWQ INT4 build I serve. The fastest model in the sweep, LFM2.5-1.2B at 309.5 tok/s, passed only 6 of the 12 test cases on the coding task.
What AI is doing to hiring, from live job data
I joined the live job ads in YubHub, my own jobs index, to Anthropic's Economic Index, which measures the share of an occupation's tasks that already turn up in real Claude conversations. Then I cross-checked the result against LinkedIn's Work Change Report and BCG's analysis. I wanted to name what AI is doing inside specific jobs, and to say so where the data disagrees with the headlines.
ConclusionIn this data, employers are still hiring in volume for the most AI-exposed roles, which reads as augmentation rather than displacement. Stanford's figures show a 16% relative fall in employment for 22 to 25 year olds in the most exposed jobs. It's a real effect, but it's smaller than the headlines make out, and other causes are tangled up in it.
What frontier models cost, and how I route work between them
The local fleet exists to take work off paid APIs, so I need to know what those APIs cost and how well they score. The benchmark scores come from public leaderboards rather than my own runs, but the three tiers of routing further down are mine.
| Model | $/Mtok in / out | SWE-bench Verified | The one-line read |
|---|---|---|---|
| Claude Fable 5.1 | $10 / $50 | - | Anthropic's top tier, for long-horizon agent work |
| Claude Opus 5.5 | $4 / $20 | - | The orchestrator in my own routing |
| Kimi K3 | $0.19 / $15 | 76.8% | Vision in the agentic loop - fixes CSS from screenshots |
| DeepSeek V4 Pro | $1.6 / $3.2 | 80.6% | The build tier in my own routing |
| Qwen Coder (latest) | $0.3 / $1 | - | The open-weight Qwen coder line, hosted |
| Model | $/Mtok in / out | Context | The one-line read |
|---|---|---|---|
| Claude Opus 5.5 | $4 / $20 | 1M | Orchestration, architecture and hard debugging |
| Claude Sonnet 5 | $2 / $10 | 1M | The mid-price workhorse |
| GPT-6 Astra | $10 / $50 | 1.1M | OpenAI's top tier |
| Gemini 3.1 Pro | $2 / $12 | 1M | Grounded search in the loop - my research workhorse for current-state questions |
| MiniMax M3 | $0.3 / $1.2 | 1M | Budget text throughput; agentic scores trail on public leaderboards |
| Model | $/Mtok in / out | Context | The one-line read |
|---|---|---|---|
| Kimi K3 | $0.19 / $15 | 1M | Open weights at frontier coding scores, as reported |
| DeepSeek V4 Pro | $1.6 / $3.2 | 1M | The hosted open-weight model my build tier runs on |
| DeepSeek V4 Flash | $0.091 / $0.18 | 1M | Bulk pipelines at pennies per million tokens |
| Qwen3.6 27B | $0.45 / $2.7 | 262K | The local evaluator in my routing, served on my own rig |
| Llama 3.3 70B | $0.1 / $0.32 | 131K | The open-weight baseline most hosts carry |
| gpt-oss-20B | $0.03 / $0.13 | 131K | OpenAI's open-weight small model - near-free at volume |
Prices live from OpenRouter's public model list · * = curated snapshot where unlisted · benchmarks and reads as reported
ConclusionThis routing is in use now: Claude scopes the job, DeepSeek V4 Pro builds it, a local Qwen3.6-27B checks it, and Claude assembles the result. When the Qwen model isn't loaded on my cards, DeepSeek V4 Flash does the checking, on non-sensitive work only. A model that reasons at length spends more tokens on every job, so a lower price per token doesn't always mean a lower bill.
The local fleet runs free on my own two cards and handles evaluation, bug-checks, boilerplate, tests written from a spec and extraction, and it's the only tier I use for sensitive work.
DeepSeek V4 Pro has been the build tier since 24 Jul, when it passed all 12 test cases on one bounded coding task sent through my model router, in a single run.
Claude takes the jobs where being wrong costs more than the tokens.
Other things I've tested
Some of what I test doesn't fit those three tracks. Here are five of them.
Memories in my Claude Code folder were pointing at procedures I'd moved or rebuilt, and two sessions pruned the same folder on 25 Sep. So I wrote a lint that runs seven checks over the files without changing any of them, and a prune that retires stale memories rather than deleting them. Across 129 files, the first run found 5 broken links; after the fixes it found none.
Once an article goes live, it starts to go stale. The Daily Visual, a site I built around that problem, rebuilds its charts, data and the numbers in its sentences every morning, pulling from 11 free feeds across 9 publishers. By 23 Sep it was carrying 23 stories across 5 topics, drawing on 56 data series.
I cut my Content Radar SaaS, which had no users, down to a free one-page tool, then ran one brief through eight engines to test demand before paying to run it. On the free tiers, the limit showed up as waiting, not as worse ideas, with each engine taking anywhere from about 15 seconds to just over a minute to answer.
Over 31 days of my logs, an active Claude Code day averaged about $139 at Opus 5.5 list prices. On one grunt task, DeepSeek V4.1 Flash was about 17x cheaper per output token, but because it reasoned at length it was only 2.4x cheaper per job at peak-hour prices and about 4.7x cheaper off-peak ($11.15 and $5.57 against $26.24 per 1,000 jobs). On my own machine, I've watched houtini-lm, my MCP server for handing work to other models, save over 150,000 tokens across roughly 200 calls.
I pointed YubHub at hiring inside the AI labs themselves: 7,325 enriched listings from 82 AI companies, pulled on 15 Mar 2026. Senior roles made up 48% of all the listings against 18.6% at entry level, and the hiring at Anthropic, OpenAI, xAI, DeepMind, Mistral and ElevenLabs pointed to six quite different strategic bets.