Tag All articles

Tagged: local-llm

8 articles tagged local-llm.

vLLM settings for a pair of RTX 4090s: the flags I run for twelve local models, and why

The flag-by-flag vLLM settings page for a dual-4090 rig: the baseline every preset shares, the six flags that change per model, the rig discipline you set once, and one verified launch block for each of twelve open-weight models with the reason attached.

· local-llm · vllm
The VRAM traps: why a 16GB model wouldn't load on a 48GB card

A 16GB model would not load on my 48GB card, and the reason was a shortfall of one kilobyte in a memory no spec sheet mentions. These are the fit traps a VRAM figure will never warn you about.

· local-llm · vllm
The broken rules of local LLM inference

I used to lock the clocks on my mining GPUs. The same instinct just helped kill five rules of local LLM inference on a £6,200, 96GB rig.

· local-llm · vllm
The best local coding setup isn't one model: how I route across Claude, Kimi and my own rig

The question I get asked is which local model is best for coding. Wrong question. The setup that works routes three tiers - Claude reasons, Kimi builds, and a Qwen coder on my own rig does the volume for nothing. Here's the whole thing, wired up.

· claude-code · local-llm
How to set up vLLM in Docker on Windows (WSL2): serve an open-weight model on your own GPU

vLLM doesn't run on Windows natively, so this is the Docker Desktop and WSL2 route my rig has served on since July: the compose file, the image pin that stops it upgrading itself, the walls WSL2 puts in your way, and how the settings were found, with Claude writing the test harness.

· local-llm · vllm
Moving houtini-lm to vLLM: What I learned

I decommissioned Hopper (my local LLM bootstrapped server) and moved my local models to a two-card 4090 rig with vLLM on Docker. It's so much faster - but houtini-lm spat its dummy. Two bugs, one hiding behind the other, and how v3.2.1 fixes it.

· claude-code · mcp
Best GPUs for Running Local LLMs (2026): VRAM, Bandwidth & Picks

In today's guide, we're picking the best GPUs for running local LLMs - why memory bandwidth beats FLOPS, why VRAM decides everything, what I'd buy at each budget from the used RTX 3090 to my own pair of modded 48GB 4090s, the PCIe lanes a second card needs on a gaming board, and a calculator to check whether your model fits.

· local-llm · gpu
How to set up LM Studio: GPU offload, developer mode and your first local model

Turn a gaming GPU into a working local AI model with LM Studio - free, no billing, no API key. In today's guide, we install it, point it at the GPU, download a first model, and then go through the settings: GPU offload, context length, Flash Attention, temperature and top-k, and the Developer mode that unlocks them.

· claude-code · local-llm

Tagged: local-llm

vLLM settings for a pair of RTX 4090s: the flags I run for twelve local models, and why

The VRAM traps: why a 16GB model wouldn't load on a 48GB card

The broken rules of local LLM inference

The best local coding setup isn't one model: how I route across Claude, Kimi and my own rig

How to set up vLLM in Docker on Windows (WSL2): serve an open-weight model on your own GPU

Moving houtini-lm to vLLM: What I learned

Best GPUs for Running Local LLMs (2026): VRAM, Bandwidth & Picks

How to set up LM Studio: GPU offload, developer mode and your first local model