3 articles tagged docker.
In today's post I'm sharing how I benchmark local models on vLLM by changing one thing at a time, along with the traps that quietly skew the numbers and my results from 21 setups on two 48GB RTX 4090Ds. The harness, vllm-bench, is free, so you can run the same tests on your own card.
local-llm · vllm
A lot of MCP servers get pulled straight off npm and run with your permissions, which makes them a supply-chain risk. In today's post we're taking a closer look at why I run mine in Docker containers, how to set up Docker's MCP gateway for Claude Code and Claude Desktop, and what a container won't stop.
Claude · mcp
vLLM doesn't run on Windows natively, so this is the Docker Desktop and WSL2 route my rig has served on since July: the compose file, the image pin that stops it upgrading itself, the walls WSL2 puts in your way, and how the settings were found, with Claude writing the test harness.