Operating · accepting briefs Work with me

Tagged: Qwen

4 articles tagged Qwen.

How to Run Qwen3-Coder-Next Locally: My vLLM Settings for Two 48GB RTX 4090s

I loaded Qwen3-Coder-Next across my two modded 48GB RTX 4090s with vLLM, and today's post covers the vLLM settings for 192k of context, and how it did on the jobs I use AI for. It answered questions on a whole 86k-token repo in 26 seconds, where Qwen3.8-27B on its 32k one-card preset couldn't take the prompt at all, and it scoped an upgrade in under 30 seconds against about 3 minutes for the 27B. But it got facts wrong, and given git tools it committed on a failing test in 2 of 3 runs.

Local LLMs · vllm
How to Benchmark vLLM: Find the Best Model, Quant and Settings for Your GPU

In today's post I'm sharing how I benchmark local models on vLLM by changing one thing at a time, along with the traps that quietly skew the numbers and my results from 21 setups on two 48GB RTX 4090Ds. The harness, vllm-bench, is free, so you can run the same tests on your own card.

local-llm · vllm
Muse Glimmer 30B vs qwen: my day-one local benchmark on a dual RTX 4090 rig

Meta dropped Muse Glimmer 30B and I spent the day trying to unseat my qwen daily driver on the dual-4090 rig. Four apparent hangs, one day-one bug, and a same-harness bench later, I had my answer - and it wasn't the coronation I'd half expected. So, is it better than my daily driver?

gpu · hardware
How to set up vLLM in Docker on Windows (WSL2): serve an open-weight model on your own GPU

vLLM doesn't run on Windows natively, so this is the Docker Desktop and WSL2 route my rig has served on since July: the compose file, the image pin that stops it upgrading itself, the walls WSL2 puts in your way, and how the settings were found, with Claude writing the test harness.

local-llm · vllm