Tagged: inference

How to set up vLLM: Docker, compose, and the flags that earn their keep

vLLM from nothing to a working OpenAI-compatible server: WSL2 or Linux, Docker, the compose file we actually run, a preset per model, and the flags that measurably changed things on a dual-4090 bench.