2 articles tagged gpu.
Meta dropped Muse Glimmer 30B and I spent the day trying to unseat my qwen daily driver on the dual-4090 rig. Four apparent hangs, one day-one bug, and a same-harness bench later, I had my answer - and it wasn't the coronation I'd half expected. So, is it better than my daily driver?
gpu · hardware
In today's post I'm going through the best GPU for running LLMs locally, and why memory bandwidth and VRAM decide it. I start with the cheapest VRAM you can buy and the cheapest 32GB cards, then the modded 48GB RTX 4090s I run, and finish on when an RTX PRO 6000 or a DGX Spark makes more sense.
local-llm · gpu