Tagged: quantisation

The broken rules of local LLM inference

I used to lock the clocks on my mining GPUs. The same instinct just helped kill five rules of local LLM inference on a £6,200, 96GB rig.