On a 16 GB GPU, 12 catalog models run fully on the GPU at an 8,192-token context. The most capable is gemma-4-26B-A4B-it at QAT-Q4_0 (needs ~15.8 GiB). Pick a smaller model or a lower quant for more headroom.
Figures assume a 16 GB GPU paired with 32 GB of system RAM (a typical desktop), judged at an 8,192-token context. Speed depends on the specific card — this page ranks by fit, not speed.
TL;DR
16 GB is the flexible middle of local LLM hardware. Mid-size models fit with room to spare — often at higher, near-lossless quants — and the low-20B class starts to run fully on the GPU at the 4-bit sweet spot. The list below is derived live from the fit engine at a fixed context, largest model first.
You can chase capability (the largest model that fits at 4-bit) or dependability (a mid-size model at a higher quant, with context headroom left over). Models near the top of the list use most of the card, so if you run long contexts, keep other apps on the GPU, or share the card with a desktop session, something further down the list will treat you better.
gemma-4-26B-A4B-it at QAT-Q4_0 · 26.5B params · needs ~15.8 GiB
One command to run it (llama.cpp):
llama-server -m gemma-4-26B_q4_0-it.gguf -c 8192 -ngl 999| Model | Sweet-spot quant | Fits in |
|---|---|---|
| gemma-4-26B-A4B-it google | QAT-Q4_0 Runs fully on GPU | ~15.8 GiB |
| Devstral-Small-2-24B-Instruct-2512 mistralai | IQ4_XS Runs fully on GPU | ~14.9 GiB |
| Mistral-Small-3.2-24B-Instruct-2506 mistralai | IQ4_XS Runs fully on GPU | ~14.9 GiB |
| gpt-oss-20b openai | F16 Runs fully on GPU | ~15 GiB |
| Kimi-VL-A3B-Instruct moonshotai | Q4_K_M Runs fully on GPU | ~11.6 GiB |
| DeepSeek-R1-Distill-Qwen-14B deepseek-ai | Q4_K_M Runs fully on GPU | ~11.2 GiB |
| Qwen3-14B Qwen | Q4_K_M Runs fully on GPU | ~11 GiB |
| Phi-4-reasoning microsoft | Q4_K_M Runs fully on GPU | ~11.4 GiB |
| phi-4 microsoft | Q4_K_M Runs fully on GPU | ~11.2 GiB |
| gemma-4-12B-it google | Q8_0 Runs fully on GPU | ~14.3 GiB |
| Qwen3-8B Qwen | Q8_0 Runs fully on GPU | ~10.6 GiB |
| DeepSeek-R1-Distill-Llama-8B deepseek-ai | Q8_0 Runs fully on GPU | ~10.3 GiB |
Derived live from the fit engine + catalog at an 8,192-token context. "Fits in" is the modelled VRAM the sweet-spot quant needs (weights + KV cache + overhead). Speed depends on your specific card — check a GPU page or the calculator.
Two things: the low-20B class of models starts to fit fully on the GPU at 4-bit sweet-spot quants, and the popular mid-size models fit at higher quants with headroom for longer context. It's the point where you stop having to compromise on quant quality for the everyday model class.
If the biggest entry fits with room to spare at your target context, take it — parameter count generally matters most. If it's a tight fit, a mid-size model at a higher quant is usually the better daily driver: more context headroom, no out-of-memory surprises, and quality that's close in practice.
Yes. The verdicts on this page hold at the stated context; the KV cache grows as context grows, so a tight fit can become an out-of-memory error at long contexts. Check your exact model, quant, and context in the fit calculator before committing to a big download.
Not fully on the GPU at healthy quants, as a rule — that class generally wants more memory once the KV cache and overhead are counted. Offloading to system RAM works but is much slower. If 30B-class models are the goal, the 24 GB tier is where they become comfortable.
This page ranks by fit, not speed — generation speed mostly tracks memory bandwidth, which varies a lot between 16 GB cards. Check your card's GPU page or the calculator for speed context on your exact setup.
selected to compare · pick at least 2