On a 12 GB GPU, 12 catalog models run fully on the GPU at an 8,192-token context. The most capable is Kimi-VL-A3B-Instruct at Q4_K_M (needs ~11.6 GiB). Pick a smaller model or a lower quant for more headroom.
Figures assume a 12 GB GPU paired with 32 GB of system RAM (a typical desktop), judged at an 8,192-token context. Speed depends on the specific card — this page ranks by fit, not speed.
TL;DR
12 GB is where the popular mid-size chat and reasoning models get comfortable. The class of models most people actually recommend for local use runs fully on the GPU here at 4-bit sweet-spot quants, and smaller models gain real headroom — higher quants, longer context, or both. The list below is derived live from the fit engine, largest model first.
Coming from 8 GB, you have two good ways to use the upgrade: run a larger model at the standard 4-bit sweet spot, or run a smaller model at a higher quant with a longer context. Neither is universally right — bigger models generally reason better, while higher quants and more context make a smaller model more dependable for long sessions.
Kimi-VL-A3B-Instruct at Q4_K_M · 16.4B params · needs ~11.6 GiB
One command to run it (llama.cpp):
llama-server -m Kimi-VL-A3B-Instruct.i1-Q4_K_M.gguf -c 8192 -ngl 999| Model | Sweet-spot quant | Fits in |
|---|---|---|
| Kimi-VL-A3B-Instruct moonshotai | Q4_K_M Runs fully on GPU | ~11.6 GiB |
| DeepSeek-R1-Distill-Qwen-14B deepseek-ai | Q4_K_M Runs fully on GPU | ~11.2 GiB |
| Qwen3-14B Qwen | Q4_K_M Runs fully on GPU | ~11 GiB |
| Phi-4-reasoning microsoft | Q4_K_M Runs fully on GPU | ~11.4 GiB |
| phi-4 microsoft | Q4_K_M Runs fully on GPU | ~11.2 GiB |
| gemma-4-12B-it google | Q4_K_M Runs fully on GPU | ~8.6 GiB |
| Qwen3-8B Qwen | Q8_0 Runs fully on GPU | ~10.6 GiB |
| DeepSeek-R1-Distill-Llama-8B deepseek-ai | Q8_0 Runs fully on GPU | ~10.3 GiB |
| gemma-4-E4B-it google | Q8_0 Runs fully on GPU | ~9.1 GiB |
| DeepSeek-R1-Distill-Qwen-7B deepseek-ai | Q8_0 Runs fully on GPU | ~9.3 GiB |
| Qwen3-Embedding-8B Qwen | Q4_K_M Runs fully on GPU | ~6.4 GiB |
| gemma-4-E2B-it google | Q8_0 Runs fully on GPU | ~5.8 GiB |
Derived live from the fit engine + catalog at an 8,192-token context. "Fits in" is the modelled VRAM the sweet-spot quant needs (weights + KV cache + overhead). Speed depends on your specific card — check a GPU page or the calculator.
The mid-size instruct and reasoning models — the tier most local-LLM communities treat as the everyday recommendation — fit fully on GPU at 4-bit sweet-spot quants, where on 8 GB they generally don't. Smaller models also gain headroom for higher quants or longer context.
Generally not fully on the GPU at sensible quants — that class typically needs more memory than 12 GB once you account for the KV cache and overhead. You can offload part of a larger model to system RAM, but expect a big speed penalty; the table on this page only lists full-GPU fits.
As a rule of thumb, parameter count wins: a larger model at a healthy 4-bit quant usually outperforms a smaller model at 8-bit. The exception is when the bigger model only fits with no headroom at your target context — then the smaller, higher-quant model is the more dependable choice.
Yes — the KV cache grows with context, so the fit verdicts on this page hold at the stated context and can flip at much longer ones. If long documents or long chats are the point, verify your model and context in the fit calculator first.
This page ranks by fit, not speed. Generation speed tracks your card's memory bandwidth more than anything else, so check your GPU's page or the calculator for what to expect on your exact hardware.
selected to compare · pick at least 2