On a 8 GB Mac, 8 catalog models run fully on the GPU at an 8,192-token context. The most capable is DeepSeek-R1-Distill-Qwen-7B at IQ4_XS (needs ~5.3 GiB). Pick a smaller model or a lower quant for more headroom.
Figures assume a 8 GB unified-memory Mac (one pool shared by CPU and GPU), judged at an 8,192-token context. Speeds depend on the specific chip — see the GPU pages.
TL;DR
An 8 GB Mac runs the genuinely small instruct models — the 3B and 4B class — fully on the GPU at 4-bit sweet-spot quants. The 7B and 8B class is right at the edge here and often past it, because you are not working with 8 GB: macOS hands the GPU only part of the unified pool by default — roughly three quarters, and about two thirds on machines at 32 GB and below, which leaves something closer to five. The list below is derived live from the fit engine, largest first, and if it comes out short that is the honest answer rather than a gap in our data.
This is the tier where that matters most. The pool holding your model also holds macOS and every open app, and the GPU's own default share is the smaller number we budget against. On a machine this size the difference between the sticker figure and the working figure is most of the decision.
Two habits make it work. Close heavy apps before loading anything near the top of the list. And treat quant choice as the main lever — stepping down a rung often turns "doesn't fit" into "runs on the GPU", though below roughly 4-bit a smaller model at a healthier quant usually beats a bigger one crushed further. If you find yourself fighting for memory daily, the 16 GB tier is where the squeeze eases.
DeepSeek-R1-Distill-Qwen-7B at IQ4_XS · 7.6B params · needs ~5.3 GiB
One command to run it (llama.cpp):
llama-server -m DeepSeek-R1-Distill-Qwen-7B-IQ4_XS.gguf -c 8192 -ngl 999| Model | Sweet-spot quant | Fits in |
|---|---|---|
| DeepSeek-R1-Distill-Qwen-7B deepseek-ai | IQ4_XS Runs fully on GPU | ~5.3 GiB |
| gemma-4-E2B-it google | QAT-Q4_0 Runs fully on GPU | ~4 GiB |
| Qwen3-Embedding-4B Qwen | Q4_K_M Runs fully on GPU | ~4.2 GiB |
| Phi-4-mini-instruct microsoft | Q4_K_M Runs fully on GPU | ~4.1 GiB |
| SmolLM3-3B HuggingFaceTB | Q8_0 Runs fully on GPU | ~4.4 GiB |
| DeepSeek-R1-Distill-Qwen-1.5B deepseek-ai | Q8_0 Runs fully on GPU | ~2.7 GiB |
| Qwen3-Reranker-0.6B Qwen | BF16 Runs fully on GPU | ~2.6 GiB |
| Qwen3-Embedding-0.6B Qwen | Q8_0 Runs fully on GPU | ~2 GiB |
Derived live from the fit engine + catalog at an 8,192-token context. "Fits in" is the modelled VRAM the sweet-spot quant needs (weights + KV cache + overhead). Speed depends on your specific card — check a GPU page or the calculator.
Yes, for the small end, with expectations set right. The 3B and 4B instruct class runs fully on the GPU at 4-bit sweet-spot quants and is practical for chat, drafting and light coding help. The 7B–8B class is at the edge and often past it. The reason is worth knowing: macOS hands the GPU only part of the unified pool by default — about two thirds on a machine this size — so you are budgeting against roughly 5 GB rather than 8. Larger classes need a bigger machine, not a cleverer quant.
Usually because the rest of the system is holding memory. The fit verdicts assume the model gets most of the pool, but macOS, your browser, and background apps all draw from the same place — and macOS caps how much of the pool the GPU may claim by default. Close heavy apps and try again; if it's still tight, take the next quant down or a smaller model.
The highest quant that still fits with a little headroom at your target context — the list shows a sweet-spot quant per model, and the quant helper ranks the alternatives. As a rule of thumb, once you're forced below roughly 4-bit, switch to a smaller model at a healthier quant instead: same footprint, usually better output.
Yes, and proportionally more here than anywhere else — the KV cache grows with context and comes out of the same small shared pool as the weights and the OS. The verdicts on this page hold at the stated context; for long chats or big documents, check your model and context in the fit calculator first.
This page ranks by fit, not speed. Generation speed mostly tracks the chip's memory bandwidth, which differs between Apple Silicon tiers and generations — check your specific chip's page for speed context. The good news at this tier: the models that fit are small, and small models are quick.
selected to compare · pick at least 2