deepseek-ai

DeepSeek-R1-Distill-Qwen-32B

32.8B parameters · reasoning · Reasoning · DeepSeek family

Long context — how it uses memory →
mit hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Balanced size/quality — good default.

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick? What do these mean?

GGUF · IQ4_XS

16.48 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski IQ4_XS quant of DeepSeek-R1-Distill-Qwen-32B (32.764B dense) — compact low-bit option. MIT.

Magnet .torrent

GGUF · Q4_K_M

18.49 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Balanced size/quality — good default.

Magnet .torrent

GGUF · Q8_0

32.43 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski Q8_0 quant of DeepSeek-R1-Distill-Qwen-32B (32.764B dense) — high-quality option. MIT.

Magnet .torrent

Want a different quant? Request it on the board →

Runs fully on

RTX 5090 · Q4_K_M RTX 4090 · Q4_K_M RTX 3090 · Q4_K_M RTX 3090 Ti · Q4_K_M RTX Pro 6000 Blackwell 96GB · Q8_0

Green = the model's best quant fits fully in GPU/unified memory at 8K context. Tap a card for its full "what runs on it" page.

Context

Advertised 128k · usable ≈ 32k (estimated — community report)

How we know

Evidence grade: estimated — derived, including structured community reports.

Community (HF repo discussion #5, methodology stated, no maintainer response): shipped config.json omits the YaRN rope_scaling block needed to reach 128k; the Qwen2.5-32B base is natively 32,768 — full 128k is not reachable on a stock config (e.g. llama.cpp) without adding it. No published long-context evals for any R1-Distill model.

Reviewed on Jul 18, 2026.

Capabilities (as declared by the maintainer): reasoning tool calling

Run it

Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –

  • – Ollama — runs the GGUF directly; no Modelfile bundled
  • – vision sidecar — not a vision model

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server 
llama-server -m DeepSeek-R1-Distill-Qwen-32B-IQ4_XS.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile) 
FROM ./DeepSeek-R1-Distill-Qwen-32B-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER stop "<｜end▁of▁sentence｜>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).
Create 
ollama create deepseek-ai-deepseek-r1-distill-qwen-32b -f Modelfile
Run 
ollama run deepseek-ai-deepseek-r1-distill-qwen-32b

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

transformers does not load GGUF weights.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Load DeepSeek-R1-Distill-Qwen-32B-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Technical details

Chat template

{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- set ns.is_first = true -%}{%- else %}{{'\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\n<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜><think>\n'}}{% endif %}

Sampling defaults

top_p
0.95
temperature
0.6

Stop strings

<｜end▁of▁sentence｜>

Evidence & provenance

Source

Provenance
https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
Revision pin
711ad2ea6aa40cfca18895e8aca02ab92df1a746
Manifest
Present

License

Name
mit
Commercial use
yes
Access
Open

How verification works →

File hashes (SHA-256)

  • deepseek-r1-distill-qwen-32b-iq4-xs/DeepSeek-R1-Distill-Qwen-32B-IQ4_XS.gguf 91eb0de25c80c0b33a3e89777d96040622f8fa70c71610bf13e19dab297be6b6
  • deepseek-r1-distill-qwen-32b-iq4-xs/LICENSE 1c8f573e830ca9b3ebfeb7ace1823146e22b66f99ee223840e7637c9e745e1c7
  • deepseek-r1-distill-qwen-32b-iq4-xs/README.md 2f3c131605537200b9997009ce82a962f7e9009e8b3c43d442526b9e4b788eb3
  • deepseek-r1-distill-qwen-32b-q4-k-m/DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf bed9b0f551f5b95bf9da5888a48f0f87c37ad6b72519c4cbd775f54ac0b9fc62
  • deepseek-r1-distill-qwen-32b-q4-k-m/LICENSE 1c8f573e830ca9b3ebfeb7ace1823146e22b66f99ee223840e7637c9e745e1c7
  • deepseek-r1-distill-qwen-32b-q4-k-m/README.md 2f3c131605537200b9997009ce82a962f7e9009e8b3c43d442526b9e4b788eb3
  • deepseek-r1-distill-qwen-32b-q8-0/DeepSeek-R1-Distill-Qwen-32B-Q8_0.gguf 5b321bce035d98e61636e15cd468467e57dfaee4ea28acd42edd7edd1b6098d7
  • deepseek-r1-distill-qwen-32b-q8-0/LICENSE 1c8f573e830ca9b3ebfeb7ace1823146e22b66f99ee223840e7637c9e745e1c7
  • deepseek-r1-distill-qwen-32b-q8-0/README.md 2f3c131605537200b9997009ce82a962f7e9009e8b3c43d442526b9e4b788eb3

Explore further

Appears in

What runs on 24 GB of VRAM What runs on 32 GB of VRAM What runs on 48 GB of VRAM What runs on a 36 GB Mac What runs on a 48 GB Mac What runs on a 64 GB Mac

Also consider

Local alternative to

A local alternative to Claude Opus →

Get an email when a better quant fits your rig.

Set your rig once; we'll alert you when a new or better-fitting build lands.

Set your rig

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard
Community

Reviews, sampler presets and community runtime reports. Be the first to contribute.

Reviews (0)

Log in to write a structured review of this model.

No reviews yet.

Sampler presets

Log in to share a sampler preset or vote on presets.

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

Log in to report whether a quant loads in your runtime.

No community reports yet.

selected to compare · pick at least 2

Compare ()