deepseek-ai

DeepSeek-R1

684.5B parameters · mixture-of-experts · reasoning · Reasoning · DeepSeek family

Mixture-of-experts — what it means for memory & speed → Long context — how it uses memory →
mit hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Unsloth Q4_K_M quant of DeepSeek-R1-671B (684.531B total, 256-expert MoE) — preservation copy. MIT.

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick? What do these mean?

GGUF · UD-Q2_K_XL

211.04 GB

Set your rig
Est. speed
—
Swarm
1S / 0L webseed OK

Unsloth dynamic 2-bit (UD-Q2_K_XL) — accessible 671B (684.531B total, 256-expert MoE) giant lead (~211 GiB, 5 shards; preservation copy). MIT.

Magnet .torrent

GGUF · Q4_K_M

376.65 GB

Set your rig
Est. speed
—
Swarm
1S / 0L webseed OK

Unsloth Q4_K_M quant of DeepSeek-R1-671B (684.531B total, 256-expert MoE) — preservation copy. MIT.

Magnet .torrent

Want a different quant? Request it on the board →

Runs fully on

Apple M3 Ultra (512 GB) · UD-Q2_K_XL

Green = the model's best quant fits fully in GPU/unified memory at 8K context. Tap a card for its full "what runs on it" page.

Context

Advertised 160k · usable ≈ 64–128k (measured — LongBench v2)

How we know

Evidence grade: measured — a published evaluation we can cite, with task and length stated.

LongBench v2 leaderboard: 58.3 overall w/ CoT (long bucket 59.3); card lists 128K context and FRAMES 82.5. No RULER/NIAH results published.

Reviewed on Jul 18, 2026.

Capabilities (as declared by the maintainer): reasoning tool calling

Run it

Runtime completeness (UD-Q2_K_XL torrent): llama.cpp ✅ Ollama – vision sidecar –

  • – Ollama — runs the GGUF directly; no Modelfile bundled
  • – vision sidecar — not a vision model

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server 
llama-server -m DeepSeek-R1-UD-Q2_K_XL-00001-of-00005.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt. This model compresses its KV cache, but only on llama.cpp b5137 (April 2025) or newer AND a GGUF converted after that date. Older files -- including some still-popular DeepSeek-R1 and V3-0324 quants -- fall back to the uncompressed layout and use about 5 MB per token instead of 70 KB. Check the quant repo date if long context is the point.

Save as Modelfile next to the GGUF (Modelfile) 
FROM ./DeepSeek-R1-UD-Q2_K_XL-00001-of-00005.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER stop "<｜end▁of▁sentence｜>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).
Create 
ollama create deepseek-ai-deepseek-r1 -f Modelfile
Run 
ollama run deepseek-ai-deepseek-r1

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

transformers does not load GGUF weights.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Load DeepSeek-R1-UD-Q2_K_XL-00001-of-00005.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Technical details

Chat template

{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='', is_first_sp=true) %}{%- for message in messages %}{%- if message['role'] == 'system' %}{%- if ns.is_first_sp %}{% set ns.system_prompt = ns.system_prompt + message['content'] %}{% set ns.is_first_sp = false %}{%- else %}{% set ns.system_prompt = ns.system_prompt + '\n\n' + message['content'] %}{%- endif %}{%- endif %}{%- endfor %}{{ bos_token }}{{ ns.system_prompt }}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and 'tool_calls' in message %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls'] %}{%- if not ns.is_first %}{%- if message['content'] is none %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- else %}{{'<｜Assistant｜>' + message['content'] + '<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- endif %}{%- set ns.is_first = true -%}{%- else %}{{'\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- endif %}{%- endfor %}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- if message['role'] == 'assistant' and 'tool_calls' not in message %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜><think>\n'}}{% endif %}

Sampling defaults

top_p
0.95
temperature
0.6

Stop strings

<｜end▁of▁sentence｜>

Evidence & provenance

Source

Provenance
https://huggingface.co/deepseek-ai/DeepSeek-R1
Revision pin
56d4cbbb4d29f4355bab4b9a39ccb717a14ad5ad
Manifest
Present

License

Name
mit
Commercial use
yes
Access
Open

How verification works →

File hashes (SHA-256)

  • deepseek-r1-671b-ud-q2-k-xl/DeepSeek-R1-UD-Q2_K_XL-00001-of-00005.gguf 311b7e2b72da29daffbac5e5f5df9353b1b3be9879d22d1dc498ece99529cfe5
  • deepseek-r1-671b-ud-q2-k-xl/DeepSeek-R1-UD-Q2_K_XL-00002-of-00005.gguf 6b1b63ef53a397cb6ace27961165c1cfb82051b1ba42ade07405642ed096355d
  • deepseek-r1-671b-ud-q2-k-xl/DeepSeek-R1-UD-Q2_K_XL-00003-of-00005.gguf 26dc583b72212bd0c0c0a646e78bb0863abef173de26cef5d38c2b051e3aaec6
  • deepseek-r1-671b-ud-q2-k-xl/DeepSeek-R1-UD-Q2_K_XL-00004-of-00005.gguf 25427bd9a557f820cde553cfc5f6c911b6109322bd3dfa555ecbc0f0ad55c2f1
  • deepseek-r1-671b-ud-q2-k-xl/DeepSeek-R1-UD-Q2_K_XL-00005-of-00005.gguf 9983ab89fe131158b39704b61f07215ed7c9f847ae9e97225d6a9e74692f70d2
  • deepseek-r1-671b-ud-q2-k-xl/LICENSE 1c8f573e830ca9b3ebfeb7ace1823146e22b66f99ee223840e7637c9e745e1c7
  • deepseek-r1-671b-ud-q2-k-xl/README.md 753e8ae21b5d3a0a02676833848fba197e0e474acdaef30612db8d66f4c3582d
  • deepseek-r1-671b-q4-k-m/DeepSeek-R1-Q4_K_M-00001-of-00009.gguf d111d9e28b4035e6781906b6451b7866737b4a4ee734baa1575c55d8aa1b4200
  • deepseek-r1-671b-q4-k-m/DeepSeek-R1-Q4_K_M-00002-of-00009.gguf bc15645dcad23f664f5186a261752cf360af289da0e5701d47330763a8a30ef5
  • deepseek-r1-671b-q4-k-m/DeepSeek-R1-Q4_K_M-00003-of-00009.gguf 5153167c9e62ca5e8c14e804506dd8a1020cc824e88166ae50f4d2cdad039399
  • deepseek-r1-671b-q4-k-m/DeepSeek-R1-Q4_K_M-00004-of-00009.gguf 26e3849c55a2d51e2bdaca9b9fc18b3a07f27513454c3eef9b3f95050d3a2878
  • deepseek-r1-671b-q4-k-m/DeepSeek-R1-Q4_K_M-00005-of-00009.gguf b0c741ab4e2960b574e4735a1e847a159e3371213d5be2b189d1d2ffd6388298
  • deepseek-r1-671b-q4-k-m/DeepSeek-R1-Q4_K_M-00006-of-00009.gguf e081b8ee5458c37dc891d1b0a93d9b7f4c14285a52feb1083a75c489a06a8e00
  • deepseek-r1-671b-q4-k-m/DeepSeek-R1-Q4_K_M-00007-of-00009.gguf e4673ab73c3b3b6dbc971c511ee63b9daf03cb9aa173817616eb60c824f3434b
  • deepseek-r1-671b-q4-k-m/DeepSeek-R1-Q4_K_M-00008-of-00009.gguf b960c2e008db2cdda42e163bda1187ca9516c6248fa14d71671a9c4993c7e401
  • deepseek-r1-671b-q4-k-m/DeepSeek-R1-Q4_K_M-00009-of-00009.gguf a2cedd446437830456b9378410560f2bf7f1041cab6a929b14330c1112706d47
  • deepseek-r1-671b-q4-k-m/LICENSE 1c8f573e830ca9b3ebfeb7ace1823146e22b66f99ee223840e7637c9e745e1c7
  • deepseek-r1-671b-q4-k-m/README.md 753e8ae21b5d3a0a02676833848fba197e0e474acdaef30612db8d66f4c3582d

Explore further

Also consider

Get an email when a better quant fits your rig.

Set your rig once; we'll alert you when a new or better-fitting build lands.

Set your rig

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard
Community

Reviews, sampler presets and community runtime reports. Be the first to contribute.

Reviews (0)

Log in to write a structured review of this model.

No reviews yet.

Sampler presets

Log in to share a sampler preset or vote on presets.

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

Log in to report whether a quant loads in your runtime.

No community reports yet.

selected to compare · pick at least 2

Compare ()