Qwen

Qwen3-235B-A22B-Thinking-2507

235.1B parameters · mixture-of-experts · reasoning · Reasoning · Qwen3 family

Mixture-of-experts — what it means for memory & speed → Long context — how it uses memory →
apache-2.0 hash verified source matched revision pinned
Good for: Long context

Which version should I download?

Default pick: GGUF · Q8_0 — Unsloth Q8_0 quant of the Qwen3-235B-A22B-Thinking-2507 MoE giant — preservation copy. Apache-2.0.

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick? What do these mean?

GGUF · IQ4_XS

116.89 GB

Set your rig
Est. speed
—
Swarm
1S / 0L webseed OK

Unsloth IQ4_XS (imatrix 4-bit) quant of the Qwen3-235B-A22B-Thinking-2507 MoE giant — preservation copy. Apache-2.0.

Magnet .torrent

GGUF · UD-Q4_K_XL

125.01 GB

Set your rig
Est. speed
—
Swarm
1S / 0L webseed OK

Unsloth dynamic 4-bit (UD-Q4_K_XL) — true-Q4-class 235B-A22B MoE giant lead (~126 GB, 3 shards; preservation copy). Apache-2.0.

Magnet .torrent

GGUF · Q8_0

232.77 GB

Set your rig
Est. speed
—
Swarm
1S / 0L webseed OK

Unsloth Q8_0 quant of the Qwen3-235B-A22B-Thinking-2507 MoE giant — preservation copy. Apache-2.0.

Magnet .torrent

Want a different quant? Request it on the board →

Runs fully on

Apple M3 Ultra (512 GB) · Q8_0 Apple M3 Ultra (256 GB) · UD-Q4_K_XL Apple M2 Ultra (192 GB) · UD-Q4_K_XL

Green = the model's best quant fits fully in GPU/unified memory at 8K context. Tap a card for its full "what runs on it" page.

Context

Advertised 256k · usable ≈ 512k–1M (measured — RULER)

How we know

Evidence grade: measured — a published evaluation we can cite, with task and length stated.

Model card RULER table (DCA, thinking budget 8,192, 260 samples/length): 95.4 at 256k, 90.7 at 512k, 86.6 at 1000k. Usable band exceeds the advertised native window.

Reviewed on Jul 18, 2026.

Capabilities (as declared by the maintainer): reasoning tool calling

Run it

Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –

  • – Ollama — runs the GGUF directly; no Modelfile bundled
  • – vision sidecar — not a vision model

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server 
llama-server -m Qwen3-235B-A22B-Thinking-2507-IQ4_XS-00001-of-00003.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile) 
FROM ./Qwen3-235B-A22B-Thinking-2507-IQ4_XS-00001-of-00003.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).
Create 
ollama create qwen-qwen3-235b-a22b-thinking-2507 -f Modelfile
Run 
ollama run qwen-qwen3-235b-a22b-thinking-2507

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

transformers does not load GGUF weights.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Load Qwen3-235B-A22B-Thinking-2507-IQ4_XS-00001-of-00003.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Technical details

Chat template

{%- if tools %}
    {{- '<|im_start|>system\n' }}
    {%- if messages[0].role == 'system' %}
        {{- messages[0].content + '\n\n' }}
    {%- endif %}
    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
    {%- if messages[0].role == 'system' %}
        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
    {%- set index = (messages|length - 1) - loop.index0 %}
    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
        {%- set ns.multi_step_tool = false %}
        {%- set ns.last_query_index = index %}
    {%- endif %}
{%- endfor %}
{%- for message in messages %}
    {%- if message.content is string %}
        {%- set content = message.content %}
    {%- else %}
        {%- set content = '' %}
    {%- endif %}
    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "assistant" %}
        {%- set reasoning_content = '' %}
        {%- if message.reasoning_content is string %}
            {%- set reasoning_content = message.reasoning_content %}
        {%- else %}
            {%- if '</think>' in content %}
                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
            {%- endif %}
        {%- endif %}
        {%- if loop.index0 > ns.last_query_index %}
            {%- if loop.last or (not loop.last and reasoning_content) %}
                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
            {%- else %}
                {{- '<|im_start|>' + message.role + '\n' + content }}
            {%- endif %}
        {%- else %}
            {{- '<|im_start|>' + message.role + '\n' + content }}
        {%- endif %}
        {%- if message.tool_calls %}
            {%- for tool_call in message.tool_calls %}
                {%- if (loop.first and content) or (not loop.first) %}
                    {{- '\n' }}
                {%- endif %}
                {%- if tool_call.function %}
                    {%- set tool_call = tool_call.function %}
                {%- endif %}
                {{- '<tool_call>\n{"name": "' }}
                {{- tool_call.name }}
                {{- '", "arguments": ' }}
                {%- if tool_call.arguments is string %}
                    {{- tool_call.arguments }}
                {%- else %}
                    {{- tool_call.arguments | tojson }}
                {%- endif %}
                {{- '}\n</tool_call>' }}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
            {{- '<|im_start|>user' }}
        {%- endif %}
        {{- '\n<tool_response>\n' }}
        {{- content }}
        {{- '\n</tool_response>' }}
        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n<think>\n' }}
{%- endif %}

Sampling defaults

top_k
20
top_p
0.95
temperature
0.6

Stop strings

<|im_end|> <|endoftext|>

Evidence & provenance

Source

Provenance
https://huggingface.co/Qwen/Qwen3-235B-A22B-Thinking-2507
Revision pin
6cbffae6d8e28b986a6b17bd36f42f9fa0f1f0a5
Manifest
Present

License

Name
apache-2.0
Commercial use
yes
Access
Open

How verification works →

File hashes (SHA-256)

  • qwen3-235b-a22b-thinking-2507-iq4-xs/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • qwen3-235b-a22b-thinking-2507-iq4-xs/Qwen3-235B-A22B-Thinking-2507-IQ4_XS-00001-of-00003.gguf 1c91f06e73b0dc3730ef1ab750974d498860f904eab6f64dc2d9f610fb7ae1a1
  • qwen3-235b-a22b-thinking-2507-iq4-xs/Qwen3-235B-A22B-Thinking-2507-IQ4_XS-00002-of-00003.gguf 984fbc552223ef91eaace8fc882313708e21768a2fb72e56497a0e5b4a060921
  • qwen3-235b-a22b-thinking-2507-iq4-xs/Qwen3-235B-A22B-Thinking-2507-IQ4_XS-00003-of-00003.gguf dc933b0b0b58adf2036345e22c71153fa8149b677f99b427fc49003c916c36d8
  • qwen3-235b-a22b-thinking-2507-iq4-xs/README.md 65d7fefc8e6bde05a27c331f3dd2ca9304cdc45f86f2f83f6d9b7c3fcefa20ff
  • qwen3-235b-a22b-thinking-2507-ud-q4-k-xl/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • qwen3-235b-a22b-thinking-2507-ud-q4-k-xl/Qwen3-235B-A22B-Thinking-2507-UD-Q4_K_XL-00001-of-00003.gguf 5110f159e3d320a5cec18c4497842df3b80620c11d21a9ecadadae00d94758a0
  • qwen3-235b-a22b-thinking-2507-ud-q4-k-xl/Qwen3-235B-A22B-Thinking-2507-UD-Q4_K_XL-00002-of-00003.gguf 2d8d0f614859ad7395eee95f26b1f9aaba4b9fb6bc4101f34dddc2eb6e6f3722
  • qwen3-235b-a22b-thinking-2507-ud-q4-k-xl/Qwen3-235B-A22B-Thinking-2507-UD-Q4_K_XL-00003-of-00003.gguf a43bbb3a557fbaecf29f6e25afe0f81ee9a6ba52e6c84e26258ab56838b9faa8
  • qwen3-235b-a22b-thinking-2507-ud-q4-k-xl/README.md 65d7fefc8e6bde05a27c331f3dd2ca9304cdc45f86f2f83f6d9b7c3fcefa20ff
  • qwen3-235b-a22b-thinking-2507-q8-0/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • qwen3-235b-a22b-thinking-2507-q8-0/Qwen3-235B-A22B-Thinking-2507-Q8_0-00001-of-00006.gguf 17a49655d1fe564c20d8776c04c54b0b612736adc065e6ea0be6fde5ead8017e
  • qwen3-235b-a22b-thinking-2507-q8-0/Qwen3-235B-A22B-Thinking-2507-Q8_0-00002-of-00006.gguf f8ff27e0630f0f2ab44be8efea6699c50a040023542c9b4a0c26ffc9a0bc8d51
  • qwen3-235b-a22b-thinking-2507-q8-0/Qwen3-235B-A22B-Thinking-2507-Q8_0-00003-of-00006.gguf 1f2548b087c08d8896bc2be18faad7cab7fc85af74490d3c9b42d22af5c9b01c
  • qwen3-235b-a22b-thinking-2507-q8-0/Qwen3-235B-A22B-Thinking-2507-Q8_0-00004-of-00006.gguf caaaf3fa192401c3f99b5a376803b87fefccd7233d0e2c10b58e8454f799cd3f
  • qwen3-235b-a22b-thinking-2507-q8-0/Qwen3-235B-A22B-Thinking-2507-Q8_0-00005-of-00006.gguf 2b6d735b76a6c5f8ae6b2c219c3f0d1c47af3eede844412472a9e5021df03865
  • qwen3-235b-a22b-thinking-2507-q8-0/Qwen3-235B-A22B-Thinking-2507-Q8_0-00006-of-00006.gguf aee31064653da5917538cf54daf4dba889b20271ec20ab9b88f83eef0da1e3a2
  • qwen3-235b-a22b-thinking-2507-q8-0/README.md 65d7fefc8e6bde05a27c331f3dd2ca9304cdc45f86f2f83f6d9b7c3fcefa20ff

Explore further

Appears in

What runs on a 192 GB Mac

Also consider

Get an email when a better quant fits your rig.

Set your rig once; we'll alert you when a new or better-fitting build lands.

Set your rig

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard
Community

Reviews, sampler presets and community runtime reports. Be the first to contribute.

Reviews (0)

Log in to write a structured review of this model.

No reviews yet.

Sampler presets

Log in to share a sampler preset or vote on presets.

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

Log in to report whether a quant loads in your runtime.

No community reports yet.

selected to compare · pick at least 2

Compare ()