mistralai
Mistral-Small-3.2-24B-Instruct-2506
24B parameters · Instruct · Mistral family
Recommended download
Which version should I download?
Default pick: GGUF · Q4_K_M — Balanced size/quality — good default.
Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.
| Format | Level | Size | Verdict | Est. speed | Quality note | Swarm | Download |
|---|---|---|---|---|---|---|---|
| GGUF | IQ4_XS | 11.88 GB | Set your rig | — | Bartowski IQ4_XS quant of Mistral-Small-3.2-24B-Instruct-2506 (24.011B dense) — compact low-bit option. Apache-2.0. | 2S / 0L webseed OK | |
| GGUF | Q4_K_M | 13.35 GB | Set your rig | — | Balanced size/quality — good default. | 2S / 0L webseed OK | |
| GGUF | Q8_0 | 23.33 GB | Set your rig | — | Bartowski Q8_0 quant of Mistral-Small-3.2-24B-Instruct-2506 (24.011B dense) — high-quality option. Apache-2.0. | 2S / 0L webseed OK |
GGUF · IQ4_XS
11.88 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Bartowski IQ4_XS quant of Mistral-Small-3.2-24B-Instruct-2506 (24.011B dense) — compact low-bit option. Apache-2.0.
GGUF · Q4_K_M
13.35 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Balanced size/quality — good default.
GGUF · Q8_0
23.33 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Bartowski Q8_0 quant of Mistral-Small-3.2-24B-Instruct-2506 (24.011B dense) — high-quality option. Apache-2.0.
Want a different quant? Request it on the board →
Runs fully on
Green = the model's best quant fits fully in GPU/unified memory at 8K context. Tap a card for its full "what runs on it" page.
Context
Advertised 128k · usable ≈ 128k (reported — model card)
How we know
Evidence grade: reported — a maintainer claim with no independent evaluation.
- Mistral Small 3.2 docs model card — observed Jul 18, 2026
Mistral docs model card: 128k window. No long-context evals published.
Reviewed on Jul 18, 2026.
Run it
Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –
- – Ollama — runs the GGUF directly; no Modelfile bundled
- – vision sidecar — not a vision model
Context / KV 8,192 tokens · FP16 set above the quant table
Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.
llama-server -m mistralai_Mistral-Small-3.2-24B-Instruct-2506-IQ4_XS.gguf -c 8192 -ngl 999Use llama-cli in place of llama-server for a one-shot prompt.
FROM ./mistralai_Mistral-Small-3.2-24B-Instruct-2506-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER temperature 0.15
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).ollama create mistralai-mistral-small-3-2-24b-instruct-2506 -f Modelfileollama run mistralai-mistral-small-3-2-24b-instruct-2506GGUF is not a first-class vLLM format.
vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.
transformers does not load GGUF weights.
GGUF weights run under llama.cpp, Ollama or LM Studio.
Load mistralai_Mistral-Small-3.2-24B-Instruct-2506-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).
MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.
GGUF weights run under llama.cpp, Ollama or LM Studio.
Technical details
Sampling defaults
- temperature
- 0.15
Evidence & provenance
Source
- Revision pin
-
95a6d26c4bfb886c58daf9d3f7332c857cb27b43 - Manifest
- Present
License
- Name
- apache-2.0
- Commercial use
- yes
- Access
- Open
File hashes (SHA-256)
-
mistral-small-3-2-24b-instruct-2506-iq4-xs/LICENSE
e68a2bdca4fd3d3a2ffef1ec59a6d5319caba1e2c0c8dc424cf8d76358b7ed2d -
mistral-small-3-2-24b-instruct-2506-iq4-xs/README.md
f2fdbf050f60cfaa3728568da4e70b2e8acb15da7b9d69ff2386df83e669548e -
mistral-small-3-2-24b-instruct-2506-iq4-xs/mistralai_Mistral-Small-3.2-24B-Instruct-2506-IQ4_XS.gguf
4d1764c495f176c5beb37272efa2e402f3c1d84b406726f79cd7a3a5f970b703 -
mistral-small-3.2-24b-instruct-2506-q4-k-m/LICENSE
e68a2bdca4fd3d3a2ffef1ec59a6d5319caba1e2c0c8dc424cf8d76358b7ed2d -
mistral-small-3.2-24b-instruct-2506-q4-k-m/README.md
f2fdbf050f60cfaa3728568da4e70b2e8acb15da7b9d69ff2386df83e669548e -
mistral-small-3.2-24b-instruct-2506-q4-k-m/mistralai_Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M.gguf
80f5bda68f156f12650ca03a0a2dbfae06a215ac41caa773b8631a479f82415e -
mistral-small-3.2-24b-instruct-2506-q4-k-m/mmproj-mistralai_Mistral-Small-3.2-24B-Instruct-2506-bf16.gguf
b9ff6c74328e5221a2d6d6aca41be6c203b2eaab66bdba2ba481afdf4240fb3c -
mistral-small-3.2-24b-instruct-2506-q4-k-m/mmproj-mistralai_Mistral-Small-3.2-24B-Instruct-2506-f16.gguf
e41cc0321dbd0d7e42cdada75862a5ed0b221263313b0f1b55b0b696dfec8647 -
mistral-small-3-2-24b-instruct-2506-q8-0/LICENSE
e68a2bdca4fd3d3a2ffef1ec59a6d5319caba1e2c0c8dc424cf8d76358b7ed2d -
mistral-small-3-2-24b-instruct-2506-q8-0/README.md
f2fdbf050f60cfaa3728568da4e70b2e8acb15da7b9d69ff2386df83e669548e -
mistral-small-3-2-24b-instruct-2506-q8-0/mistralai_Mistral-Small-3.2-24B-Instruct-2506-Q8_0.gguf
980c4969d46bf016ef78de4206bd74a66a6ca7de18152cb603a7b0c4d348bc55
Explore further
Appears in
Also consider
Local alternative to
Get an email when a better quant fits your rig.
Set your rig once; we'll alert you when a new or better-fitting build lands.
Performance reports
Real-world throughput reported by the community (and scraped sources).
Community
Reviews, sampler presets and community runtime reports. Be the first to contribute.
Community runtime reports
Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.
Log in to report whether a quant loads in your runtime.
No community reports yet.