HuggingFaceTB
SmolLM3-3B
3.1B parameters · reasoning · Instruct · SmolLM3 family
Recommended download
Which version should I download?
Default pick: GGUF · Q4_K_M — Unsloth Q4_K_M quant of SmolLM3-3B (3.075B dense) — recommended pick for CPU and edge devices. Apache-2.0.
Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.
| Format | Level | Size | Verdict | Est. speed | Quality note | Swarm | Download |
|---|---|---|---|---|---|---|---|
| GGUF | IQ4_XS | 1.61 GB | Set your rig | — | Unsloth IQ4_XS quant of SmolLM3-3B (3.075B dense) — compact low-bit option for CPU and edge devices. Apache-2.0. | 2S / 0L webseed OK | |
| GGUF | Q4_K_M | 1.78 GB | Set your rig | — | Unsloth Q4_K_M quant of SmolLM3-3B (3.075B dense) — recommended pick for CPU and edge devices. Apache-2.0. | 2S / 0L webseed OK | |
| GGUF | Q8_0 | 3.05 GB | Set your rig | — | Unsloth Q8_0 quant of SmolLM3-3B (3.075B dense) — high-quality option. Apache-2.0. | 2S / 0L webseed OK | |
| GGUF | BF16 | 5.74 GB | Set your rig | — | Unsloth BF16 GGUF of SmolLM3-3B (3.075B dense) — source-quality preservation copy (small model). Apache-2.0. | 2S / 0L webseed OK |
GGUF · IQ4_XS
1.61 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Unsloth IQ4_XS quant of SmolLM3-3B (3.075B dense) — compact low-bit option for CPU and edge devices. Apache-2.0.
GGUF · Q4_K_M
1.78 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Unsloth Q4_K_M quant of SmolLM3-3B (3.075B dense) — recommended pick for CPU and edge devices. Apache-2.0.
GGUF · Q8_0
3.05 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Unsloth Q8_0 quant of SmolLM3-3B (3.075B dense) — high-quality option. Apache-2.0.
GGUF · BF16
5.74 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Unsloth BF16 GGUF of SmolLM3-3B (3.075B dense) — source-quality preservation copy (small model). Apache-2.0.
Want a different quant? Request it on the board →
Runs fully on
Green = the model's best quant fits fully in GPU/unified memory at 8K context. Tap a card for its full "what runs on it" page.
Context
Advertised 64k · usable ≈ 32–64k (measured — RULER)
How we know
Evidence grade: measured — a published evaluation we can cite, with task and length stated.
- SmolLM3-3B model card (RULER) — observed Jul 18, 2026
- SmolLM3 release blog — observed Jul 18, 2026
Model card RULER (base): 76.35 at 32k, 67.85 at 64k, 61.03 at 128k — degradation visible from 32k. Blog: trained to 64k (progressive extension); 128k is YaRN 2x extrapolation.
Reviewed on Jul 18, 2026.
Run it
Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –
- – Ollama — runs the GGUF directly; no Modelfile bundled
- – vision sidecar — not a vision model
Context / KV 8,192 tokens · FP16 set above the quant table
Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.
llama-server -m SmolLM3-3B-IQ4_XS.gguf -c 8192 -ngl 999
Use llama-cli in place of llama-server for a one-shot prompt.
FROM ./SmolLM3-3B-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).
ollama create huggingfacetb-smollm3-3b -f Modelfile
ollama run huggingfacetb-smollm3-3b
GGUF is not a first-class vLLM format.
vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.
transformers does not load GGUF weights.
GGUF weights run under llama.cpp, Ollama or LM Studio.
Load SmolLM3-3B-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).
MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.
GGUF weights run under llama.cpp, Ollama or LM Studio.
Technical details
Chat template
{# ───── defaults ───── #}
{%- if enable_thinking is not defined -%}
{%- set enable_thinking = true -%}
{%- endif -%}
{# ───── reasoning mode ───── #}
{%- if enable_thinking -%}
{%- set reasoning_mode = "/think" -%}
{%- else -%}
{%- set reasoning_mode = "/no_think" -%}
{%- endif -%}
{# ───── header (system message) ───── #}
{{- "<|im_start|>system\n" -}}
{%- if messages[0].role == "system" -%}
{%- set system_message = messages[0].content -%}
{%- if "/no_think" in system_message -%}
{%- set reasoning_mode = "/no_think" -%}
{%- elif "/think" in system_message -%}
{%- set reasoning_mode = "/think" -%}
{%- endif -%}
{%- set custom_instructions = system_message.replace("/no_think", "").replace("/think", "").rstrip() -%}
{%- endif -%}
{%- if "/system_override" in system_message -%}
{{- custom_instructions.replace("/system_override", "").rstrip() -}}
{{- "<|im_end|>\n" -}}
{%- else -%}
{{- "## Metadata\n\n" -}}
{{- "Knowledge Cutoff Date: June 2025\n" -}}
{%- set today = strftime_now("%d %B %Y") -%}
{{- "Today Date: " ~ today ~ "\n" -}}
{{- "Reasoning Mode: " + reasoning_mode + "\n\n" -}}
{{- "## Custom Instructions\n\n" -}}
{%- if custom_instructions -%}
{{- custom_instructions + "\n\n" -}}
{%- elif reasoning_mode == "/think" -%}
{{- "You are a helpful AI assistant named SmolLM, trained by Hugging Face. Your role as an assistant involves thoroughly exploring questions through a systematic thinking process before providing the final precise and accurate solutions. This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracking, and iteration to develop well-considered thinking process. Please structure your response into two main sections: Thought and Solution using the specified format: <think> Thought section </think> Solution section. In the Thought section, detail your reasoning process in steps. Each step should include detailed considerations such as analysing questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps. In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The Solution section should be logical, accurate, and concise and detail necessary steps needed to reach the conclusion.\n\n" -}}
{%- else -%}
{{- "You are a helpful AI assistant named SmolLM, trained by Hugging Face.\n\n" -}}
{%- endif -%}
{%- if xml_tools or python_tools or tools -%}
{{- "### Tools\n\n" -}}
{%- if xml_tools or tools -%}
{%- if tools -%}
{%- set xml_tools = tools -%}
{%- endif -%}
{%- set ns = namespace(xml_tool_string="You may call one or more functions to assist with the user query.\nYou are provided with function signatures within <tools></tools> XML tags:\n\n<tools>\n") -%}
{%- for tool in xml_tools[:] -%} {# The slicing makes sure that xml_tools is a list #}
{%- set ns.xml_tool_string = ns.xml_tool_string ~ (tool | string) ~ "\n" -%}
{%- endfor -%}
{%- set xml_tool_string = ns.xml_tool_string + "</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>" -%}
{{- xml_tool_string -}}
{%- endif -%}
{%- if python_tools -%}
{%- set ns = namespace(python_tool_string="When you send a message containing Python code between '<code>' and '</code>' tags, it will be executed in a stateful Jupyter notebook environment, and you will then be given the output to continued reasoning in an agentic loop.\n\nYou can use the following tools in your python code like regular functions:\n<tools>\n") -%}
{%- for tool in python_tools[:] -%} {# The slicing makes sure that python_tools is a list #}
{%- set ns.python_tool_string = ns.python_tool_string ~ (tool | string) ~ "\n" -%}
{%- endfor -%}
{%- set python_tool_string = ns.python_tool_string + "</tools>\n\nThe state persists between code executions: so variables that you define in one step are still available thereafter." -%}
{{- python_tool_string -}}
{%- endif -%}
{{- "\n\n" -}}
{{- "<|im_end|>\n" -}}
{%- endif -%}
{%- endif -%}
{# ───── main loop ───── #}
{%- for message in messages -%}
{%- set content = message.content if message.content is string else "" -%}
{%- if message.role == "user" -%}
{{ "<|im_start|>" + message.role + "\n" + content + "<|im_end|>\n" }}
{%- elif message.role == "assistant" -%}
{% generation %}
{%- if reasoning_mode == "/think" -%}
{{ "<|im_start|>assistant\n" + content.lstrip("\n") + "<|im_end|>\n" }}
{%- else -%}
{{ "<|im_start|>assistant\n" + "<think>\n\n</think>\n" + content.lstrip("\n") + "<|im_end|>\n" }}
{%- endif -%}
{% endgeneration %}
{%- elif message.role == "tool" -%}
{{ "<|im_start|>" + "user\n" + content + "<|im_end|>\n" }}
{%- endif -%}
{%- endfor -%}
{# ───── generation prompt ───── #}
{%- if add_generation_prompt -%}
{%- if reasoning_mode == "/think" -%}
{{ "<|im_start|>assistant\n" }}
{%- else -%}
{{ "<|im_start|>assistant\n" + "<think>\n\n</think>\n" }}
{%- endif -%}
{%- endif -%}
Sampling defaults
- top_p
- 0.95
- temperature
- 0.6
Stop strings
<|im_end|>
Evidence & provenance
Source
- Revision pin
-
a07cc9a04f16550a088caea529712d1d335b0ac1 - Manifest
- Present
License
- Name
- apache-2.0
- Commercial use
- yes
- Access
- Open
File hashes (SHA-256)
-
smollm3-3b-iq4-xs/LICENSE
cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 -
smollm3-3b-iq4-xs/README.md
cc3e2985fe1b789e2c7b9be5fe0d226d09e1e52bf030fe049276bf9d64a571b8 -
smollm3-3b-iq4-xs/SmolLM3-3B-IQ4_XS.gguf
c1ab400f2c4375d3c949af0d6ea4ef4ac71691430bb8b9d92af887a400ee7306 -
smollm3-3b-q4-k-m/LICENSE
cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 -
smollm3-3b-q4-k-m/README.md
cc3e2985fe1b789e2c7b9be5fe0d226d09e1e52bf030fe049276bf9d64a571b8 -
smollm3-3b-q4-k-m/SmolLM3-3B-Q4_K_M.gguf
4de907d2d388a5508fb7cb443a06effe14cce3518b0a78d3bdd9e74d9edce989 -
smollm3-3b-q8-0/LICENSE
cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 -
smollm3-3b-q8-0/README.md
cc3e2985fe1b789e2c7b9be5fe0d226d09e1e52bf030fe049276bf9d64a571b8 -
smollm3-3b-q8-0/SmolLM3-3B-Q8_0.gguf
5bd828e684d883459a03ce1eec9dc9044345c6da64075d9ae7e890dedc68de28 -
smollm3-3b-bf16/LICENSE
cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 -
smollm3-3b-bf16/README.md
cc3e2985fe1b789e2c7b9be5fe0d226d09e1e52bf030fe049276bf9d64a571b8 -
smollm3-3b-bf16/SmolLM3-3B-BF16.gguf
3082a0be39c4014306e8aa688b2f8051468fe077fc99835ad576e21e83f93788
Explore further
Appears in
Get an email when a better quant fits your rig.
Set your rig once; we'll alert you when a new or better-fitting build lands.
Performance reports
Real-world throughput reported by the community (and scraped sources).
Community
Reviews, sampler presets and community runtime reports. Be the first to contribute.
Community runtime reports
Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.
Log in to report whether a quant loads in your runtime.
No community reports yet.