Qwen

Qwen2.5-VL-32B-Instruct

33.5B parameters · Instruct · Qwen2.5 family

Long context — how it uses memory →
apache-2.0 hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Bartowski Q4_K_M quant of Qwen2.5-VL-32B-Instruct (33.453B vision-language) — recommended balanced pick; bundled mmproj (bf16+f16) required for vision. Apache-2.0.

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick? What do these mean?

GGUF · IQ4_XS

16.48 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski IQ4_XS quant of Qwen2.5-VL-32B-Instruct (33.453B vision-language) — compact low-bit option; bundled mmproj (bf16+f16) required for vision. Apache-2.0.

Magnet .torrent

GGUF · Q4_K_M

18.49 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski Q4_K_M quant of Qwen2.5-VL-32B-Instruct (33.453B vision-language) — recommended balanced pick; bundled mmproj (bf16+f16) required for vision. Apache-2.0.

Magnet .torrent

GGUF · Q8_0

32.43 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski Q8_0 quant of Qwen2.5-VL-32B-Instruct (33.453B vision-language) — high-quality option; bundled mmproj (bf16+f16) required for vision. Apache-2.0.

Magnet .torrent

Want a different quant? Request it on the board →

Runs fully on

RTX 5090 · Q4_K_M RTX 4090 · Q4_K_M RTX 3090 · Q4_K_M RTX 3090 Ti · Q4_K_M RTX Pro 6000 Blackwell 96GB · Q8_0

Green = the model's best quant fits fully in GPU/unified memory at 8K context. Tap a card for its full "what runs on it" page.

Context

Advertised 125k · no independent evidence yet

No independent long-context evidence has been graded for this model yet — the advertised window above is the maintainer's number, not a usable-context claim.

Run it

Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –

  • – Ollama — runs the GGUF directly; no Modelfile bundled
  • – vision sidecar — not a vision model

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server 
llama-server -m Qwen_Qwen2.5-VL-32B-Instruct-IQ4_XS.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile) 
FROM ./Qwen_Qwen2.5-VL-32B-Instruct-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.000001
PARAMETER repeat_penalty 1.05
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).
Create 
ollama create qwen-qwen2-5-vl-32b-instruct -f Modelfile
Run 
ollama run qwen-qwen2-5-vl-32b-instruct

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

transformers does not load GGUF weights.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Load Qwen_Qwen2.5-VL-32B-Instruct-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Technical details

Chat template

{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
You are a helpful assistant.<|im_end|>
{% endif %}<|im_start|>{{ message['role'] }}
{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
{% endif %}

Sampling defaults

temperature
1.0E-6
repetition_penalty
1.05

Stop strings

<|im_end|> <|endoftext|>

Evidence & provenance

Source

Provenance
https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct
Revision pin
7cfb30d71a1f4f49a57592323337a4a4727301da
Manifest
Present

License

Name
apache-2.0
Commercial use
yes
Access
Open

How verification works →

File hashes (SHA-256)

  • qwen2-5-vl-32b-instruct-iq4-xs/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • qwen2-5-vl-32b-instruct-iq4-xs/Qwen_Qwen2.5-VL-32B-Instruct-IQ4_XS.gguf 5cc568114dca473878f1e6467c534c68c0de31cb65ee279e4a358e8ccd6ed1eb
  • qwen2-5-vl-32b-instruct-iq4-xs/README.md 5f60e0526b1bc2d2365c2287f17a68153c32c092b6f0edbfd965a9ee2266b90e
  • qwen2-5-vl-32b-instruct-iq4-xs/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-bf16.gguf ae31b1160f891180557e0ad6c6ac4875d07a074af88d2c4f438fbd8ccefbbca7
  • qwen2-5-vl-32b-instruct-iq4-xs/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-f16.gguf 8b07ad34435e512d4d0467d12dcbec49cbebc378e18c02b26c90fa41d8a5c7b9
  • qwen2-5-vl-32b-instruct-q4-k-m/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • qwen2-5-vl-32b-instruct-q4-k-m/Qwen_Qwen2.5-VL-32B-Instruct-Q4_K_M.gguf 446ce4e5e746513f12e408c7936c41a219e2661a3f462469babe2936c402b516
  • qwen2-5-vl-32b-instruct-q4-k-m/README.md 5f60e0526b1bc2d2365c2287f17a68153c32c092b6f0edbfd965a9ee2266b90e
  • qwen2-5-vl-32b-instruct-q4-k-m/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-bf16.gguf ae31b1160f891180557e0ad6c6ac4875d07a074af88d2c4f438fbd8ccefbbca7
  • qwen2-5-vl-32b-instruct-q4-k-m/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-f16.gguf 8b07ad34435e512d4d0467d12dcbec49cbebc378e18c02b26c90fa41d8a5c7b9
  • qwen2-5-vl-32b-instruct-q8-0/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • qwen2-5-vl-32b-instruct-q8-0/Qwen_Qwen2.5-VL-32B-Instruct-Q8_0.gguf eda114feaa726d2bd43de23460de28d2f704f45a033f5d4021c2ee7e28935e82
  • qwen2-5-vl-32b-instruct-q8-0/README.md 5f60e0526b1bc2d2365c2287f17a68153c32c092b6f0edbfd965a9ee2266b90e
  • qwen2-5-vl-32b-instruct-q8-0/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-bf16.gguf ae31b1160f891180557e0ad6c6ac4875d07a074af88d2c4f438fbd8ccefbbca7
  • qwen2-5-vl-32b-instruct-q8-0/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-f16.gguf 8b07ad34435e512d4d0467d12dcbec49cbebc378e18c02b26c90fa41d8a5c7b9

Explore further

Appears in

What runs on 24 GB of VRAM What runs on 32 GB of VRAM What runs on 48 GB of VRAM What runs on a 36 GB Mac What runs on a 48 GB Mac What runs on a 64 GB Mac

Also consider

Get an email when a better quant fits your rig.

Set your rig once; we'll alert you when a new or better-fitting build lands.

Set your rig

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard
Community

Reviews, sampler presets and community runtime reports. Be the first to contribute.

Reviews (0)

Log in to write a structured review of this model.

No reviews yet.

Sampler presets

Log in to share a sampler preset or vote on presets.

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

Log in to report whether a quant loads in your runtime.

No community reports yet.

selected to compare · pick at least 2

Compare ()