Qwen3.5-9B
Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q4_K_M unsloth/Qwen3.5-9B-GGUF | 5.29 GiB | 1 | 6.5 GiB+ | 8 GB GPU | HF files |
| Q5_K_M unsloth/Qwen3.5-9B-GGUF | 6.13 GiB | 1 | 7.4 GiB+ | 8 GB GPU | HF files |
| Q8_0 unsloth/Qwen3.5-9B-GGUF | 8.87 GiB | 1 | 10.1 GiB+ | 12 GB GPU | HF files |
| MLX 4-bit mlx-community/Qwen3.5-9B-4bit | 5.54 GiB | 2 | 6.8 GiB+ | 16 GB Mac | HF files |
| MLX 8-bit mlx-community/Qwen3.5-9B-8bit | 9.71 GiB | 2 | 11.0 GiB+ | 16 GB Mac | HF files |
No verified file: FP16. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
13.3 AA estimate
Variant: Qwen3.5 9B (Non-reasoning) · observed 2026-10-17 · source
Rank #45 of 75 researched 2026 models.
Arena text rating
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 1
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| MacBook Air M5, 32 GB (10 CPU / 10 GPU cores) | Q4_K_M | llama.cpp Metal | 226 | 13.2 | 2026-04-06 | r/LocalLLaMA |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database
Community threads
- 24-Layer GGUF quants (3.71GB) optimized for 6GB consumer GPUs Hugging Face · 2026-09-27
- MTP provides no throughput improvement on Apple Silicon Metal (llama.cpp b9330) — net loss at every configuration Hugging Face · 2026-05-27
- https://www.reddit.com/r/LocalLLaMA/comments/1x7g8cc/why_do_we_have_models_under_15b_so_rarely_anymore/ Reddit · 2026-10-16
Qualitative signals only; titles are not verified claims.