MiMo-V2.6-Distill-Qwen-9B
Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q4_K_M bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF | 5.44 GiB | 1 | 6.7 GiB+ | 8 GB GPU | HF files |
| Q5_K_M bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF | 6.40 GiB | 1 | 7.7 GiB+ | 12 GB GPU | HF files |
| Q8_0 bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF | 8.89 GiB | 1 | 10.1 GiB+ | 12 GB GPU | HF files |
| FP16 mradermacher/MiMo-V2.6-Distill-Qwen-9B-GGUF | 16.69 GiB | 1 | 17.9 GiB+ | 20 GB GPU | HF files |
| MLX 4-bit agnosticeng/MiMo-V2.6-Distill-Qwen-9B-4bit | 5.54 GiB | 2 | 6.8 GiB+ | 16 GB Mac | HF files |
| MLX 8-bit nativ-community/MiMo-V2.6-Distill-Qwen-9B-MLX-8bit | 9.71 GiB | 2 | 11.0 GiB+ | 16 GB Mac | HF files |
Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
Unknown
Arena text rating
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 3
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| Apple M3 Pro,18GiB unified memory | Q3_K_M | llama.cpp (Darwin arm64) | — | 14.4 | 2026-09-22 | source |
| Apple M3 Pro,18GiB unified memory | Q3_K_M | llama.cpp (Darwin arm64) | — | 13.7 | 2026-09-22 | source |
| Apple M3 Pro,18GiB unified memory | Q3_K_M | llama.cpp (Darwin arm64) | — | 15.3 | 2026-09-22 | source |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database
Community threads
- GGUF Q8_0 Forensic Benchmark Audit (llama.cpp / Ollama v0.40.4, RTX 5080): 12x Faster than Base Qwen3.5-9B at Equal 96% GSM8K Ceiling + Thermal Sweep (T=0.0–0.3) Hugging Face · 2026-10-16
- GGUF/llama.cpp results: YaRN 512K works: 40/40 needle retrieval at 512K, zero short-task quality loss, ~660 tok/s prefill (llama.cpp, RX 7900 XTX) Hugging Face · 2026-10-09
Qualitative signals only; titles are not verified claims.