Mellum2.1-12B-A2.5B-Thinking
Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q4_K_M JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF | 7.52 GiB | 1 | 8.7 GiB | 12 GB GPU | HF files |
| Q5_K_M bartowski/Mellum2.1-12B-A2.5B-Thinking-GGUF | 9.28 GiB | 1 | 10.4 GiB | 12 GB GPU | HF files |
| Q8_0 JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF | 12.04 GiB | 1 | 13.2 GiB | 16 GB GPU | HF files |
No verified file: FP16, MLX 4-bit, MLX 8-bit. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
Unknown
Arena text rating
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 3
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| 16 GB M4 MacBook Pro (GPU core count not stated) | MLX 6-bit (affine, group 64; routers 8-bit) | MLX | — | 47 | 2026-10-11 | huggingface.co |
| 16 GB M4 MacBook Pro (GPU core count not stated) | MLX 6-bit (affine, group 64; routers 8-bit) | MLX | — | 46 | 2026-10-11 | huggingface.co |
| 16 GB M4 MacBook Pro (GPU core count not stated) | MLX 6-bit (affine, group 64; routers 8-bit) | MLX | — | 43 | 2026-10-11 | huggingface.co |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database
Community threads
- When is MTP coming? Hugging Face · 2026-10-17
- Appreciation and image processing Hugging Face · 2026-10-13
Qualitative signals only; titles are not verified claims.