Mistral Medium 3.5 (128B dense)
Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q4_K_M unsloth/Mistral-Medium-3.5-128B-GGUF | 69.75 GiB | 3 | 73.5 GiB+ | 80 GB GPU | HF files |
| Q5_K_M unsloth/Mistral-Medium-3.5-128B-GGUF | 82.25 GiB | 3 | 86.0 GiB+ | 96 GB GPU | HF files |
| Q8_0 unsloth/Mistral-Medium-3.5-128B-GGUF | 124 GiB | 4 | 127 GiB+ | 128 GB GPU | HF files |
| BF16 unsloth/Mistral-Medium-3.5-128B-GGUF | 233 GiB | 6 | 237 GiB+ | over 192 GB GPU | HF files |
| MLX 4-bit mlx-community/Mistral-Medium-3.5-128B-4bit | 72.65 GiB | 15 | 76.4 GiB+ | 128 GB Mac | HF files |
No verified file: FP16, MLX 8-bit. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
14.2
Variant: Mistral Medium 3.5 · observed 2026-10-17 · source
Rank #43 of 75 researched 2026 models.
Arena text rating
1,421
95% CI 1415–1427 · 11,778 votes · published 2026-10-15 · dataset
Exact listed model name; hosted reasoning/quantization may differ from local checkpoint.
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 5
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| Apple M1 Ultra (48-core GPU), 128 GB unified memory | MLX 4-bit | MLX | 34.1 | 6.3 | 2026-09-27 | omlx.ai |
| Apple M3 Ultra (80-core GPU), 256 GB unified memory | MLX 4-bit | MLX | 91.2 | 9.5 | 2026-08-11 | omlx.ai |
| Apple M5 Max (40-core GPU), 128 GB unified memory | MLX 4-bit | MLX | 194 | 6.5 | 2026-06-07 | omlx.ai |
| Apple M2 Ultra (60-core GPU), 128 GB unified memory | MLX 4-bit | MLX | 37.4 | 6.0 | 2026-05-15 | omlx.ai |
| Apple M3 Max (40-core GPU), 128 GB unified memory | MLX 4-bit | MLX | 44.9 | 4.3 | 2026-05-03 | omlx.ai |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database