Files and memory at 8K context
No verified quantized file for the requested formats.
Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory. Legacy model: sizes on device pages are estimated from parameter count.
Quality
Artificial Analysis Intelligence Index
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 10
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|
| Mac Studio M1 Ultra (64 core GPU, 128GB) | Q4_K_M | llama.cpp Metal | 1,416 | 62.7 | 2026-02-07 | ai-benchmarks #48 |
| Mac Studio M1 Ultra (64 core GPU, 128GB) | Q4_K_M | llama.cpp Metal | 1,209 | — | 2026-02-07 | ai-benchmarks #48 |
| 64GB M4 Max | MLX 8-bit | MLX | 1,719 | 83.2 | 2025-10-15 | ml-explore/mlx-lm |
| 64GB M4 Max | MLX 6-bit | MLX | 1,667 | 94.1 | 2025-10-15 | ml-explore/mlx-lm |
| 64GB M4 Max | MLX 5-bit | MLX | 1,664 | 101 | 2025-10-15 | ml-explore/mlx-lm |
| 64GB M4 Max | MLX 4-bit | MLX | 1,754 | 113 | 2025-10-15 | ml-explore/mlx-lm |
| Framework Desktop (AMD Ryzen AI Max+ 395, 128GB LPDDR5x-8000, pre-production) | UD-Q4_K_XL | llama.cpp ROCm | 669 | 58.5 | 2025-08-08 | lhl/strix-halo-testing |
| Framework Desktop (AMD Ryzen AI Max+ 395, 128GB LPDDR5x-8000, pre-production) | UD-Q4_K_XL | llama.cpp Vulkan | 646 | 78 | 2025-08-08 | lhl/strix-halo-testing |
| Framework Desktop (AMD Ryzen AI Max+ 395, 128GB LPDDR5x-8000, pre-production) | UD-Q4_K_XL | llama.cpp Vulkan | 605 | 66.3 | 2025-07-22 | forum.level1techs.com |
| Framework Desktop (AMD Ryzen AI Max+ 395, 128GB LPDDR5x-8000, pre-production) | UD-Q4_K_XL | llama.cpp Vulkan | 591 | 72 | 2025-07-22 | forum.level1techs.com |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database