Files and memory at 8K context
No verified quantized file for the requested formats.
Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory. Legacy model: sizes on device pages are estimated from parameter count.
Quality
Artificial Analysis Intelligence Index
Unknown
Arena text rating
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 125
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| Mac Studio M1 Ultra (64 core GPU, 128GB) | Q4_K_M | llama.cpp Metal | 896 | 68.3 | 2026-02-07 | ai-benchmarks #48 |
| Mac Studio M1 Ultra (64 core GPU, 128GB) | Q4_K_M | llama.cpp Metal | 844 | — | 2026-02-07 | ai-benchmarks #48 |
| Framework Desktop Mainboard (AMD Ryzen AI Max+ 395, 128 GB) | F16 | llama.cpp Vulkan | 217 | 33.1 | 2025-08-07 | ai-benchmarks #21 |
| Apple M3 Max (40-core GPU), 128 GB unified memory | MXFP4 (MLX, oMLX) | MLX | 665 | 64.6 | 2026-09-29 | omlx.ai |
| Dell Pro Max with GB10 (NVIDIA GB10, 128GB; DGX Spark OEM) | F16 | llama.cpp CUDA | 1,640 | 38.8 | 2026-06-21 | openbenchmarking.org |
| Dell Pro Max with GB10 (NVIDIA GB10, 128GB; DGX Spark OEM) | F16 | llama.cpp CUDA | 1,787 | — | 2026-06-21 | openbenchmarking.org |
| Dell Pro Max with GB10 (NVIDIA GB10, 128GB; DGX Spark OEM) | Q8_0 | llama.cpp CUDA | 1,754 | 50.9 | 2026-06-21 | openbenchmarking.org |
| Dell Pro Max with GB10 (NVIDIA GB10, 128GB; DGX Spark OEM) | Q8_0 | llama.cpp CUDA | 1,824 | — | 2026-06-21 | openbenchmarking.org |
| Apple M2 Max (38-core GPU), 96 GB unified memory | MXFP4 (MLX, oMLX) | MLX | 331 | 46.2 | 2026-06-02 | omlx.ai |
| NVIDIA DGX Spark (GB10) | MXFP4 | llama.cpp CUDA | 2,444 | 58.7 | 2026-02-06 | raw.githubusercontent.com |
| NVIDIA DGX Spark (GB10) | MXFP4 | llama.cpp CUDA | 1,567 | 42.8 | 2026-02-06 | raw.githubusercontent.com |
| Apple Mac Studio M2 Ultra (per the bench directory name "mac-m2-ultra"; GPU core count not stated) | MXFP4 | llama.cpp Metal | 1,649 | 85.6 | 2026-02-06 | raw.githubusercontent.com |
| Apple Mac Studio M2 Ultra (per the bench directory name "mac-m2-ultra"; GPU core count not stated) | MXFP4 | llama.cpp Metal | 721 | 65.5 | 2026-02-06 | raw.githubusercontent.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | MXFP4 | llama.cpp CUDA | 1,917 | 59.5 | 2026-01-02 | forum.level1techs.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | MXFP4 | llama.cpp CUDA | 1,836 | 56.1 | 2026-01-02 | forum.level1techs.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | MXFP4 | llama.cpp CUDA | 1,734 | 51.9 | 2026-01-02 | forum.level1techs.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | MXFP4 | llama.cpp CUDA | 1,531 | 48.9 | 2026-01-02 | forum.level1techs.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | MXFP4 | llama.cpp CUDA | 1,236 | 41.9 | 2026-01-02 | forum.level1techs.com |
| NVIDIA DGX Spark (repasted with PTM7950) | MXFP4 | llama.cpp CUDA | 1,608 | 48.5 | 2025-12-08 | forum.level1techs.com |
| NVIDIA DGX Spark (repasted with PTM7950) | MXFP4 | llama.cpp CUDA | 1,558 | 46 | 2025-12-08 | forum.level1techs.com |
| NVIDIA DGX Spark (repasted with PTM7950) | MXFP4 | llama.cpp CUDA | 1,507 | 43.3 | 2025-12-08 | forum.level1techs.com |
| NVIDIA DGX Spark (repasted with PTM7950) | MXFP4 | llama.cpp CUDA | 1,308 | 40 | 2025-12-08 | forum.level1techs.com |
| NVIDIA DGX Spark (repasted with PTM7950) | MXFP4 | llama.cpp CUDA | 1,061 | 34.5 | 2025-12-08 | forum.level1techs.com |
| MSI EdgeXpert (GB10, DGX Spark OEM) | MXFP4 | llama.cpp CUDA | 1,729 | 52.6 | 2025-12-08 | forum.level1techs.com |
| MSI EdgeXpert (GB10, DGX Spark OEM) | MXFP4 | llama.cpp CUDA | 1,681 | 49.1 | 2025-12-08 | forum.level1techs.com |
Reported by the linked sources, not reproduced by LocalFit. Showing 25 of 125. All rows in the speed database