Files and memory at 8K context
No verified quantized file for the requested formats.
Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory. Legacy model: sizes on device pages are estimated from parameter count.
Quality
Artificial Analysis Intelligence Index
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 30
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|
| NVIDIA DGX Spark | Q4_K_M | Ollama | — | 24.3 | 2025-10-23 | ollama.com |
| NVIDIA DGX Spark | Q8_0 | Ollama | — | 15.5 | 2025-10-23 | ollama.com |
| NVIDIA DGX Spark | Q4_K_M | Ollama | — | 26.5 | 2025-10-13 | docs.google.com |
| NVIDIA DGX Spark | Q8_0 | Ollama | — | 16.1 | 2025-10-13 | docs.google.com |
| RTX Pro 6000 Blackwell Edition | Q4_K_M | Ollama | — | 112 | 2025-10-13 | docs.google.com |
| RTX Pro 6000 Blackwell Edition | Q8_0 | Ollama | — | 82.8 | 2025-10-13 | docs.google.com |
| GeForce RTX 5090 | Q4_K_M | Ollama | — | 112 | 2025-10-13 | docs.google.com |
| GeForce RTX 5090 | Q8_0 | Ollama | — | 84.4 | 2025-10-13 | docs.google.com |
| GeForce RTX 5080 | Q4_K_M | Ollama | — | 76.1 | 2025-10-13 | docs.google.com |
| GeForce RTX 5080 | Q8_0 | Ollama | — | 53 | 2025-10-13 | docs.google.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | Q4_0 (QAT) | llama.cpp CUDA | 2,043 | 31.8 | 2026-01-02 | forum.level1techs.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | Q4_0 (QAT) | llama.cpp CUDA | 1,808 | 29.6 | 2026-01-02 | forum.level1techs.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | Q4_0 (QAT) | llama.cpp CUDA | 1,682 | 27.5 | 2026-01-02 | forum.level1techs.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | Q4_0 (QAT) | llama.cpp CUDA | 1,442 | 26.6 | 2026-01-02 | forum.level1techs.com |
| Gigabyte AI TOP (GB10, DGX Spark OEM) | Q4_0 (QAT) | llama.cpp CUDA | 1,212 | 22.4 | 2026-01-02 | forum.level1techs.com |
| GB10 DGX Spark-class system (DGX Spark or MSI EdgeXpert; unit not specified) | Q4_0 (QAT) | llama.cpp CUDA | 1,783 | 26.9 | 2025-12-08 | forum.level1techs.com |
| GB10 DGX Spark-class system (DGX Spark or MSI EdgeXpert; unit not specified) | Q4_0 (QAT) | llama.cpp CUDA | 1,639 | 25.4 | 2025-12-08 | forum.level1techs.com |
| GB10 DGX Spark-class system (DGX Spark or MSI EdgeXpert; unit not specified) | Q4_0 (QAT) | llama.cpp CUDA | 1,499 | 23.4 | 2025-12-08 | forum.level1techs.com |
| GB10 DGX Spark-class system (DGX Spark or MSI EdgeXpert; unit not specified) | Q4_0 (QAT) | llama.cpp CUDA | 1,329 | 22.5 | 2025-12-08 | forum.level1techs.com |
| GB10 DGX Spark-class system (DGX Spark or MSI EdgeXpert; unit not specified) | Q4_0 (QAT) | llama.cpp CUDA | 1,112 | 19.2 | 2025-12-08 | forum.level1techs.com |
| NVIDIA RTX PRO 4500 Blackwell | Q4_K_M | llama.cpp CUDA | 4,075 | 86 | 2025-11-14 | llama.cpp #15013 |
| NVIDIA RTX PRO 4500 Blackwell | Q4_K_M | llama.cpp CUDA | 4,365 | 88.4 | 2025-11-14 | llama.cpp #15013 |
| NVIDIA DGX Spark | FP8 | SGLang | 1,296 | 6.8 | 2025-10-13 | docs.google.com |
| RTX Pro 6000 Blackwell Edition | FP8 | SGLang | 13,372 | 44.1 | 2025-10-13 | docs.google.com |
| Mac Studio M1 Max | Q4_K_M | Ollama | — | 26.5 | 2025-10-13 | docs.google.com |
Reported by the linked sources, not reproduced by LocalFit. Showing 25 of 30. All rows in the speed database