Files and memory at 8K context
No verified quantized file for the requested formats.
Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory. Legacy model: sizes on device pages are estimated from parameter count.
Quality
Artificial Analysis Intelligence Index
Unknown
Arena text rating
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 25
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| NVIDIA GeForce RTX 4090 (Intel Core Ultra 265K custom PC) | Q4_K_M | llama.cpp Vulkan | 2,302 | 42.5 | 2025-12-09 | ai-benchmarks #43 |
| NVIDIA GeForce RTX 4090 (Intel Core Ultra 265K custom PC) | Q4_K_M | llama.cpp Vulkan | 1,598 | — | 2025-12-09 | ai-benchmarks #43 |
| AMD Radeon AI Pro R9700 32GB (Intel Core Ultra 7 265K system) | Q4_K_M | llama.cpp Vulkan | 415 | 25 | 2025-11-25 | ai-benchmarks #35 |
| AMD Radeon AI Pro R9700 32GB (Intel Core Ultra 7 265K system) | Q4_K_M | llama.cpp Vulkan | 366 | — | 2025-11-25 | ai-benchmarks #35 |
| NVIDIA DGX Spark | Q4_K_M | Ollama | 705 | 9.4 | 2025-10-23 | ollama.com |
| NVIDIA DGX Spark | Q8_0 | Ollama | 487 | 6.2 | 2025-10-23 | ollama.com |
| NVIDIA DGX Spark | Q4_K_M | Ollama | — | 9.5 | 2025-10-13 | docs.google.com |
| NVIDIA DGX Spark | Q8_0 | Ollama | — | 6.2 | 2025-10-13 | docs.google.com |
| RTX Pro 6000 Blackwell Edition | Q4_K_M | Ollama | — | 57.1 | 2025-10-13 | docs.google.com |
| RTX Pro 6000 Blackwell Edition | Q8_0 | Ollama | — | 38.4 | 2025-10-13 | docs.google.com |
| GeForce RTX 5090 | Q4_K_M | Ollama | — | 58.9 | 2025-10-13 | docs.google.com |
| Framework Desktop (AMD Ryzen AI Max+ 395, 128GB LPDDR5x-8000, pre-production) | Q8_0 | llama.cpp ROCm | 226 | 6.4 | 2025-08-08 | lhl/strix-halo-testing |
| Framework Desktop (AMD Ryzen AI Max+ 395, 128GB LPDDR5x-8000, pre-production) | Q8_0 | llama.cpp Vulkan | 102 | 6.4 | 2025-08-08 | lhl/strix-halo-testing |
| Mac Studio M3 Ultra 512GB | Q4_K_M | llama.cpp Metal | 335 | 26.7 | 2025-12-08 | ai-benchmarks #42 |
| Mac Studio M3 Ultra 512GB | Q4_K_M | llama.cpp Metal | 306 | — | 2025-12-08 | ai-benchmarks #42 |
| Tesla M40 24GB | Q4_K_M | llama.cpp CUDA | 60.7 | 6.9 | 2025-11-19 | llama.cpp #15013 |
| Tesla M40 24GB | Q4_K_M | llama.cpp CUDA | 61.4 | 7.2 | 2025-11-19 | llama.cpp #15013 |
| NVIDIA RTX PRO 4500 Blackwell | Q4_K_M | llama.cpp CUDA | 1,514 | 36.4 | 2025-11-14 | llama.cpp #15013 |
| NVIDIA RTX PRO 4500 Blackwell | Q4_K_M | llama.cpp CUDA | 1,676 | 37.9 | 2025-11-14 | llama.cpp #15013 |
| NVIDIA DGX Spark | FP8 | SGLang | 1,146 | 6.1 | 2025-10-13 | docs.google.com |
| RTX Pro 6000 Blackwell Edition | FP8 | SGLang | 6,807 | 23.4 | 2025-10-13 | docs.google.com |
| Mac Studio M1 Max | Q4_K_M | Ollama | — | 10.4 | 2025-10-13 | docs.google.com |
| Mac Studio M1 Max | Q8_0 | Ollama | — | 8.1 | 2025-10-13 | docs.google.com |
| Tesla V100-SXM2-32GB | Q4_K_M | llama.cpp CUDA | 642 | 30.7 | 2025-09-22 | llama.cpp #15013 |
| Tesla V100-SXM2-32GB | Q4_K_M | llama.cpp CUDA | 613 | 31.8 | 2025-09-22 | llama.cpp #15013 |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database