Files and memory at 8K context
No verified quantized file for the requested formats.
Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory. Legacy model: sizes on device pages are estimated from parameter count.
Quality
Artificial Analysis Intelligence Index
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 320
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|
| ASUS NVIDIA GeForce RTX 5060 Ti 16GB | Q8_0 | llama.cpp Vulkan | 3,573 | 127 | 2026-10-05 | openbenchmarking.org |
| ASUS NVIDIA GeForce RTX 5060 Ti 16GB | Q8_0 | llama.cpp Vulkan | 3,601 | — | 2026-10-05 | openbenchmarking.org |
| ASUS NVIDIA GeForce RTX 5060 Ti 16GB | Q8_0 | llama.cpp CUDA | 5,803 | 140 | 2026-10-05 | openbenchmarking.org |
| ASUS NVIDIA GeForce RTX 5060 Ti 16GB | Q8_0 | llama.cpp CUDA | 5,843 | — | 2026-10-05 | openbenchmarking.org |
| Sapphire AMD Radeon AI PRO R9700 | Q8_0 | llama.cpp Vulkan | 4,511 | 181 | 2026-05-13 | openbenchmarking.org |
| Sapphire AMD Radeon AI PRO R9700 | Q8_0 | llama.cpp Vulkan | 4,485 | — | 2026-05-13 | openbenchmarking.org |
| AMD Radeon PRO W7900 | Q8_0 | llama.cpp Vulkan | 3,364 | 179 | 2026-05-11 | openbenchmarking.org |
| AMD Radeon PRO W7900 | Q8_0 | llama.cpp Vulkan | 3,307 | — | 2026-05-11 | openbenchmarking.org |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | Q8_0 | llama.cpp CUDA | 23,598 | 395 | 2026-05-03 | openbenchmarking.org |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | Q8_0 | llama.cpp CUDA | 24,078 | — | 2026-05-03 | openbenchmarking.org |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | Q8_0 | llama.cpp Vulkan | 12,525 | 390 | 2026-05-03 | openbenchmarking.org |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | Q8_0 | llama.cpp Vulkan | 12,702 | — | 2026-05-03 | openbenchmarking.org |
| NVIDIA RTX 6000 Ada Generation 48GB | Q8_0 | llama.cpp CUDA | 9,015 | 252 | 2026-05-02 | openbenchmarking.org |
| NVIDIA RTX 6000 Ada Generation 48GB | Q8_0 | llama.cpp CUDA | 8,906 | — | 2026-05-02 | openbenchmarking.org |
| NVIDIA RTX 6000 Ada Generation 48GB | Q8_0 | llama.cpp Vulkan | 7,165 | 253 | 2026-05-02 | openbenchmarking.org |
| NVIDIA RTX 6000 Ada Generation 48GB | Q8_0 | llama.cpp Vulkan | 7,024 | — | 2026-05-02 | openbenchmarking.org |
| Mac Studio M1 Ultra (64 core GPU, 128GB) | Q4_K_M | llama.cpp Metal | 1,741 | 100 | 2026-02-07 | ai-benchmarks #48 |
| Mac Studio M1 Ultra (64 core GPU, 128GB) | Q4_K_M | llama.cpp Metal | 1,595 | — | 2026-02-07 | ai-benchmarks #48 |
| NVIDIA GeForce RTX 4070 Ti (Intel Core Ultra 265K custom PC) | Q4_K_M | llama.cpp Vulkan | 4,421 | 163 | 2025-12-10 | ai-benchmarks #45 |
| NVIDIA GeForce RTX 4090 (Intel Core Ultra 265K custom PC) | Q4_K_M | llama.cpp Vulkan | 7,216 | 280 | 2025-12-09 | ai-benchmarks #43 |
| NVIDIA GeForce RTX 4090 (Intel Core Ultra 265K custom PC) | Q4_K_M | llama.cpp Vulkan | 5,908 | — | 2025-12-09 | ai-benchmarks #43 |
| NVIDIA GeForce RTX 3060 12GB (Intel Core Ultra 265K custom PC) | Q4_K_M | llama.cpp Vulkan | 1,589 | 84 | 2025-12-06 | ai-benchmarks #40 |
| AMD Radeon AI Pro R9700 32GB (Intel Core Ultra 7 265K system) | Q4_K_M | llama.cpp Vulkan | 2,969 | 163 | 2025-11-25 | ai-benchmarks #35 |
| AMD Radeon AI Pro R9700 32GB (Intel Core Ultra 7 265K system) | Q4_K_M | llama.cpp Vulkan | 2,613 | — | 2025-11-25 | ai-benchmarks #35 |
| NVIDIA DGX Spark (bugparty's comment) | F16 | llama.cpp CUDA | 2,599 | 60.6 | 2025-11-22 | ai-benchmarks #21 |
Reported by the linked sources, not reproduced by LocalFit. Showing 25 of 320. All rows in the speed database