Files and memory at 8K context
No verified quantized file for the requested formats.
Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory. Legacy model: sizes on device pages are estimated from parameter count.
Quality
Artificial Analysis Intelligence Index
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 75
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|
| ASUS NVIDIA GeForce RTX 5060 Ti 16GB | Q8_0 | llama.cpp Vulkan | 3,666 | 50.7 | 2026-10-05 | openbenchmarking.org |
| ASUS NVIDIA GeForce RTX 5060 Ti 16GB | Q8_0 | llama.cpp Vulkan | 3,570 | — | 2026-10-05 | openbenchmarking.org |
| ASUS NVIDIA GeForce RTX 5060 Ti 16GB | Q8_0 | llama.cpp CUDA | 3,925 | 50.5 | 2026-10-05 | openbenchmarking.org |
| ASUS NVIDIA GeForce RTX 5060 Ti 16GB | Q8_0 | llama.cpp CUDA | 3,837 | — | 2026-10-05 | openbenchmarking.org |
| Sapphire AMD Radeon AI PRO R9700 | Q8_0 | llama.cpp Vulkan | 4,408 | 71.4 | 2026-05-13 | openbenchmarking.org |
| Sapphire AMD Radeon AI PRO R9700 | Q8_0 | llama.cpp Vulkan | 4,246 | — | 2026-05-13 | openbenchmarking.org |
| AMD Radeon PRO W7900 | Q8_0 | llama.cpp Vulkan | 2,701 | 75.7 | 2026-05-11 | openbenchmarking.org |
| AMD Radeon PRO W7900 | Q8_0 | llama.cpp Vulkan | 2,554 | — | 2026-05-11 | openbenchmarking.org |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | Q8_0 | llama.cpp CUDA | 16,645 | 170 | 2026-05-03 | openbenchmarking.org |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | Q8_0 | llama.cpp CUDA | 16,231 | — | 2026-05-03 | openbenchmarking.org |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | Q8_0 | llama.cpp Vulkan | 14,168 | 164 | 2026-05-03 | openbenchmarking.org |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | Q8_0 | llama.cpp Vulkan | 13,911 | — | 2026-05-03 | openbenchmarking.org |
| NVIDIA RTX 6000 Ada Generation 48GB | Q8_0 | llama.cpp CUDA | 8,149 | 99.2 | 2026-05-02 | openbenchmarking.org |
| NVIDIA RTX 6000 Ada Generation 48GB | Q8_0 | llama.cpp CUDA | 7,754 | — | 2026-05-02 | openbenchmarking.org |
| NVIDIA RTX 6000 Ada Generation 48GB | Q8_0 | llama.cpp Vulkan | 6,606 | 98.1 | 2026-05-02 | openbenchmarking.org |
| NVIDIA RTX 6000 Ada Generation 48GB | Q8_0 | llama.cpp Vulkan | 6,346 | — | 2026-05-02 | openbenchmarking.org |
| Gigabyte NVIDIA GeForce RTX 5090 32GB | Q8_0 | llama.cpp CUDA | 14,799 | 169 | 2025-11-02 | openbenchmarking.org |
| Gigabyte NVIDIA GeForce RTX 5090 32GB | Q8_0 | llama.cpp CUDA | 14,576 | — | 2025-11-02 | openbenchmarking.org |
| Zotac NVIDIA GeForce RTX 5080 16GB | Q8_0 | llama.cpp CUDA | — | 100 | 2025-10-27 | openbenchmarking.org |
| AMD Radeon RX 9070 XT | Q8_0 | llama.cpp Vulkan | 2,977 | 68.8 | 2025-09-06 | openbenchmarking.org |
| AMD Radeon RX 9070 XT | Q8_0 | llama.cpp Vulkan | 2,729 | — | 2025-09-06 | openbenchmarking.org |
| Intel(R) Arc(TM) B390 GPU | Q4_0 | llama.cpp Vulkan | 1,292 | — | 2026-09-12 | llama.cpp #10879 |
| Intel(R) Arc(TM) B390 GPU | Q4_0 | llama.cpp Vulkan | 1,172 | — | 2026-09-12 | llama.cpp #10879 |
| Intel(R) Arc(TM) B390 GPU | Q4_0 | llama.cpp Vulkan | 893 | — | 2026-09-12 | llama.cpp #10879 |
| Intel(R) Arc(TM) B390 GPU | Q4_0 | llama.cpp Vulkan | 734 | — | 2026-09-12 | llama.cpp #10879 |
Reported by the linked sources, not reproduced by LocalFit. Showing 25 of 75. All rows in the speed database