NVIDIA Nemotron 3.5 Bolt 74B-A9B
Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q8_0 unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF | 73.44 GiB | 3 | 74.5 GiB+ | 80 GB GPU | HF files |
| BF16 unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF | 138 GiB | 4 | 139 GiB+ | 192 GB GPU | HF files |
| MLX 4-bit mixed precision rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-4bit | 40.29 GiB | 9 | 41.3 GiB+ | 64 GB Mac | HF files |
| MLX 8-bit nativ-community/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-8bit | 74.72 GiB | 17 | 75.8 GiB+ | 128 GB Mac | HF files |
No verified file: Q4_K_M, Q5_K_M, FP16. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
21.4
Variant: Nemotron 3.5 Bolt 74B-A9B · observed 2026-10-17 · source
Rank #32 of 75 researched 2026 models.
Arena text rating
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 3
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| Surface Laptop Ultra,128GB,RTX Spark | 24GB heterogeneous IQ2_XXS/NVFP4/Q8_0 + LiLiCorr | llama.cpp | — | 84 | unknown | source |
| Apple M1 Max (24-core GPU), 64 GB unified memory | MLX oQ4e (oMLX custom 4-bit mix) | MLX | 242 | 27.2 | 2026-10-16 | omlx.ai |
| Apple M1 Max (24-core GPU), 64 GB unified memory | MLX oQ4e (oMLX custom 4-bit mix) | MLX | 232 | 23.9 | 2026-10-16 | omlx.ai |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database