NVIDIA Nemotron 3 Nano 4B

Released2026-03-16Size4B denseArchitecturehybrid linear attnLicensenvidia-nemotron-open-model-licenseAA rank#75 of 75

Files and memory at 8K context

Quant	File	Files	+ KV + runtime	Smallest fit	Source
Q4_K_M nvidia/NVIDIA-Nemotron-3-Nano-4B-GGUF	2.64 GiB	1	3.8 GiB+	8 GB GPU	HF files
Q5_K_M unsloth/NVIDIA-Nemotron-3-Nano-4B-GGUF	2.94 GiB	1	4.1 GiB+	8 GB GPU	HF files
Q8_0 unsloth/NVIDIA-Nemotron-3-Nano-4B-GGUF	3.94 GiB	1	5.1 GiB+	8 GB GPU	HF files
MLX 4-bit mlx-community/NVIDIA-Nemotron-3-Nano-4B-4bit	2.08 GiB	1	3.2 GiB+	8 GB Mac	HF files
MLX 8-bit mlx-community/NVIDIA-Nemotron-3-Nano-4B-8bit	3.93 GiB	1	5.1 GiB+	8 GB Mac	HF files

No verified file: FP16. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.

Quality

Artificial Analysis Intelligence Index

4.8

Variant: NVIDIA Nemotron 3 Nano 4B · observed 2026-10-17 · source

Rank #75 of 75 researched 2026 models.

Arena text rating

Unknown

Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.

Reported speeds 2

Hardware	Quant	Backend	Prompt t/s	Gen t/s	Date	Source
Apple M4 Max, 128 GB (GPU core count not stated)	Q4_K_M (nvidia, 2.84 GB)	llama.cpp Metal	1,309	88.4	2026-09-04	llama.cpp #20421
Apple M4 Max, 128 GB (GPU core count not stated)	MLX 4-bit (mlx-community, 2.24 GB)	MLX	—	177	2026-09-04	llama.cpp #20421

Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database

Community threads

Does the AIME benchmark report use a tool or not? Hugging Face · 2026-03-29
TemporalMesh Transformer: 29.4 PPL at 48% compute — beats Mamba, new open-source architecture Hugging Face · 2026-06-07

Qualitative signals only; titles are not verified claims.