Gemma 4 31B (it)
Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q4_K_M unsloth/gemma-4-31B-it-GGUF | 17.07 GiB | 1 | unknown | 20 GB GPU (cache unknown) | HF files |
| Q5_K_M unsloth/gemma-4-31B-it-GGUF | 20.17 GiB | 1 | unknown | 24 GB GPU (cache unknown) | HF files |
| Q8_0 unsloth/gemma-4-31B-it-GGUF | 30.39 GiB | 1 | unknown | 32 GB GPU (cache unknown) | HF files |
| MLX 4-bit mlx-community/gemma-4-31B-it-4bit | 17.15 GiB | 4 | unknown | 32 GB Mac (cache unknown) | HF files |
| MLX 8-bit mlx-community/gemma-4-31B-it-8bit | 31.44 GiB | 7 | unknown | 48 GB Mac (cache unknown) | HF files |
No verified file: FP16. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
14.7
Variant: Gemma 4 31B (Reasoning) · observed 2026-10-17 · source
Rank #42 of 75 researched 2026 models.
Arena text rating
1,443
95% CI 1435–1450 · 6,147 votes · published 2026-10-15 · dataset
Exact listed model name; hosted reasoning/quantization may differ from local checkpoint.
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 20
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| M2 Ultra (60c GPU) 64 GB | MLX 4bit | MLX | 175 | 24.8 | 2026-10-15 | omlx.ai |
| M4 Pro (20c GPU) 64 GB | MLX 4bit | MLX | 103 | 12.6 | 2026-10-15 | omlx.ai |
| M5 Max (32c GPU) 36 GB | MLX 4bit | MLX | 590 | 21.3 | 2026-10-13 | omlx.ai |
| M5 Max (40c GPU) 128 GB | MLX 4bit | MLX | 644 | 25 | 2026-10-13 | omlx.ai |
| M5 Pro (20c GPU) 64 GB | MLX 4bit | MLX | 342 | 14.1 | 2026-10-13 | omlx.ai |
| M2 Ultra (76c GPU) 128 GB | MLX 4bit | MLX | 186 | 19 | 2026-10-12 | omlx.ai |
| M4 Max (40c GPU) 128 GB | MLX 4bit | MLX | 188 | 23.1 | 2026-10-12 | omlx.ai |
| M4 Max (32c GPU) 36 GB | MLX 4bit | MLX | 162 | 17.7 | 2026-10-11 | omlx.ai |
| M5 Pro (16c GPU) 48 GB | MLX 4bit | MLX | 311 | 14.1 | 2026-10-10 | omlx.ai |
| M1 Max (24c GPU) 32 GB | MLX 4bit | MLX | 66.6 | 11.1 | 2026-10-06 | omlx.ai |
| M3 Max (40c GPU) 64 GB | MLX 4bit | MLX | 162 | 17.4 | 2026-10-05 | omlx.ai |
| M2 Max (38c GPU) 64 GB | MLX 4bit | MLX | 114 | 16.2 | 2026-09-23 | omlx.ai |
| NVIDIA GeForce RTX 3090 24 GB | Q4_K (subvariant not stated) | llama.cpp CUDA | 1,156 | 34.7 | unknown | hardware-corner.net |
| NVIDIA GeForce RTX 3090 24 GB | Q4_K (subvariant not stated) | llama.cpp CUDA | 724 | 31.4 | unknown | hardware-corner.net |
| NVIDIA GeForce RTX 5090 32 GB | Q4_K (subvariant not stated) | llama.cpp CUDA | 3,395 | 61.1 | unknown | hardware-corner.net |
| NVIDIA GeForce RTX 5090 32 GB | Q4_K (subvariant not stated) | llama.cpp CUDA | 1,459 | 51.6 | unknown | hardware-corner.net |
| NVIDIA GeForce RTX 5090 32 GB | Q4_K (subvariant not stated) | llama.cpp CUDA | 900 | 43.4 | unknown | hardware-corner.net |
| NVIDIA RTX PRO 6000 Blackwell 96 GB | Q4_K (subvariant not stated) | llama.cpp CUDA | 3,750 | 61.6 | unknown | hardware-corner.net |
| 2x AMD Radeon RX 7900 XTX 24 GB (Ryzen 7 5800X3D, PCIe 4.0 x8 each), --split-mode layer multi-device | Q8_0 | llama.cpp Vulkan | 852 | 24.4 | 2026-08-23 | llama.cpp #10879 |
| 2x AMD Radeon RX 7900 XTX 24 GB (Ryzen 7 5800X3D, PCIe 4.0 x8 each), --split-mode layer multi-device | Q8_0 | llama.cpp ROCm | 960 | 23.4 | 2026-08-23 | llama.cpp #15021 |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database
Community threads
- Same tool call infinite loop Hugging Face · 2026-07-30
- When Gemma doesn't have the right tool schema it hallucinates it, create outputs of the fake-calls and use them as memory (WIP) Hugging Face · 2026-07-23
Qualitative signals only; titles are not verified claims.