Gemma 4 12B (it)
Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q4_K_M bartowski/gemma-4-12B-it-GGUF | 7.14 GiB | 1 | unknown | 12 GB GPU (cache unknown) | HF files |
| Q5_K_M bartowski/gemma-4-12B-it-GGUF | 8.17 GiB | 1 | unknown | 12 GB GPU (cache unknown) | HF files |
| Q8_0 bartowski/gemma-4-12B-it-GGUF | 11.80 GiB | 1 | unknown | 16 GB GPU (cache unknown) | HF files |
| MLX 4-bit mlx-community/gemma-4-12B-it-4bit | 6.28 GiB | 2 | unknown | 16 GB Mac (cache unknown) | HF files |
| MLX 8-bit mlx-community/gemma-4-12B-it-8bit | 11.84 GiB | 3 | unknown | 18 GB Mac (cache unknown) | HF files |
No verified file: FP16. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
14.2 AA estimate
Variant: Gemma 4 12B (Reasoning) · observed 2026-10-17 · source
Rank #44 of 75 researched 2026 models.
Arena text rating
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 4
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| Steam Deck (AMD Custom GPU 0932 / RADV VANGOGH, Vulkan) | QAT UD-Q4_K_XL (llama-bench reports Q4_0) | llama.cpp Vulkan | 84.7 | 9.1 | 2026-08-20 | llama.cpp #10879 |
| Steam Deck (AMD Custom GPU 0932 / RADV VANGOGH, Vulkan) | QAT UD-Q4_K_XL (llama-bench reports Q4_0) | llama.cpp Vulkan | 90.8 | 9.2 | 2026-08-20 | llama.cpp #10879 |
| NVIDIA GeForce RTX 4070 SUPER 12 GB (Ryzen 7 9700X, DDR5-6000) | QAT UD-Q4_K_XL (draft: MTP Q8_0) | llama.cpp CUDA | — | 131 | 2026-06-06 | r/LocalLLaMA |
| NVIDIA GeForce RTX 4070 SUPER 12 GB (Ryzen 7 9700X, DDR5-6000) | QAT UD-Q4_K_XL (draft: MTP Q8_0) | llama.cpp CUDA | — | 94 | 2026-06-06 | r/LocalLLaMA |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database
Community threads
- Bug Report: MAJOR issues with parallel generation and left-padding (different response from normal generatiln) Hugging Face · 2026-07-30
- Chat template may re-inject prior-turn reasoning during multi-turn tool use → repetition loops Hugging Face · 2026-06-10
- https://www.reddit.com/r/LocalLLaMA/comments/1x7g8cc/why_do_we_have_models_under_15b_so_rarely_anymore/ Reddit · 2026-10-16
- https://www.reddit.com/r/LocalLLM/comments/1tvx2h7/google_introduces_gemma_4_12b_a_unified/ Reddit · 2026-06-03
Qualitative signals only; titles are not verified claims.