Qwen3.5-4B
Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q4_K_M unsloth/Qwen3.5-4B-GGUF | 2.55 GiB | 1 | 3.8 GiB+ | 8 GB GPU | HF files |
| Q5_K_M unsloth/Qwen3.5-4B-GGUF | 2.93 GiB | 1 | 4.2 GiB+ | 8 GB GPU | HF files |
| Q8_0 unsloth/Qwen3.5-4B-GGUF | 4.17 GiB | 1 | 5.4 GiB+ | 8 GB GPU | HF files |
| MLX 4-bit mlx-community/Qwen3.5-4B-4bit | 2.83 GiB | 1 | 4.1 GiB+ | 8 GB Mac | HF files |
| MLX 8-bit mlx-community/Qwen3.5-4B-8bit | 4.78 GiB | 1 | 6.0 GiB+ | 16 GB Mac | HF files |
No verified file: FP16. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
13.1 AA estimate
Variant: Qwen3.5 4B (Reasoning) · observed 2026-10-17 · source
Rank #48 of 75 researched 2026 models.
Arena text rating
Unknown
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 6
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| MacBook Air M5, 32 GB (10 CPU / 10 GPU cores) | Q4_K_M | llama.cpp Metal | 631 | 29.4 | 2026-04-06 | r/LocalLLaMA |
| Steam Deck (AMD Custom GPU 0932 / RADV VANGOGH, Vulkan) | Q4_K_M | llama.cpp Vulkan | 189 | 22.6 | 2026-08-20 | llama.cpp #10879 |
| Steam Deck (AMD Custom GPU 0932 / RADV VANGOGH, Vulkan) | Q4_K_M | llama.cpp Vulkan | 190 | 22.7 | 2026-08-20 | llama.cpp #10879 |
| NVIDIA GeForce RTX 3090 24 GB | Q4_K_XL (unsloth UD) | llama.cpp Vulkan | 5,234 | 163 | 2026-04-23 | r/LocalLLaMA |
| Intel Arc Pro B70 32 GB | Q4_K_XL (unsloth UD) | llama.cpp Vulkan | 1,431 | 60.5 | 2026-04-23 | r/LocalLLaMA |
| Intel Arc Pro B70 32 GB | Q4_K_XL (unsloth UD) | llama.cpp SYCL | 1,767 | 79.1 | 2026-04-23 | r/LocalLLaMA |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database
Community threads
- openai.APIConnectionError: Connection error. Hugging Face · 2026-03-06
- Request: DOI Hugging Face · 2026-10-14
Qualitative signals only; titles are not verified claims.