Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q4_K_M AngelSlim/Hy3-GGUF | 170 GiB | 1 | 173 GiB+ | 192 GB GPU | HF files |
No verified file: Q5_K_M, Q8_0, FP16, BF16, MLX 4-bit, MLX 8-bit. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
25.3
Variant: Hy3 · observed 2026-10-17 · source
Rank #23 of 75 researched 2026 models.
Arena text rating
1,438
95% CI 1432–1444 · 12,208 votes · published 2026-10-15 · dataset
Exact listed model name; hosted reasoning/quantization may differ from local checkpoint.
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 3
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| Apple M5 Max (40-core GPU), 128 GB unified memory | 2bit (Hy3-oQ2) | MLX (oMLX server) | 569 | 26.3 | 2026-08-17 | omlx.ai |
| Apple M5 Max (40-core GPU), 128 GB unified memory | 2bit (Hy3-oQ2) | MLX (oMLX server) | 530 | 28.5 | 2026-08-17 | omlx.ai |
| Apple M5 Max (40-core GPU), 128 GB unified memory | 2bit (Hy3-oQ2) | MLX (oMLX server) | 273 | 21.3 | 2026-08-06 | omlx.ai |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database