Inkling-Small
thinkingmachines/Inkling-Small
Files and memory at 8K context
| Quant | File | Files | + KV + runtime | Smallest fit | Source |
|---|---|---|---|---|---|
| Q8_0 unsloth/Inkling-Small-GGUF | 261 GiB | 7 | 262 GiB+ | over 192 GB GPU | HF files |
| BF16 unsloth/Inkling-Small-GGUF | 491 GiB | 11 | 492 GiB+ | over 192 GB GPU | HF files |
| MLX 4-bit mixed precision mlx-community/Inkling-Small-mlx-4bit | 143 GiB | 30 | 144 GiB+ | 256 GB Mac | HF files |
No verified file: Q4_K_M, Q5_K_M, FP16, MLX 8-bit. Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory.
Quality
Artificial Analysis Intelligence Index
25.7
Variant: Inkling Small · observed 2026-10-17 · source
Rank #21 of 75 researched 2026 models.
Arena text rating
1,413
95% CI 1408–1417 · 32,304 votes · published 2026-10-15 · dataset
Exact listed model name; hosted reasoning/quantization may differ from local checkpoint.
Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.
Reported speeds 2
| Hardware | Quant | Backend | Prompt t/s | Gen t/s | Date | Source |
|---|---|---|---|---|---|---|
| Apple M3 Ultra (60-core GPU), 256 GB unified memory | 4bit (Inkling-Small-oQ4e-mtp) | MLX (oMLX server) | 575 | 44.3 | 2026-08-27 | omlx.ai |
| Apple M3 Ultra (60-core GPU), 256 GB unified memory | 4bit (Inkling-Small-oQ4e-mtp) | MLX (oMLX server) | 544 | 39.5 | 2026-08-27 | omlx.ai |
Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database