Olmo-3.1-32B-Think Specialist
Where it runs 8K context · ≥ 5 t/s floor
| Setup | Best plan | Memory | Generate |
|---|---|---|---|
| 8 GB GPU + 32 GB RAM | Q4_K_Mllama.cpp CUDAGPU + CPU layers · 21/64 layers on GPU | Fits | ~1.4–1.7 t/s Estimated extrapolated |
| 12 GB GPU + 32 GB RAM | Q4_K_Mllama.cpp CUDAGPU + CPU layers · 34/64 layers on GPU | Fits | ~1.2–1.4 t/s Estimated extrapolated |
| 16 GB GPU + 64 GB RAM | Q4_K_Mllama.cpp CUDAGPU + CPU layers · 47/64 layers on GPU | Fits | ~3.4–4.0 t/s Estimated extrapolated |
| 24 GB GPU + 64 GB RAM | Q4_K_Mllama.cpp CUDAFull GPU | Fits | ~28–35 t/s Estimated |
| 32 GB GPU + 128 GB RAM | Q5_K_Mllama.cpp CUDAFull GPU | Fits | ~38–60 t/s Estimated |
| 2× 24 GB GPUs + 128 GB RAM | Q8_0llama.cpp CUDALayer split across GPUs | Fits | ~15–19 t/s Estimated extrapolated |
| 96 GB workstation GPU + 256 GB RAM | Q8_0llama.cpp CUDAFull GPU | Fits | ~26–41 t/s Estimated |
| Mac, 24 GB | Does not fit | ||
| Mac, 48 GB | MLX 8-bitMLX-LMUnified memory | Fits | ~6.6–6.8 t/s Estimated |
| Mac, 128 GB | MLX 8-bitMLX-LMUnified memory | Fits | ~12–14 t/s Estimated |
| Mac, 512 GB | MLX 8-bitMLX-LMUnified memory | Fits | ~14–21 t/s Estimated |
| DGX Spark 128 GB | Q8_0llama.cpp CUDAUnified memory | Fits | ~5.9–6.5 t/s Estimated |
| Ryzen AI Max+ 395, 128 GB | Q5_K_Mllama.cpp VulkanUnified memory | Fits | ~5.8–8.5 t/s Estimated |
| CPU only, 64 GB DDR5-5600 dual channel | Q4_K_Mllama.cpp CPUCPU only | Fits | ~1.0–1.2 t/s Estimated extrapolated |
Same plan engine as the planner, with its default reserves and RAM. How plans are chosen.
Files exact Hugging Face bytes
| Artifact | Size | Files | Revision | Source |
|---|---|---|---|---|
| Q8_0 bartowski/allenai_Olmo-3.1-32B-Think-GGUF | 31.90 GiB | 1 | 7258621a8a | HF files |
| Q5_K_M bartowski/allenai_Olmo-3.1-32B-Think-GGUF | 21.29 GiB | 1 | 7258621a8a | HF files |
| Q4_K_M bartowski/allenai_Olmo-3.1-32B-Think-GGUF | 18.14 GiB | 1 | 7258621a8a | HF files |
| MLX 8-bit 8-bit mlx-community/Olmo-3.1-32B-Think-8bit | 31.90 GiB | 7 | 306e81d6b0 | HF files |
| MLX 4-bit 4-bit mlx-community/Olmo-3.1-32B-Think-4bit | 16.89 GiB | 4 | aec94ecef2 | HF files |
| Native safetensors BF16 allenai/Olmo-3.1-32B-Think | 60.04 GiB | 14 | 97ade57a9c | HF files |
Not found in the representative quant search (not proven unavailable): GGUF FP16, EXL3, AWQ, GPTQ. Complete selected file sets; projectors, MTP/draft and lookup files are listed separately and budgeted only by plans that use them. File bytes are not allocated memory.
Quality
| AA evaluated mode | Index | LiveCodeBench | Terminal-Bench 2.1 | GPQA | HLE | Source |
|---|---|---|---|---|---|---|
| Olmo 3.1 32B Think AA estimate | 7.1 | 0.0% | — | 59.1% | 6.3% | source |
Arena text rating
1,272
CI 1264–1279 · 8,301 votes · published 2026-10-15 · dataset
Explicit name identity; exact hosted revision/local quant equivalence not verified
Publisher-reported benchmarks
None extracted from the model card.
AA and Arena describe the hosted model in the stated mode, not each quantized file; scores are never inherited by base, pruned, abliterated or quantized variants and never averaged together.
How to run it
No model-specific recipe in the research. The planner binds the backend’s documented command template (llama.cpp, Ollama, LM Studio, MLX, vLLM) to your plan: open a plan’s details.
Recipes are source examples (settings are the publisher’s or author’s, not measured optima) and were not executed by LocalFit. A downloadable file does not prove your backend build supports the architecture.
Reported speeds 0
No reported measurement yet. Submit one.