Apple M6 Mac mini12CPU/12GPU 24GB: what runs best?

RTX 4090 RTX 5090 RTX 3090 RTX 4060 Ti (16 GB)RTX 5070 Ti Radeon RX 7900 XTX M4 Max 128 GB M3 Ultra 512 GB DGX Spark (128 GB)All devices

Apple · Apple Silicon Mac

Apple M6 Mac mini12CPU/12GPU 24GB

Memory24 GB unifiedBandwidth170 GB/s vendor specPlanned for models75% of unified memory (editable)

GPUs

Custom GPU

Unified memory

Chip / systemGPU coresMemory

Advanced: reserves, bandwidth, workload, models

Available VRAM per GPUGiB Runtime allowance per deviceGiB Spare margin per deviceGiB Unified memory for models% RAM kept for OS and appsGiB Measured RAM bandwidthGB/s GPU linkLanes Prompt length (first token)tokens Custom contexttokens Parallel sequences

Allow disk-streamed experts (documented cases)Include base checkpointsInclude specialistsInclude community & quant-trained variantsInclude models with unverified release dates

Defaults are editorial planning choices, not measurements: 1 GiB runtime + 0.5 GiB margin per device, 75% of unified memory, 8 GiB RAM for the OS, a vendor “24 GB” treated as 24 GiB (enter your driver’s available VRAM to override), 5 t/s minimum speed (research editorial default). Memory ledger

Apple M6 Mac mini12CPU/12GPU 24GB · 75% (18.0 GiB) planned for models · 170 GB/s8K contextF16 KVany backend≥ 5 t/s

Best plans Apple M6 Mac mini12CPU/12GPU 24GB

Best quality at ≥ 5 t/s

Qwen3.8-27B

33.7 AA index Xhigh

MLX 4-bitMLX-LMUnified memory

~6.4–9.3 t/s Estimated

Faster tradeoff

K2-Horizon-7B

20.6 AA index single mode

Q8_0llama.cpp MetalUnified memory

~9.3–15 t/s Estimated

Next point on the quality/speed front; then K2-Horizon-3.7B.

Possibly better · speed not established

No higher-scoring model has a feasible plan here.

WeightsKV cacheReserveFreeFits* = a state term unsizedMeasuredlinks to its sourceEstimatedcalibrated or extrapolated range, ~Ceilingupper bound, ≤Unknownnever zero

Meets your 5 t/s floor 25

Best plan per model that reaches your floor by a matching measurement, a calibrated range, or an extrapolated estimate (labelled). Sorted by the selected quality metric, then speed.

1
Qwen3.8-27B
Qwen/Qwen3.8-27B
2026-08-1427B dense
AA index33.7Xhigh
Plan & memory
MLX 4-bitMLX-LMUnified memoryFits*
Unified
17.0 / 18.0 GiB
Generate~6.4–9.3 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
2
K2-Horizon-7B
IFM/K2-Horizon-7B
2026-09-039B tensors · dense
AA index20.6single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
11.5 / 18.0 GiB
Generate~9.3–15 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
3
Muse-Glimmer-30B
meta-models/Muse-Glimmer-30B
2026-08-1030B dense
AA index17.5High
Plan & memory
Q4_K_Mllama.cpp MetalUnified memoryFits
Unified
17.3 / 18.0 GiB
Generate~5.6–8.7 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
4
K2-Horizon-3.7B
IFM/K2-Horizon-3.7B
2026-09-035.1B tensors · dense
AA index15.6single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
7.6 / 18.0 GiB
Generate~16–25 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
5
Qwen3.5-4B
Qwen/Qwen3.5-4B
2026-03-024B dense
AA index13.1 est.Reasoning
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits*
Unified
6.1 / 18.0 GiB
Generate~20–31 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
6
MiniCPM5-2B
openbmb/MiniCPM5-2B
2026-09-072.5B dense
AA index12.5single mode
Plan & memory
FP16llama.cpp MetalUnified memoryFits
Unified
6.5 / 18.0 GiB
Generate~18–28 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
7
Qwen3.5-9B
Qwen/Qwen3.5-9B
2026-03-029B dense
AA index11.2Reasoning
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits*
Unified
10.9 / 18.0 GiB
Generate~9.5–15 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
8
granite-4.2-8b
ibm-granite/granite-4.2-8b
2026-08-258B dense
AA index11.1single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
11.5 / 18.0 GiB
Generate~9.5–15 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
9
G9v3-3B
ai9stars/G9v3-3B
2026-07 (day unknown)3B dense
AA index10.8 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryFits
Unified
7.5 / 18.0 GiB
Generate~15–24 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
10
granite-4.2-3b
ibm-granite/granite-4.2-3b
2026-08-253B dense
AA index9.1single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
5.8 / 18.0 GiB
Generate~22–35 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
11
Tri-21B-Think
trillionlabs/Tri-21B-Think
2026-02 (day unknown)21B dense
AA index9.0 est.single mode
Plan & memory
Q5_K_Mllama.cpp MetalUnified memoryFits
Unified
16.8 / 18.0 GiB
Generate~6.1–9.5 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
12
LFM2.5-2.6B
LiquidAI/LFM2.5-2.6B
2026-08-042.7B dense
AA index8.4 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryFits*
Unified
6.7 / 18.0 GiB
Generate~17–27 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
13
LFM2.5-8B-A1B
LiquidAI/LFM2.5-8B-A1B
2026-05-288.3B MoE · 1.5B active
AA index7.2 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryFits*
Unified
17.4 / 18.0 GiB
Generate~10–30 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
14
Qwen3.5-2B
Qwen/Qwen3.5-2B
2026-03-022B dense
AA index6.9 est.Reasoning
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits*
Unified
3.5 / 18.0 GiB
Generate~44–69 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
15
Qwen3.5-0.8B
Qwen/Qwen3.5-0.8B
2026-03-020.8B dense
AA index6.1 est.Reasoning
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits*
Unified
2.4 / 18.0 GiB
Generate~107–167 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
16
Phi-4Older
microsoft/phi-4
2024-12-1215B tensors · dense
AA index5.9 est.single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
17.6 / 18.0 GiB
Generate~5.8–9.0 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
17
Olmo-3-7B-ThinkSpecialist
allenai/Olmo-3-7B-Think
2025-11-207.3B tensors · dense
AA index5.6 est.single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
11.2 / 18.0 GiB
Generate~9.6–15 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
18
Olmo-3-7B-Instruct
allenai/Olmo-3-7B-Instruct
2025-11-207.3B tensors · dense
AA index5.2 est.single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
11.2 / 18.0 GiB
Generate~9.6–15 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
19
LFM2.5-1.2B-ThinkingSpecialist
LiquidAI/LFM2.5-1.2B-Thinking
2026-01-201.2B tensors · dense
AA index5.2 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryFits*
Unified
3.8 / 18.0 GiB
Generate~40–62 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
20
LFM2.5-1.2B-Instruct
LiquidAI/LFM2.5-1.2B-Instruct
2026-01-051.2B tensors · dense
AA index5.2 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryFits*
Unified
3.8 / 18.0 GiB
Generate~40–62 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
21
Molmo2-8BSpecialist
allenai/Molmo2-8B
2025-12-118.7B tensors · dense
AA index5.0 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryFits
Unified
17.9 / 18.0 GiB
Generate~5.6–8.7 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
22
LFM2.5-VL-1.6BSpecialist
LiquidAI/LFM2.5-VL-1.6B
2026-01-051.6B tensors · dense
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryFits*
Unified
3.8 / 18.0 GiB
Generate~40–62 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
23
Apertus-8B-Instruct-2509
swiss-ai/Apertus-8B-Instruct-2509
2025-09-028.1B tensors · dense
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryFits
Unified
17.5 / 18.0 GiB
Generate~5.7–8.9 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
24
NVIDIA-Nemotron-3-Nano-4B-BF16
nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
2026-03-164B dense
AA index4.8single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits*
Unified
5.6 / 18.0 GiB
Generate~22–34 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
25
K2-Horizon-0.9B
IFM/K2-Horizon-0.9B
2026-09-031.1B tensors · dense
AA index3.0single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
3.0 / 18.0 GiB
Generate~68–107 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown

+ 43 models without a AA index score

LFM2.5-230MFP16 · unified memory · ~254 t/s
LFM2.5-350MFP16 · unified memory · ~171 t/s
LittleLambFP16 · unified memory · ~122 t/s
LittleLamb-ToolCallingFP16 · unified memory · ~122 t/s
Jais-3-0.9BQ8_0 · unified memory · ~94 t/s
Qwen3-0.6BFP16 · unified memory · ~66 t/s
LFM2.5-1.2B-JPFP16 · unified memory · ~54 t/s
LFM2.5-24B-A2BQ5_K_M · unified memory · ~44 t/s
granite-4.0-h-tinyQ8_0 · unified memory · ~41 t/s
granite-4.1-3bQ8_0 · unified memory · ~33 t/s
Qwen3-1.7BFP16 · unified memory · ~29 t/s
Agents-A1-4BQ8_0 · unified memory · ~28 t/s
FrogNano-4B-2609Q8_0 · unified memory · ~27 t/s
Qwen3-4BQ8_0 · unified memory · ~27 t/s
AgentCPM-ReportQ4_K_M · unified memory · ~26 t/s
Mellum2.1-12B-A2.5B-ThinkingQ8_0 · unified memory · ~23 t/s
Jais-3-3.7BQ8_0 · unified memory · ~22 t/s
PaCoRe-8BQ8_0 · unified memory · ~14 t/s
Qwen3-8BQ8_0 · unified memory · ~14 t/s
AgentCPM-ExploreFP16 · unified memory · ~14 t/s
NVIDIA-Nemotron-Nano-9B-v2-JapaneseQ8_0 · unified memory · ~14 t/s
MiMo-V2.6-Distill-Qwen-9BQ8_0 · unified memory · ~13 t/s
Ornith-1.5-9BQ8_0 · unified memory · ~13 t/s
granite-4.1-8bQ8_0 · unified memory · ~13 t/s
Jais-3-7BQ8_0 · unified memory · ~13 t/s
Mistral NeMo 12B Instruct (2407)Q8_0 · unified memory · ~9.5 t/s
Mistral Small 3.2 24B Instruct (2506)Q4_K_M · unified memory · ~8.7 t/s
Mistral 7B Instruct v0.3FP16 · unified memory · ~8.7 t/s
IQuest-Coder-V1-7B-Base-Stage1FP16 · unified memory · ~8.4 t/s
IQuest-Coder-V1-7B-BaseFP16 · unified memory · ~8.4 t/s
IQuest-Coder-V1-7B-InstructFP16 · unified memory · ~8.4 t/s
IQuest-Coder-V1-7B-ThinkingFP16 · unified memory · ~8.4 t/s
Qwen3.6-27BMLX 4-bit · unified memory · ~8.3 t/s
IQuest-Coder-V1-14B-Base-Stage1Q8_0 · unified memory · ~8.2 t/s
IQuest-Coder-V1-14B-BaseQ8_0 · unified memory · ~8.2 t/s
IQuest-Coder-V1-14B-InstructQ8_0 · unified memory · ~8.2 t/s
IQuest-Coder-V1-14B-ThinkingQ8_0 · unified memory · ~8.2 t/s
Qwen3-14BQ8_0 · unified memory · ~7.9 t/s
MagenticBrainQ8_0 · unified memory · ~7.9 t/s
FrogMini-14B-2510Q8_0 · unified memory · ~7.9 t/s
DeepSeek-R1-Distill-Llama-8BFP16 · unified memory · ~7.8 t/s
RLVR-8B-0926FP16 · unified memory · ~7.7 t/s
Nemotron-Cascade-8BFP16 · unified memory · ~7.7 t/s

Memory not fully resolved 8

The weights fit, but the KV cache for this attention type (or the chosen cache layout) is not established, so a fit cannot be guaranteed.

1
gemma-4-26B-A4B-it
google/gemma-4-26B-A4B-it
2026-04-0225B MoE · 3.8B active
AA index16.7 est.Reasoning
Plan & memory
MLX 4-bitMLX-LMUnified memoryKV unknown
Unified
15.8+ / 18.0 GiB
Generate~14–39 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
2
gemma-4-12B-it
google/gemma-4-12B-it
2026-06-0312B dense
AA index14.2 est.Reasoning
Plan & memory
Q8_0llama.cpp MetalUnified memoryKV unknown
Unified
13.3+ / 18.0 GiB
Generate~7.5–12 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
3
Ling-3.0-tiny
inclusionAI/Ling-3.0-tiny
2026-08 (day unknown)7.9B MoE · 1.3B active
AA index11.1single mode
Plan & memory
FP16llama.cpp MetalUnified memoryKV unknown
Unified
16.2+ / 18.0 GiB
Generate~12–36 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
4
diffusiongemma-26B-A4B-it
google/diffusiongemma-26B-A4B-it
2026-06-1025B MoE · 3.8B active
AA index9.5 est.single mode
Plan & memory
Q4_K_Mllama.cpp MetalUnified memoryKV unknown
Unified
17.2+ / 18.0 GiB
Generate~13–37 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
5
gemma-4-E4B-it
google/gemma-4-E4B-it
2026-04-028B dense
AA index8.9Reasoning
Plan & memory
Q8_0llama.cpp MetalUnified memoryKV unknown
Unified
9.0+ / 18.0 GiB
Generate~12–18 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
6
gemma-4-E2B-it
google/gemma-4-E2B-it
2026-04-025.1B dense
AA index7.8 est.Reasoning
Plan & memory
Q8_0llama.cpp MetalUnified memoryKV unknown
Unified
6.1+ / 18.0 GiB
Generate~19–30 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
7
Phi-4-mini-instructOlder
microsoft/Phi-4-mini-instruct
2025-02-263.8B tensors · dense
AA index6.3 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryKV unknown
Unified
8.7+ / 18.0 GiB
Generate~12–19 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
8
tiny-aya-globalGated
CohereLabs/tiny-aya-global
2026-02-173.3B tensors
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp MetalUnified memoryKV unknown
Unified
7.7+ / 18.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown

+ 20 models without a AA index score

DeepSeek-R1-Distill-Qwen-1.5BQ8_0 · unified memory · ~69 t/s
NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16Q5_K_M · unified memory · ~35 t/s
Youtu-Parsing-OmniMLX 8-bit · unified memory · ~22 t/s
DeepSeek-R1-Distill-Qwen-7BFP16 · unified memory · ~8.5 t/s
Qwen2.5-7B-InstructFP16 · unified memory · ~8.5 t/s
DeepSeek-R1-Distill-Qwen-14BQ8_0 · unified memory · ~8.3 t/s
Qwen2.5-14B-InstructQ8_0 · unified memory · ~8.3 t/s
Qwen2.5-Coder-7B-InstructQ8_0 · unified memory · ~8.0 t/s
Apertus-v1.5-8BMLX 8-bit · unified memory
Gemma 3 12B ITQ8_0 · unified memory
Gemma 3 1B ITQ8_0 · unified memory
Gemma 3 27B ITQ4_K_M · unified memory
Gemma 3 4B ITFP16 · unified memory
Llama 3.1 8B InstructQ8_0 · unified memory
Llama 3.2 1B InstructFP16 · unified memory
Llama 3.2 3B InstructFP16 · unified memory
Nemotron-Elastic-12BQ8_0 · unified memory
tiny-aya-earthFP16 · unified memory
tiny-aya-fireFP16 · unified memory
tiny-aya-waterFP16 · unified memory

Speed unknown 5

No measurement, calibration or bandwidth figure for this plan.

1
gpt-oss-20bOlder
openai/gpt-oss-20b
2025-08-0521B tensors · MoE
AA index9.0High
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
13.0 / 18.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
2
granite-4.0-h-1b
ibm-granite/granite-4.0-h-1b
2025-10-281.5B tensors · MoE
AA index5.2 est.single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits*
Unified
3.0 / 18.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
3
granite-4.0-micro
ibm-granite/granite-4.0-micro
2025-10-023.4B tensors · MoE
AA index5.1 est.single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
5.5 / 18.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
4
granite-4.0-350m
ibm-granite/granite-4.0-350m
2025-10-280.4B tensors · MoE
AA index4.8 est.single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits
Unified
2.1 / 18.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
5
granite-4.0-h-350m
ibm-granite/granite-4.0-h-350m
2025-10-280.3B tensors · MoE
AA index4.8 est.single mode
Plan & memory
Q8_0llama.cpp MetalUnified memoryFits*
Unified
1.9 / 18.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown

+ 2 models without a AA index score

granite-4.0-1bQ8_0 · unified memory
granite-4.0-h-microQ8_0 · unified memory

134 models with no feasible plan on this setup

MiMo-V2.6-Pro-MOPDno backend/format combination
GLM-5.3Q8_0 746 GiB exceeds memory
Step-5-Previewno backend/format combination
Kimi-K3no backend/format combination
GLM-5.3-FlashMLX 4-bit 190 GiB exceeds memory
Ling-3.1-flashQ4_K_M 333 GiB exceeds memory
Qwen3.8-2.4T-A95Bno backend/format combination
Qwen3.8-Flash-NextQ4_K_M 111 GiB exceeds memory
DeepSeek-V4.1-FlashQ4_K_M 414 GiB exceeds memory
Muse-Spark-Open-816B-A42BQ4_K_M 465 GiB exceeds memory
MiMo-V2.6-Flash-MOPDno backend/format combination
DeepSeek-V4-Pro-0813no backend/format combination
Motif-3no backend/format combination
K2-Horizon-375B-A23Bno backend/format combination
Beam-1.0-501B-A23BQ5_K_M 331 GiB exceeds memory
MiniMax-M3Q4_K_M 247 GiB exceeds memory
Nex-N2-ProMLX 4-bit 209 GiB exceeds memory
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16MLX 4-bit 64.6 GiB exceeds memory
Kimi-K2.7-Codeno backend/format combination
Inkling-SmallQ4_K_M 151 GiB exceeds memory
K2-Horizon-MoVA-36B-A4BQ4_K_M 20.8 GiB exceeds memory
Hy3Q4_K_M 170 GiB exceeds memory
InklingQ8_0 798 GiB exceeds memory
Solar-Open2-250Bno backend/format combination
Ling-3.0-flash-VLQ4_K_M 73.3 GiB exceeds memory
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16no backend/format combination
Ling-3.0-flash-FinQ4_K_M 72.5 GiB exceeds memory
G9v3-39A5B (preview)no backend/format combination
A.X-K2no backend/format combination
NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16MLX 4-bit 38.0 GiB exceeds memory
K-EXAONE-2.0-750B-A37Bno backend/format combination
LongCat-2.0no backend/format combination
Qwen3.5-397B-A17BQ4_K_M 227 GiB exceeds memory
Qwen3.6-35B-A3BMLX 4-bit 19.0 GiB exceeds memory
Ring-2.6-1Tno backend/format combination
Qwen3.5-122B-A10BQ4_K_M 72.3 GiB exceeds memory
gemma-4-31B-itMLX 4-bit 17.1 GiB exceeds memory
Mistral-Medium-3.5-128BQ4_K_M 69.8 GiB exceeds memory
EXAONE-4.5-33BQ4_K_M 18.7 GiB exceeds memory
command-a-plus-05-2026-bf16no backend/format combination
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16Q4_K_M 23.5 GiB exceeds memory
granite-4.2-30bMLX 4-bit 15.3 GiB exceeds memory
Hypernova-60B-2605Q4_K_M 37.9 GiB exceeds memory
Nemotron-Cascade-2-30B-A3BMLX 4-bit 16.6 GiB exceeds memory
gpt-oss-120bQ4_K_M 58.5 GiB exceeds memory
LongCat-Flash-LiteMLX 4-bit 35.9 GiB exceeds memory
Mistral-Small-4-119B-2603Q4_K_M 67.6 GiB exceeds memory
Qwen3-Next-80B-A3B-Thinkingno backend/format combination
Trinity-Large-ThinkingMLX 4-bit 209 GiB exceeds memory
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16no backend/format combination
North-Mini-Code-1.0no backend/format combination
Qwen3-Next-80B-A3B-InstructQ4_K_M 45.2 GiB exceeds memory
Llama 4 Maverick 17B-128E InstructQ4_K_M 226 GiB exceeds memory
Qwen3-Coder-NextQ4_K_M 45.4 GiB exceeds memory
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16no backend/format combination
MiniCPM5-1Bno backend/format combination
sarvam-105bno backend/format combination
Nanbeige4.1-3Bno backend/format combination
Falcon-H1R-7Bno backend/format combination
Step3-VL-10Bno backend/format combination
Llama 3.3 70B InstructQ4_K_M 39.6 GiB exceeds memory
NVIDIA-Nemotron-Nano-12B-v2-VL-BF16no backend/format combination
Olmo-3.1-32B-ThinkMLX 4-bit 16.9 GiB exceeds memory
Olmo-3.1-32B-InstructMLX 4-bit 16.9 GiB exceeds memory
Llama 4 Scout 17B-16E InstructQ4_K_M 62.9 GiB exceeds memory
granite-4.0-h-smallQ4_K_M 18.1 GiB exceeds memory
Apertus-70B-Instruct-2509Q4_K_M 40.7 GiB exceeds memory
Agents-A1no backend/format combination
Agnes-2.5-Pro-Alphano backend/format combination
Agnes-3.0-Qwenno backend/format combination
Apertus-v1.1-0.5B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.1-1.5B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.1-4B-InstructContext 8K exceeds this model's configured maximum (4K)
BaguettotronContext 8K exceeds this model's configured maximum (4K)
DeepSeek-R1Q4_K_M 377 GiB exceeds memory
DeepSeek-R1-Distill-Llama-70BQ4_K_M 39.6 GiB exceeds memory
DeepSeek-R1-Distill-Qwen-32BQ4_K_M 18.5 GiB exceeds memory
DeepSeek-V3MLX 4-bit 352 GiB exceeds memory
DeepSeek-V4-Flash-0731no backend/format combination
FrogBoss-32B-2510MLX 4-bit 17.2 GiB exceeds memory
GLM-4.5-AirQ4_K_M 68.0 GiB exceeds memory
GLM-4.7-FlashQ4_K_M 17.1 GiB exceeds memory
GLM-5.2Q4_K_M 434 GiB exceeds memory
gpt-oss-puzzle-88Bno backend/format combination
granite-4.1-30bQ4_K_M 16.3 GiB exceeds memory
Hy3-previewMLX 4-bit 155 GiB exceeds memory
Hy4-previewno backend/format combination
IQuest-Coder-V1-40B-Loop-Thinkingno backend/format combination
IQuest-Coder-V1-40B-ThinkingQ4_K_M 22.4 GiB exceeds memory
Jais-3-32BQ4_K_M 19.6 GiB exceeds memory
Jais-3-375B-A23Bno backend/format combination
Jais-3-MoVA-36B-A4BQ4_K_M 20.8 GiB exceeds memory
K2-Horizon-32Bno backend/format combination
Kimi-K2.6no backend/format combination
Kolibri-1MLX 4-bit 41.3 GiB exceeds memory
Laguna-S-2.1Q5_K_M 81.8 GiB exceeds memory
Ling-3.0-flashQ4_K_M 72.9 GiB exceeds memory
Llama 2 7Bno backend/format combination
Llama 3.1 70B InstructQ4_K_M 39.6 GiB exceeds memory
Ming-flash-omni-2.0no backend/format combination
Ming-flash-omni-Previewno backend/format combination
MiniCPM-SALAno backend/format combination
MiniCPM-V-4.7-35B-A3BQ4_K_M 19.7 GiB exceeds memory
MiniMax-M2.7Q4_K_M 131 GiB exceeds memory
Mixtral 8x22B Instruct v0.1no backend/format combination
Mixtral 8x7B Instruct v0.1Q4_K_M 24.6 GiB exceeds memory
MonadContext 8K exceeds this model's configured maximum (2K)
Nemotron-3-Super-120B-A12B-BF16-MTPv2no backend/format combination
NVIDIA-Nemotron-3-Super-120B-A12B-BF16no backend/format combination
NVIDIA-Nemotron-Labs-3-Competitive-Coding-550B-A55B-NVFP4no backend/format combination
NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-BF16Q4_K_M 22.8 GiB exceeds memory
NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16no backend/format combination
NVIDIA-Nemotron-Labs-Teacher-Chatno backend/format combination
NVIDIA-Nemotron-Labs-Teacher-Competition-Codingno backend/format combination
NVIDIA-Nemotron-Labs-Teacher-Instruction-Followingno backend/format combination
NVIDIA-Nemotron-Labs-Teacher-STEMno backend/format combination
Olmo-3-32B-ThinkMLX 4-bit 16.9 GiB exceeds memory
Penguin-VL-2Bno backend/format combination
Penguin-VL-8Bno backend/format combination
Qwen2.5-32B-InstructQ4_K_M 18.5 GiB exceeds memory
Qwen2.5-72B-InstructQ4_K_M 41.0 GiB exceeds memory
Qwen2.5-Coder-32B-InstructMLX 4-bit 17.2 GiB exceeds memory
Qwen3-235B-A22BQ4_K_M 132 GiB exceeds memory
Qwen3-30B-A3BQ4_K_M 17.3 GiB exceeds memory
Qwen3-32BMLX 4-bit 17.2 GiB exceeds memory
Qwen3-Coder-30B-A3B-InstructMLX 4-bit 16.0 GiB exceeds memory
Qwen3-Coder-480B-A35B-InstructQ4_K_M 270 GiB exceeds memory
Qwen3-Nemotron-32B-RLBFFMLX 4-bit 16.2 GiB exceeds memory
Qwen3-VL-235B-A22B-InstructQ4_K_M 132 GiB exceeds memory
Qwen3-VL-235B-A22B-ThinkingQ4_K_M 132 GiB exceeds memory
Qwen3.5-27BQ4_K_M 16.7 GiB exceeds memory
Qwen3.5-35B-A3BMLX 4-bit 18.2 GiB exceeds memory
Step-3.7-FlashQ4_K_M 114 GiB exceeds memory
Apertus-v1.5-70Bno verified artifact for these backends

Specs & sources

Memory	24 GB unified	source
Bandwidth	170 GB/s	source
FP16	unknown	No unambiguous comparable value verified; never substitute AI TOPS, BF16, or whole-system power.
Power	unknown	No unambiguous comparable value verified; never substitute AI TOPS, BF16, or whole-system power.

Support page lists 153/170GB/s and assigns 170GB/s to 24GB/32GB configurable memory.16GB therefore retains 153GB/s; do not give every M6 memory bin 170GB/s. GPU FP16 and GPU-only TDP not published.

Reported speeds

0 observations

No reported observation resolved to this exact device yet.

Open in speed database Submit yours

Doesn't fit? Rent a GPU

RunpodPlaceholderCloud GPU pods you can rent for running larger models.
Vast.aiPlaceholderGPU rental marketplace with many hosts and GPU types.
LambdaPlaceholderCloud GPU instances for AI training and inference.
TensorDockPlaceholderGPU cloud marketplace with consumer and datacenter GPUs.

Sponsored placeholder links: LocalFit may earn a commission if these become affiliate links. They never influence fit results, estimates or picks.