LocalFit
Submit a benchmark

Apple M5 (10-core GPU), 32 GB: which models fit?

RTX 4090RTX 5090RTX 3090RTX 4060 Ti (16 GB)RTX 5070 TiRadeon RX 7900 XTXM4 Max 128 GBM3 Ultra 512 GBDGX Spark (128 GB)All devices

Apple Silicon Mac

Apple M5 (10-core GPU), 32 GB

Memory32 GB unifiedBandwidth153 GB/s vendor specPlanned for models24.0 GiB 75% of unified memoryFilesMLX Apple MLX format

Editorial pick · 32 GB MacsQwen3.8-27B MLX 4-bit All picks

Context
Files

Budget = file + KV cache + 1 GiB runtime + 0.5 GiB spare. Macs: 75% of memory. Details

32 models fit M5 10c GPU · 32 GB · 8K context

Ranked by Artificial Analysis Intelligence Index, unranked models last. Arena is shown separately. How ranking works

Intelligence

Artificial Analysis Intelligence Index · top 10 models that fit M5 10c GPU · 32 GB · higher is better

  1. Qwen3.8-27B33.7
  2. G9v3-39A5B (preview)21.8
  3. K2-Horizon-7B20.6
  4. Qwen3.6-35B-A3B18.2
  5. Muse Glimmer 30B17.5
  6. K2-Horizon-3.7B15.6
  7. Qwen3.5-9B13.3
  8. Qwen3.5-4B13.1
  9. NVIDIA Nemotron 3.5 Lightning 30B-A3B12.9
  10. IBM Granite 4.2 30B12.7

AA estimate

Generation speed

Tokens per second on M5 10c GPU · 32 GB, best file for each of the same models · higher is better

  1. Qwen3.6-35B-A3B~29.9
  2. NVIDIA Nemotron 3.5 Lightning 30B-A3B~29.4
  3. Qwen3.5-4B~24.5
  4. K2-Horizon-3.7B~23.4
  5. G9v3-39A5B (preview)~17.9
  6. K2-Horizon-7B~13.2
  7. Qwen3.5-9B~12.1
  8. Qwen3.8-27B~7.8
  9. IBM Granite 4.2 30B~7.6
  10. Muse Glimmer 30B~6.5

MeasuredEstimated (~)

WeightsKV cacheRuntimeFreeMeasuredlinks to its sourceEstimatedshown with ~Unknownnever zero
  1. 1

    Qwen3.8-27B

    Qwen/Qwen3.8-27B

    2026-08-1427B dense

    AA index33.7rank #13
    Arena1,441
    Best file & memory
    MLX 4-bit15.0 GiB fileFits · conditional
    16.5 of 24.0 GiB+8K ctx · max ≈121K
    Prompt~235 t/sEstimated
    Generate~7.8 t/sEstimated
    First token · 4K prompt~17 sEstimated

    Memory budget

    Weights file14.95 GiBmlx-community/Qwen3.8-27B-4bit · 3 files HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted16.45 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈121K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 27.5 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 15.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 33.7 Qwen3.8 27B (Xhigh) · Arena 1,441 · Model page

  2. 2

    G9v3-39A5B (preview)

    ai9stars/G9v3-39A5B

    2026-08 (day unknown)39B · 5B active

    AA index21.8 AA estimaterank #29
    Arena—
    Best file & memory
    MLX 4-bit21.2 GiB fileFits
    22.5 of 24.0 GiB8K ctx · max ≈34K
    Prompt~459 t/sEstimated
    Generate~17.9 t/sEstimated
    First token · 4K prompt~8.9 sEstimated

    Memory budget

    Weights file21.24 GiBrapid-mlx/G9v3-39A5B-MLX-4bit · 5 files HF files
    KV cache · 8K0.30 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted22.53 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈34K tokens (native 128K).

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 2.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 21.8 AA estimate G9v3-39A5B · Arena unknown · Model page

  3. 3

    K2-Horizon-7B

    IFM/K2-Horizon-7B

    2026-09-03size unknown

    AA index20.6rank #33
    Arena—
    Best file & memory
    MLX 8-bit8.9 GiB fileFits
    11.0 of 24.0 GiB8K ctx · max ≈97K
    Prompt—Unknown
    Generate~13.2 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file8.91 GiBmlx-community/K2-Horizon-7B-8bit · 2 files HF files
    KV cache · 8K1.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted11.03 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈97K tokens (native 512K).

    Other MLX files

    • MLX 4-bit 6.7 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.9 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 20.6 K2 Horizon 7B · Arena unknown · Model page

  4. 4

    Qwen3.6-35B-A3B

    Qwen/Qwen3.6-35B-A3B

    2026-04-1635B · 3B active

    AA index18.2rank #36
    Arena—
    Best file & memory
    MLX 4-bit19.0 GiB fileFits · conditional
    20.2 of 24.0 GiB+8K ctx · max ≈179K
    Prompt~766 t/sEstimated
    Generate~29.9 t/sEstimated
    First token · 4K prompt~5.3 sEstimated

    Memory budget

    Weights file19.00 GiBmlx-community/Qwen3.6-35B-A3B-4bit · 4 files HF files
    KV cache · 8K0.16 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted20.16 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈179K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 35.1 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 18.2 Qwen3.6 35B A3B (Reasoning) · Arena unknown · Model page

  5. 5

    Muse Glimmer 30B

    meta-models/Muse-Glimmer-30B

    2026-08-1030B dense

    AA index17.5rank #38
    Arena1,389
    Best file & memory
    MLX 4-bit18.1 GiB fileFits
    19.3 of 24.0 GiB8K ctx · max ≈128K
    Prompt~214 t/sEstimated
    Generate~6.5 t/sEstimated
    First token · 4K prompt~19 sEstimated

    Memory budget

    Weights file18.08 GiBmlx-community/Muse-Glimmer-30B-4bit · 4 files HF files
    KV cache · 8K0.18 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted19.26 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 8-bit 31.1 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 18.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 17.5 Muse Glimmer (High) · Arena 1,389 · Model page

  6. 6

    K2-Horizon-3.7B

    IFM/K2-Horizon-3.7B

    2026-09-03size unknown

    AA index15.6rank #41
    Arena—
    Best file & memory
    MLX 8-bit5.0 GiB fileFits
    7.1 of 24.0 GiB8K ctx · max ≈124K
    Prompt—Unknown
    Generate~23.4 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file5.01 GiBmlx-community/K2-Horizon-3.7B-8bit · 2 files HF files
    KV cache · 8K1.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted7.13 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈124K tokens (native 512K).

    Other MLX files

    • MLX 4-bit 3.9 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 5.0 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 15.6 K2 Horizon 3.7B · Arena unknown · Model page

  7. 7

    Qwen3.5-9B

    Qwen/Qwen3.5-9B

    2026-03-029B dense

    AA index13.3 AA estimaterank #45
    Arena—
    Best file & memory
    MLX 8-bit9.7 GiB fileFits · conditional
    11.0 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~704 t/sEstimated
    Generate~12.1 t/sEstimated
    First token · 4K prompt~5.8 sEstimated

    Memory budget

    Weights file9.71 GiBmlx-community/Qwen3.5-9B-8bit · 2 files HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted10.96 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 5.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 9.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 13.3 AA estimate Qwen3.5 9B (Non-reasoning) · Arena unknown · Model page

  8. 8

    Qwen3.5-4B

    Qwen/Qwen3.5-4B

    2026-03-024B dense

    AA index13.1 AA estimaterank #48
    Arena—
    Best file & memory
    MLX 8-bit4.8 GiB fileFits · conditional
    6.0 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~1,584 t/sEstimated
    Generate~24.5 t/sEstimated
    First token · 4K prompt~2.6 sEstimated

    Memory budget

    Weights file4.78 GiBmlx-community/Qwen3.5-4B-8bit HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted6.03 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 2.8 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 4.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 13.1 AA estimate Qwen3.5 4B (Reasoning) · Arena unknown · Model page

  9. 9

    NVIDIA Nemotron 3.5 Lightning 30B-A3B

    nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

    2026-08-1130B · 3B active

    AA index12.9rank #49
    Arena1,329
    Best file & memory
    MLX 4-bit16.6 GiB fileFits · conditional
    17.6 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~766 t/sEstimated
    Generate~29.4 t/sEstimated
    First token · 4K prompt~5.3 sEstimated

    Memory budget

    Weights file16.55 GiBmlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-4bit · 4 files HF files
    KV cache · 8K0.05 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted17.60 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 31.3 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 12.9 Nemotron 3.5 Lightning · Arena 1,329 · Model page

  10. 10

    IBM Granite 4.2 30B

    ibm-granite/granite-4.2-30b

    2026-08-2530B dense

    AA index12.7rank #50
    Arena1,361
    Best file & memory
    MLX 4-bit15.3 GiB fileFits
    18.3 of 24.0 GiB8K ctx · max ≈29K
    Prompt~211 t/sEstimated
    Generate~7.6 t/sEstimated
    First token · 4K prompt~19 sEstimated

    Memory budget

    Weights file15.34 GiBlmstudio-community/granite-4.2-30b-MLX-4bit · 4 files HF files
    KV cache · 8K2.00 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted18.34 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈29K tokens (native 128K).

    Other MLX files

    • MLX 8-bit 29.0 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 15.3 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 12.7 Granite 4.2 30B · Arena 1,361 · Model page

  11. 11

    MiniCPM5-2B

    openbmb/MiniCPM5-2B

    2026-09-072.5B dense

    AA index12.5rank #51
    Arena—
    Best file & memory
    MLX 8-bit2.5 GiB fileFits
    3.8 of 24.0 GiB8K ctx · max ≈128K
    Prompt~2,517 t/sEstimated
    Generate~47.1 t/sEstimated
    First token · 4K prompt~1.6 sEstimated

    Memory budget

    Weights file2.49 GiBmlx-community/MiniCPM5-2B-8bit HF files
    KV cache · 8K0.33 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted3.82 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 1.3 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 12.5 MiniCPM5-2B · Arena unknown · Model page

  12. 12

    Nemotron Cascade 2 30B A3B

    nvidia/Nemotron-Cascade-2-30B-A3B

    2026-03-1930B · 3B active

    AA index11.7 AA estimaterank #53
    Arena—
    Best file & memory
    MLX 4-bit16.6 GiB fileFits · conditional
    17.6 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~766 t/sEstimated
    Generate~29.4 t/sEstimated
    First token · 4K prompt~5.3 sEstimated

    Memory budget

    Weights file16.55 GiBmlx-community/Nemotron-Cascade-2-30B-A3B-4bit · 4 files HF files
    KV cache · 8K0.05 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted17.60 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 31.3 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.7 AA estimate Nemotron Cascade 2 30B A3B · Arena unknown · Model page

  13. 13

    Granite 4.2 8B

    ibm-granite/granite-4.2-8b

    2026-08-258B dense

    AA index11.1rank #56
    Arena1,320
    Best file & memory
    MLX 8-bit8.7 GiB fileFits
    10.9 of 24.0 GiB8K ctx · max ≈88K
    Prompt~792 t/sEstimated
    Generate~13.5 t/sEstimated
    First token · 4K prompt~5.2 sEstimated

    Memory budget

    Weights file8.70 GiBlmstudio-community/granite-4.2-8b-MLX-8bit · 2 files HF files
    KV cache · 8K1.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted10.95 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈88K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 4.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.1 Granite 4.2 8B · Arena 1,320 · Model page

  14. 14

    G9v3-3B

    AI9Stars/G9v3-3B

    2026-07 (day unknown)3B dense

    AA index10.8 AA estimaterank #58
    Arena—
    Best file & memory
    MLX 4-bit1.6 GiB fileFits
    3.0 of 24.0 GiB8K ctx · max ≈128K
    Prompt~2,112 t/sEstimated
    Generate~74.9 t/sEstimated
    First token · 4K prompt~1.9 sEstimated

    Memory budget

    Weights file1.57 GiBcof139/G9v3-3B-mlx-4Bit HF files
    KV cache · 8K0.41 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.97 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 10.8 AA estimate G9v3-3B · Arena unknown · Model page

  15. 15

    North Mini Code

    CohereLabs/North-Mini-Code-1.0

    2026-06-0930B · 3B active

    AA index9.9rank #61
    Arena—
    Best file & memory
    MLX 4-bit17.2 GiB fileFits
    18.7 of 24.0 GiB8K ctx · max ≈197K
    Prompt~766 t/sEstimated
    Generate~28.3 t/sEstimated
    First token · 4K prompt~5.3 sEstimated

    Memory budget

    Weights file17.23 GiBmlx-community/North-Mini-Code-1.0-4bit · 4 files HF files
    KV cache · 8K0.48 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted18.71 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈197K tokens (native 250K).

    Other MLX files

    • MLX 8-bit 31.0 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.9 North Mini Code · Arena unknown · Model page

  16. 16

    Granite 4.2 3B

    ibm-granite/granite-4.2-3b

    2026-08-253B dense

    AA index9.1rank #64
    Arena1,293
    Best file & memory
    MLX 8-bit3.6 GiB fileFits
    5.2 of 24.0 GiB8K ctx · max ≈128K
    Prompt~2,112 t/sEstimated
    Generate~32.4 t/sEstimated
    First token · 4K prompt~1.9 sEstimated

    Memory budget

    Weights file3.62 GiBlmstudio-community/granite-4.2-3b-MLX-8bit HF files
    KV cache · 8K0.63 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted5.25 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 1.9 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.1 Granite 4.2 3B · Arena 1,293 · Model page

  17. 17

    MiniCPM5-1B

    openbmb/MiniCPM5-1B

    2026-05-191.1B dense

    AA index8.8 AA estimaterank #67
    Arena—
    Best file & memory
    MLX 8-bit1.1 GiB fileFits
    2.3 of 24.0 GiB8K ctx · max ≈128K
    Prompt~5,862 t/sEstimated
    Generate~110 t/sEstimated
    First token · 4K prompt~699 msEstimated

    Memory budget

    Weights file1.07 GiBmlx-community/MiniCPM5-1B-8bit HF files
    KV cache · 8K0.19 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.26 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 0.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.8 AA estimate MiniCPM5-1B (Reasoning) · Arena unknown · Model page

  18. 18

    Nanbeige4.1-3B

    Nanbeige/Nanbeige4.1-3B

    2026-02 (day unknown)3B dense

    AA index8.4 AA estimaterank #69
    Arena—
    Best file & memory
    MLX 8-bit3.9 GiB fileFits
    5.4 of 24.0 GiB8K ctx · max ≈256K
    Prompt~2,112 t/sEstimated
    Generate~30.1 t/sEstimated
    First token · 4K prompt~1.9 sEstimated

    Memory budget

    Weights file3.89 GiBmlx-community/Nanbeige4.1-3B-8bit HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted5.39 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 2.1 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.4 AA estimate Nanbeige4.1-3B · Arena unknown · Model page

  19. 19

    LFM2.5-2.6B

    LiquidAI/LFM2.5-2.6B

    2026-08-042.7B dense

    AA index8.4 AA estimaterank #70
    Arena—
    Best file & memory
    MLX 8-bit2.7 GiB fileFits · conditional
    3.8 of 24.0 GiB+8K ctx · max ≈128K
    Prompt~2,355 t/sEstimated
    Generate~43.9 t/sEstimated
    First token · 4K prompt~1.7 sEstimated

    Memory budget

    Weights file2.67 GiBLiquidAI/LFM2.5-2.6B-MLX-8bit HF files
    KV cache · 8K0.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted3.79 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 1.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.4 AA estimate LFM2.5-2.6B · Arena unknown · Model page

  20. 20

    Falcon-H1R-7B

    tiiuae/Falcon-H1R-7B

    2026-01-057B dense

    AA index7.8 AA estimaterank #71
    Arena—
    Best file & memory
    MLX 8-bit7.5 GiB fileFits · conditional
    8.9 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~905 t/sEstimated
    Generate~15.6 t/sEstimated
    First token · 4K prompt~4.5 sEstimated

    Memory budget

    Weights file7.51 GiBmlx-community/Falcon-H1R-7B-8bit · 2 files HF files
    KV cache · 8K0.34 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted8.85 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 4.0 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 7.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.8 AA estimate Falcon-H1R-7B · Arena unknown · Model page

  21. 21

    LFM2.5-8B-A1B

    LiquidAI/LFM2.5-8B-A1B

    2026-05-288.3B · 1.5B active

    AA index7.2 AA estimaterank #74
    Arena—
    Best file & memory
    MLX 8-bit8.4 GiB fileFits · conditional
    9.5 of 24.0 GiB+8K ctx · max ≈125K
    Prompt~1,531 t/sEstimated
    Generate~32.1 t/sEstimated
    First token · 4K prompt~2.7 sEstimated

    Memory budget

    Weights file8.38 GiBLiquidAI/LFM2.5-8B-A1B-MLX-8bit · 2 files HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted9.48 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈125K tokens (native 125K).

    Other MLX files

    • MLX 4-bit 4.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.2 AA estimate LFM2.5-8B-A1B · Arena unknown · Model page

  22. 22

    NVIDIA Nemotron 3 Nano 4B

    nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16

    2026-03-164B dense

    AA index4.8rank #75
    Arena—
    Best file & memory
    MLX 8-bit3.9 GiB fileFits · conditional
    5.1 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~1,596 t/sEstimated
    Generate~29.8 t/sEstimated
    First token · 4K prompt~2.6 sEstimated

    Memory budget

    Weights file3.93 GiBmlx-community/NVIDIA-Nemotron-3-Nano-4B-8bit HF files
    KV cache · 8K0.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.06 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 2.1 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 4.8 NVIDIA Nemotron 3 Nano 4B · Arena unknown · Model page

  23. 23

    LFM2.5-24B-A2B

    LiquidAI/LFM2.5-24B-A2B

    2026-10-1324B · 2B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 4-bit12.6 GiB fileFits · conditional
    13.8 of 24.0 GiB+8K ctx · max ≈128K
    Prompt~1,149 t/sEstimated
    Generate~46.2 t/sEstimated
    First token · 4K prompt~3.6 sEstimated

    Memory budget

    Weights file12.64 GiBLiquidAI/LFM2.5-24B-A2B-MLX-4bit · 3 files HF files
    KV cache · 8K0.16 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted13.79 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 8-bit 23.7 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  24. 24

    MiniCPM-V 4.7 35B-A3B

    openbmb/MiniCPM-V-4.7-35B-A3B

    2026-10-1235B · 3B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 4-bit21.9 GiB fileFits · conditional
    23.1 of 24.0 GiB+8K ctx · max ≈30K
    Prompt~766 t/sEstimated
    Generate~25.9 t/sEstimated
    First token · 4K prompt~5.3 sEstimated

    Memory budget

    Weights file21.91 GiBmlx-community/MiniCPM-V-4.7-35B-A3B-4bit · 5 files HF files
    KV cache · 8K0.16 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted23.07 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈30K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 37.1 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  25. 25

    MiMo-V2.6-Distill-Qwen-9B

    XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B

    2026-09-229B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit9.7 GiB fileFits · conditional
    11.0 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~704 t/sEstimated
    Generate~12.1 t/sEstimated
    First token · 4K prompt~5.8 sEstimated

    Memory budget

    Weights file9.71 GiBnativ-community/MiMo-V2.6-Distill-Qwen-9B-MLX-8bit · 2 files HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted10.96 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 5.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 9.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  26. 26

    K2-Horizon-0.9B

    IFM/K2-Horizon-0.9B

    2026-09-03size unknown

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit1.1 GiB fileFits
    2.5 of 24.0 GiB8K ctx · max ≈128K
    Prompt—Unknown
    Generate~110 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file1.07 GiBmlx-community/K2-Horizon-0.9B-8bit HF files
    KV cache · 8K0.44 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.50 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 0.8 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  27. 27

    Ornith-1.5-9B

    ornith-ai/Ornith-1.5-9B

    2026-08 (day unknown)9B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit8.9 GiB fileFits · conditional
    10.1 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~704 t/sEstimated
    Generate~13.2 t/sEstimated
    First token · 4K prompt~5.8 sEstimated

    Memory budget

    Weights file8.86 GiBornith-ai/Ornith-1.5-9B-MLX-8bit · 2 files HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted10.11 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 4.7 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  28. 28

    Qwen3.6-27B

    Qwen/Qwen3.6-27B

    2026-04-2227B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 4-bit15.0 GiB fileFits · conditional
    16.5 of 24.0 GiB+8K ctx · max ≈121K
    Prompt~235 t/sEstimated
    Generate~7.8 t/sEstimated
    First token · 4K prompt~17 sEstimated

    Memory budget

    Weights file14.95 GiBmlx-community/Qwen3.6-27B-4bit · 3 files HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted16.45 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈121K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 27.5 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 15.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  29. 29

    Qwen3.5-0.8B

    Qwen/Qwen3.5-0.8B

    2026-03-020.8B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit0.9 GiB fileFits · conditional
    2.0 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~7,919 t/sEstimated
    Generate~126 t/sEstimated
    First token · 4K prompt~517 msEstimated

    Memory budget

    Weights file0.93 GiBmlx-community/Qwen3.5-0.8B-8bit HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted2.03 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 0.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 0.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  30. 30

    Qwen3.5-2B

    Qwen/Qwen3.5-2B

    2026-03-022B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit2.5 GiB fileFits · conditional
    3.6 of 24.0 GiB+8K ctx · max ≈256K
    Prompt~3,168 t/sEstimated
    Generate~47.3 t/sEstimated
    First token · 4K prompt~1.3 sEstimated

    Memory budget

    Weights file2.48 GiBmlx-community/Qwen3.5-2B-8bit HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted3.57 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 1.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  31. 31

    Qwen3.5-27B

    Qwen/Qwen3.5-27B

    2026-02-2427B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 4-bit15.0 GiB fileFits · conditional
    16.5 of 24.0 GiB+8K ctx · max ≈121K
    Prompt~235 t/sEstimated
    Generate~7.8 t/sEstimated
    First token · 4K prompt~17 sEstimated

    Memory budget

    Weights file14.95 GiBmlx-community/Qwen3.5-27B-4bit · 3 files HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted16.45 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈121K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 27.5 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 15.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  32. 32

    Qwen3.5-35B-A3B

    Qwen/Qwen3.5-35B-A3B

    2026-02-2435B · 3B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 4-bit19.0 GiB fileFits · conditional
    20.1 of 24.0 GiB+8K ctx · max ≈180K
    Prompt~766 t/sEstimated
    Generate~29.9 t/sEstimated
    First token · 4K prompt~5.3 sEstimated

    Memory budget

    Weights file18.99 GiBmlx-community/Qwen3.5-35B-A3B-4bit · 4 files HF files
    KV cache · 8K0.16 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted20.15 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈180K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 35.1 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

May fit: KV-cache size unknown 9

Weights and runtime fit, but the cache for this attention type is not established. Ranked by AA index.

  1. Gemma 4 26B A4B (it)

    google/gemma-4-26B-A4B-it

    2026-04-0225B · 3.8B active

    AA index16.7 AA estimaterank #39
    Arena1,434
    Best file & memory
    MLX 4-bit14.3 GiB fileCache unknown
    15.3 of 24.0 GiB+8K ctx
    Prompt~604 t/sEstimated
    Generate~22.6 t/sEstimated
    First token · 4K prompt~6.8 sEstimated

    Memory budget

    Weights file14.29 GiBmlx-community/gemma-4-26B-A4B-it-4bit · 3 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted15.29 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 8-bit 26.0 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 2.2 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 16.7 AA estimate Gemma 4 26B A4B (Reasoning) · Arena 1,434 · Model page

  2. Gemma 4 31B (it)

    google/gemma-4-31B-it

    2026-04-0231B dense

    AA index14.7rank #42
    Arena1,443
    Best file & memory
    MLX 4-bit17.1 GiB fileCache unknown
    18.1 of 24.0 GiB+8K ctx
    Prompt~206 t/sEstimated
    Generate~6.8 t/sEstimated
    First token · 4K prompt~20 sEstimated

    Memory budget

    Weights file17.15 GiBmlx-community/gemma-4-31B-it-4bit · 4 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted18.15 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 8-bit 31.4 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 17.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 14.7 Gemma 4 31B (Reasoning) · Arena 1,443 · Model page

  3. Gemma 4 12B (it)

    google/gemma-4-12B-it

    2026-06-0312B dense

    AA index14.2 AA estimaterank #44
    Arena—
    Best file & memory
    MLX 8-bit11.8 GiB fileCache unknown
    12.8 of 24.0 GiB+8K ctx
    Prompt~530 t/sEstimated
    Generate~9.9 t/sEstimated
    First token · 4K prompt~7.7 sEstimated

    Memory budget

    Weights file11.84 GiBmlx-community/gemma-4-12B-it-8bit · 3 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted12.84 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 4-bit 6.3 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 11.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 14.2 AA estimate Gemma 4 12B (Reasoning) · Arena unknown · Model page

  4. Ling-3.0-tiny

    inclusionAI/Ling-3.0-tiny

    2026-08 (day unknown)7.9B · 1.3B active

    AA index11.1rank #57
    Arena—
    Best file & memory
    MLX 4-bit4.1 GiB fileCache unknown
    5.1 of 24.0 GiB+8K ctx
    Prompt~1,767 t/sEstimated
    Generate~71.4 t/sEstimated
    First token · 4K prompt~2.3 sEstimated

    Memory budget

    Weights file4.14 GiBrapid-mlx/Ling-3.0-tiny-MLX-4bit HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.14 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 0.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.1 Ling 3.0 Tiny · Arena unknown · Model page

  5. Nemotron 3 Nano Omni 30B A3B Reasoning

    nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16

    2026-04-2830B · 3B active

    AA index10.3 AA estimaterank #60
    Arena—
    Best file & memory
    MLX 4-bit18.3 GiB fileCache unknown
    19.3 of 24.0 GiB+8K ctx
    Prompt~766 t/sEstimated
    Generate~26.6 t/sEstimated
    First token · 4K prompt~5.3 sEstimated

    Memory budget

    Weights file18.29 GiBmlx-community/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-4bit · 4 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted19.29 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 8-bit 33.3 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 10.3 AA estimate Nemotron 3 Nano Omni 30B A3B Reasoning · Arena unknown · Model page

  6. DiffusionGemma 26B A4B

    google/diffusiongemma-26B-A4B-it

    2026-06-1025B · 3.8B active

    AA index9.5 AA estimaterank #62
    Arena—
    Best file & memory
    MLX 4-bit15.4 GiB fileCache unknown
    16.4 of 24.0 GiB+8K ctx
    Prompt~604 t/sEstimated
    Generate~21 t/sEstimated
    First token · 4K prompt~6.8 sEstimated

    Memory budget

    Weights file15.41 GiBmlx-community/diffusiongemma-26B-A4B-it-4bit · 4 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted16.41 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 8-bit 26.1 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 2.3 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.5 AA estimate DiffusionGemma 26B A4B · Arena unknown · Model page

  7. Gemma 4 E4B (it)

    google/gemma-4-E4B-it

    2026-04-028B dense

    AA index8.9rank #66
    Arena—
    Best file & memory
    MLX 8-bit8.3 GiB fileCache unknown
    9.3 of 24.0 GiB+8K ctx
    Prompt~792 t/sEstimated
    Generate~14.2 t/sEstimated
    First token · 4K prompt~5.2 sEstimated

    Memory budget

    Weights file8.27 GiBmlx-community/gemma-4-E4B-it-8bit · 2 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted9.27 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Other MLX files

    • MLX 4-bit 4.8 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.3 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.9 Gemma 4 E4B (Reasoning) · Arena unknown · Model page

  8. Gemma 4 E2B

    google/gemma-4-E2B-it

    2026-04-025.1B dense

    AA index7.8 AA estimaterank #72
    Arena—
    Best file & memory
    MLX 8-bit5.5 GiB fileCache unknown
    6.5 of 24.0 GiB+8K ctx
    Prompt~1,242 t/sEstimated
    Generate~21.5 t/sEstimated
    First token · 4K prompt~3.3 sEstimated

    Memory budget

    Weights file5.46 GiBmlx-community/gemma-4-E2B-it-8bit · 2 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted6.46 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Other MLX files

    • MLX 4-bit 3.3 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 5.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.8 AA estimate Gemma 4 E2B (Reasoning) · Arena unknown · Model page

  9. GLM-4.7-Flash

    zai-org/GLM-4.7-Flash

    2026-01 (day unknown)30B · 3B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 4-bit15.7 GiB fileCache unknown
    16.7 of 24.0 GiB+8K ctx
    Prompt~766 t/sEstimated
    Generate~31 t/sEstimated
    First token · 4K prompt~5.3 sEstimated

    Memory budget

    Weights file15.70 GiBmlx-community/GLM-4.7-Flash-4bit · 4 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted16.70 GiBof 24.0 GiB usable (75% of 32 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 198K).

    Other MLX files

    • MLX 8-bit 29.6 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 37 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

30 models too big for M5 10c GPU · 32 GB

Specs & sources

Memory32 GB unifiedsource
Bandwidth153 GB/ssource
FP16unknownApple publishes no comparable FP16 throughput or GPU-only TDP in reviewed chip/device technical specifications. Power adapter rating and whole-system maximum continuous power are not GPU TDP.
PowerunknownApple publishes no comparable FP16 throughput or GPU-only TDP in reviewed chip/device technical specifications. Power adapter rating and whole-system maximum continuous power are not GPU TDP.

Found in: MacBook Air (13-inch, M5, 2026)

Measured speeds

8 reported observations

On this device (same chip and GPU-core count). Reported by the linked sources; not reproduced by LocalFit.

Open in speed databaseSubmit yours

Doesn't fit? Rent a GPU

  • RunpodPlaceholderCloud GPU pods you can rent for running larger models.
  • Vast.aiPlaceholderGPU rental marketplace with many hosts and GPU types.
  • LambdaPlaceholderCloud GPU instances for AI training and inference.
  • TensorDockPlaceholderGPU cloud marketplace with consumer and datacenter GPUs.

Sponsored placeholder links: LocalFit may earn a commission if these become affiliate links. They never influence fit results, estimates or picks.