LocalFit
Submit a benchmark

Apple M5 Ultra (80-core GPU), 512 GB: which models fit?

RTX 4090RTX 5090RTX 3090RTX 4060 Ti (16 GB)RTX 5070 TiRadeon RX 7900 XTXM4 Max 128 GBM3 Ultra 512 GBDGX Spark (128 GB)All devices

Apple Silicon Mac

Apple M5 Ultra (80-core GPU), 512 GB Not shipping yet

Memory512 GB unifiedBandwidth1,200 GB/s derivedPlanned for models384 GiB 75% of unified memoryFilesMLX Apple MLX format

Editorial pick · 512 GB MacsQwen3.8-27B MLX 8-bit All picks

Context
Files

Budget = file + KV cache + 1 GiB runtime + 0.5 GiB spare. Macs: 75% of memory. Details

46 models fit M5 Ultra 80c GPU · 512 GB · 8K context

Ranked by Artificial Analysis Intelligence Index, unranked models last. Arena is shown separately. How ranking works

Intelligence

Artificial Analysis Intelligence Index · top 10 models that fit M5 Ultra 80c GPU · 512 GB · higher is better

  1. Qwen3.8-27B33.7
  2. NVIDIA Nemotron 3.5 Super VL 120B-A12B26.5
  3. Inkling-Small25.7
  4. Solar Open 2 (250B-A15B)24.7
  5. NVIDIA Nemotron 3 Ultra 550B-A55B22.9
  6. G9v3-39A5B (preview)21.8
  7. Qwen3.5-397B-A17B21.4
  8. NVIDIA Nemotron 3.5 Bolt 74B-A9B21.4
  9. K2-Horizon-7B20.6
  10. Qwen3.6-35B-A3B18.2

AA estimate

Generation speed

Tokens per second on M5 Ultra 80c GPU · 512 GB, best file for each of the same models · higher is better

  1. G9v3-39A5B (preview)~140
  2. Qwen3.6-35B-A3B~127
  3. K2-Horizon-7B~103
  4. Inkling-Small~61.4
  5. NVIDIA Nemotron 3.5 Super VL 120B-A12B~59.1
  6. Qwen3.5-397B-A17B~42.8
  7. NVIDIA Nemotron 3.5 Bolt 74B-A9B~42
  8. Qwen3.8-27B~33.5
  9. Solar Open 2 (250B-A15B)~25.7
  10. NVIDIA Nemotron 3 Ultra 550B-A55B~13.3

MeasuredEstimated (~)

WeightsKV cacheRuntimeFreeMeasuredlinks to its sourceEstimatedshown with ~Unknownnever zero
  1. 1

    Qwen3.8-27B

    Qwen/Qwen3.8-27B

    2026-08-1427B dense

    AA index33.7rank #13
    Arena1,441
    Best file & memory
    MLX 8-bit27.5 GiB fileFits · conditional
    29.0 of 384 GiB+8K ctx · max ≈256K
    Prompt~1,877 t/sEstimated
    Generate~33.5 t/sEstimated
    First token · 4K prompt~2.2 sEstimated

    Memory budget

    Weights file27.48 GiBmlx-community/Qwen3.8-27B-8bit · 6 files HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted28.98 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 15.0 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 27.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 33.7 Qwen3.8 27B (Xhigh) · Arena 1,441 · Model page

  2. 2

    NVIDIA Nemotron 3.5 Super VL 120B-A12B

    nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16

    2026-10-15120B · 12B active

    AA index26.5rank #19
    Arena1,409
    Best file & memory
    MLX 4-bit64.6 GiB fileFits · conditional
    65.7 of 384 GiB+8K ctx · max ≈1M
    Prompt~861 t/sEstimated
    Generate~59.1 t/sEstimated
    First token · 4K prompt~4.8 sEstimated

    Memory budget

    Weights file64.65 GiBDuckets/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit · 14 files HF files
    KV cache · 8K0.06 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted65.71 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈1M tokens (native 1M).

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 6.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Sliding-window allocation is not established.
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 26.5 Nemotron 3.5 Super VL · Arena 1,409 · Model page

  3. 3

    Inkling-Small

    thinkingmachines/Inkling-Small

    2026-07-30276B · 12B active

    AA index25.7rank #21
    Arena1,413
    Best file & memory
    MLX 4-bit143 GiB fileFits · conditional
    144 of 384 GiB+8K ctx · max ≈1M
    Prompt~861 t/sEstimated
    Generate~61.4 t/sEstimated
    First token · 4K prompt~4.8 sEstimated

    Memory budget

    Weights file143 GiBmlx-community/Inkling-Small-mlx-4bit · 30 files HF files
    KV cache · 8K0.29 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted144 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈1M tokens (native 1M).

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 6.2 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 25.7 Inkling Small · Arena 1,413 · Model page

  4. 4

    Solar Open 2 (250B-A15B)

    upstage/Solar-Open2-250B

    2026-07-22250B · 15B active

    AA index24.7 AA estimaterank #25
    Arena—
    Best file & memory
    MLX 8-bit248 GiB fileFits · conditional
    249 of 384 GiB+8K ctx · max ≈1M
    Prompt~689 t/sEstimated
    Generate~25.7 t/sEstimated
    First token · 4K prompt~5.9 sEstimated

    Memory budget

    Weights file248 GiBTensorFold/Solar-Open2-250B-MLX-8bit · 62 files HF files
    KV cache · 8K0.38 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted249 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈1M tokens (native 1M).

    Other MLX files

    • MLX 4-bit 131 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 14.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Sliding-window allocation is not established.
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 24.7 AA estimate Solar Open2 250B · Arena unknown · Model page

  5. 5

    NVIDIA Nemotron 3 Ultra 550B-A55B

    nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16

    2026-06-04550B · 55B active

    AA index22.9rank #27
    Arena1,445
    Best file & memory
    MLX 4-bit288 GiB fileFits · conditional
    289 of 384 GiB+8K ctx · max ≈256K
    Prompt~188 t/sEstimated
    Generate~13.3 t/sEstimated
    First token · 4K prompt~22 sEstimated

    Memory budget

    Weights file288 GiBpipenetwork/NVIDIA-Nemotron-3-Ultra-550B-A55B-MLX-4bit · 96 files HF files
    KV cache · 8K0.09 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted289 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 544 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 28.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Sliding-window allocation is not established.
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 22.9 Nemotron 3 Ultra 550B A55B (Reasoning) · Arena 1,445 · Model page

  6. 6

    G9v3-39A5B (preview)

    ai9stars/G9v3-39A5B

    2026-08 (day unknown)39B · 5B active

    AA index21.8 AA estimaterank #29
    Arena—
    Best file & memory
    MLX 4-bit21.2 GiB fileFits
    22.5 of 384 GiB8K ctx · max ≈128K
    Prompt~2,067 t/sEstimated
    Generate~140 t/sEstimated
    First token · 4K prompt~2.0 sEstimated

    Memory budget

    Weights file21.24 GiBrapid-mlx/G9v3-39A5B-MLX-4bit · 5 files HF files
    KV cache · 8K0.30 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted22.53 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 2.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 21.8 AA estimate G9v3-39A5B · Arena unknown · Model page

  7. 7

    Qwen3.5-397B-A17B

    Qwen/Qwen3.5-397B-A17B

    2026-02-16397B · 17B active

    AA index21.4 AA estimaterank #31
    Arena1,438
    Best file & memory
    MLX 4-bit208 GiB fileFits · conditional
    210 of 384 GiB+8K ctx · max ≈256K
    Prompt~608 t/sEstimated
    Generate~42.8 t/sEstimated
    First token · 4K prompt~6.7 sEstimated

    Memory budget

    Weights file208 GiBmlx-community/Qwen3.5-397B-A17B-4bit · 46 files HF files
    KV cache · 8K0.23 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted210 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 393 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 8.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 21.4 AA estimate Qwen3.5 397B A17B (Non-reasoning) · Arena 1,438 · Model page

  8. 8

    NVIDIA Nemotron 3.5 Bolt 74B-A9B

    nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16

    2026-10-1574B · 9B active

    AA index21.4rank #32
    Arena—
    Best file & memory
    MLX 8-bit74.7 GiB fileFits · conditional
    75.8 of 384 GiB+8K ctx · max ≈256K
    Prompt~1,148 t/sEstimated
    Generate~42 t/sEstimated
    First token · 4K prompt~3.6 sEstimated

    Memory budget

    Weights file74.72 GiBnativ-community/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-8bit · 17 files HF files
    KV cache · 8K0.06 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted75.79 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 40.3 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 9.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Sliding-window allocation is not established.
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 21.4 Nemotron 3.5 Bolt 74B-A9B · Arena unknown · Model page

  9. 9

    K2-Horizon-7B

    IFM/K2-Horizon-7B

    2026-09-03size unknown

    AA index20.6rank #33
    Arena—
    Best file & memory
    MLX 8-bit8.9 GiB fileFits
    11.0 of 384 GiB8K ctx · max ≈512K
    Prompt—Unknown
    Generate~103 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file8.91 GiBmlx-community/K2-Horizon-7B-8bit · 2 files HF files
    KV cache · 8K1.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted11.03 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈512K tokens (native 512K).

    Other MLX files

    • MLX 4-bit 6.7 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.9 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 20.6 K2 Horizon 7B · Arena unknown · Model page

  10. 10

    Qwen3.6-35B-A3B

    Qwen/Qwen3.6-35B-A3B

    2026-04-1635B · 3B active

    AA index18.2rank #36
    Arena—
    Best file & memory
    MLX 8-bit35.1 GiB fileFits · conditional
    36.3 of 384 GiB+8K ctx · max ≈256K
    Prompt~3,445 t/sEstimated
    Generate~127 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file35.13 GiBmlx-community/Qwen3.6-35B-A3B-8bit · 8 files HF files
    KV cache · 8K0.16 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted36.29 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 19.0 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 18.2 Qwen3.6 35B A3B (Reasoning) · Arena unknown · Model page

  11. 11

    Qwen3.5-122B-A10B

    Qwen/Qwen3.5-122B-A10B

    2026-02-24122B · 10B active

    AA index17.7 AA estimaterank #37
    Arena1,417
    Best file & memory
    MLX 8-bit122 GiB fileFits · conditional
    123 of 384 GiB+8K ctx · max ≈256K
    Prompt~1,034 t/sEstimated
    Generate~38.3 t/sEstimated
    First token · 4K prompt~4.0 sEstimated

    Memory budget

    Weights file122 GiBmlx-community/Qwen3.5-122B-A10B-8bit · 25 files HF files
    KV cache · 8K0.19 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted123 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 64.8 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 10.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 17.7 AA estimate Qwen3.5 122B A10B (Non-reasoning) · Arena 1,417 · Model page

  12. 12

    Muse Glimmer 30B

    meta-models/Muse-Glimmer-30B

    2026-08-1030B dense

    AA index17.5rank #38
    Arena1,389
    Best file & memory
    MLX 8-bit31.1 GiB fileFits
    32.3 of 384 GiB8K ctx · max ≈128K
    Prompt~1,712 t/sEstimated
    Generate~29.6 t/sEstimated
    First token · 4K prompt~2.4 sEstimated

    Memory budget

    Weights file31.08 GiBmlx-community/Muse-Glimmer-30B-8bit · 153 files HF files
    KV cache · 8K0.18 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted32.26 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 18.1 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 31.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 17.5 Muse Glimmer (High) · Arena 1,389 · Model page

  13. 13

    K2-Horizon-3.7B

    IFM/K2-Horizon-3.7B

    2026-09-03size unknown

    AA index15.6rank #41
    Arena—
    Best file & memory
    MLX 8-bit5.0 GiB fileFits
    7.1 of 384 GiB8K ctx · max ≈512K
    Prompt—Unknown
    Generate~184 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file5.01 GiBmlx-community/K2-Horizon-3.7B-8bit · 2 files HF files
    KV cache · 8K1.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted7.13 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈512K tokens (native 512K).

    Other MLX files

    • MLX 4-bit 3.9 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 5.0 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 15.6 K2 Horizon 3.7B · Arena unknown · Model page

  14. 14

    Mistral Medium 3.5 (128B dense)

    mistralai/Mistral-Medium-3.5-128B

    2026-04-28128B dense

    AA index14.2rank #43
    Arena1,421
    Best file & memory
    MLX 4-bit72.6 GiB fileFits · conditional
    76.4 of 384 GiB+8K ctx · max ≈256K
    Prompt~396 t/sEstimated
    Generate~12.7 t/sEstimated
    First token · 4K prompt~10 sEstimated

    Memory budget

    Weights file72.65 GiBmlx-community/Mistral-Medium-3.5-128B-4bit · 15 files HF files
    KV cache · 8K2.75 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted76.40 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 72.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 14.2 Mistral Medium 3.5 · Arena 1,421 · Model page

  15. 15

    Qwen3.5-9B

    Qwen/Qwen3.5-9B

    2026-03-029B dense

    AA index13.3 AA estimaterank #45
    Arena—
    Best file & memory
    MLX 8-bit9.7 GiB fileFits · conditional
    11.0 of 384 GiB+8K ctx · max ≈256K
    Prompt~5,631 t/sEstimated
    Generate~94.7 t/sEstimated
    First token · 4K prompt~727 msEstimated

    Memory budget

    Weights file9.71 GiBmlx-community/Qwen3.5-9B-8bit · 2 files HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted10.96 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 5.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 9.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 13.3 AA estimate Qwen3.5 9B (Non-reasoning) · Arena unknown · Model page

  16. 16

    Qwen3.5-4B

    Qwen/Qwen3.5-4B

    2026-03-024B dense

    AA index13.1 AA estimaterank #48
    Arena—
    Best file & memory
    MLX 8-bit4.8 GiB fileFits · conditional
    6.0 of 384 GiB+8K ctx · max ≈256K
    Prompt~12,670 t/sEstimated
    Generate~192 t/sEstimated
    First token · 4K prompt~323 msEstimated

    Memory budget

    Weights file4.78 GiBmlx-community/Qwen3.5-4B-8bit HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted6.03 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 2.8 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 4.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 13.1 AA estimate Qwen3.5 4B (Reasoning) · Arena unknown · Model page

  17. 17

    NVIDIA Nemotron 3.5 Lightning 30B-A3B

    nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

    2026-08-1130B · 3B active

    AA index12.9rank #49
    Arena1,329
    Best file & memory
    MLX 8-bit31.3 GiB fileFits · conditional
    32.3 of 384 GiB+8K ctx · max ≈256K
    Prompt~3,445 t/sEstimated
    Generate~122 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file31.26 GiBmlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-8bit · 7 files HF files
    KV cache · 8K0.05 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted32.30 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 16.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 12.9 Nemotron 3.5 Lightning · Arena 1,329 · Model page

  18. 18

    IBM Granite 4.2 30B

    ibm-granite/granite-4.2-30b

    2026-08-2530B dense

    AA index12.7rank #50
    Arena1,361
    Best file & memory
    MLX 8-bit29.0 GiB fileFits
    32.0 of 384 GiB8K ctx · max ≈128K
    Prompt~1,689 t/sEstimated
    Generate~31.8 t/sEstimated
    First token · 4K prompt~2.4 sEstimated

    Memory budget

    Weights file28.97 GiBlmstudio-community/granite-4.2-30b-MLX-8bit · 6 files HF files
    KV cache · 8K2.00 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted31.97 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 15.3 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 29.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 12.7 Granite 4.2 30B · Arena 1,361 · Model page

  19. 19

    MiniCPM5-2B

    openbmb/MiniCPM5-2B

    2026-09-072.5B dense

    AA index12.5rank #51
    Arena—
    Best file & memory
    MLX 8-bit2.5 GiB fileFits
    3.8 of 384 GiB8K ctx · max ≈128K
    Prompt~20,137 t/sEstimated
    Generate~369 t/sEstimated
    First token · 4K prompt~203 msEstimated

    Memory budget

    Weights file2.49 GiBmlx-community/MiniCPM5-2B-8bit HF files
    KV cache · 8K0.33 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted3.82 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 1.3 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 12.5 MiniCPM5-2B · Arena unknown · Model page

  20. 20

    Nemotron Cascade 2 30B A3B

    nvidia/Nemotron-Cascade-2-30B-A3B

    2026-03-1930B · 3B active

    AA index11.7 AA estimaterank #53
    Arena—
    Best file & memory
    MLX 8-bit31.3 GiB fileFits · conditional
    32.3 of 384 GiB+8K ctx · max ≈256K
    Prompt~3,445 t/sEstimated
    Generate~122 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file31.26 GiBmlx-community/Nemotron-Cascade-2-30B-A3B-8bit · 7 files HF files
    KV cache · 8K0.05 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted32.30 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 16.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.7 AA estimate Nemotron Cascade 2 30B A3B · Arena unknown · Model page

  21. 21

    Granite 4.2 8B

    ibm-granite/granite-4.2-8b

    2026-08-258B dense

    AA index11.1rank #56
    Arena1,320
    Best file & memory
    MLX 8-bit8.7 GiB fileFits
    10.9 of 384 GiB8K ctx · max ≈128K
    Prompt~6,335 t/sEstimated
    Generate~106 t/sEstimated
    First token · 4K prompt~647 msEstimated

    Memory budget

    Weights file8.70 GiBlmstudio-community/granite-4.2-8b-MLX-8bit · 2 files HF files
    KV cache · 8K1.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted10.95 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 4.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.1 Granite 4.2 8B · Arena 1,320 · Model page

  22. 22

    G9v3-3B

    AI9Stars/G9v3-3B

    2026-07 (day unknown)3B dense

    AA index10.8 AA estimaterank #58
    Arena—
    Best file & memory
    MLX 4-bit1.6 GiB fileFits
    3.0 of 384 GiB8K ctx · max ≈128K
    Prompt~16,893 t/sEstimated
    Generate~587 t/sEstimated
    First token · 4K prompt~242 msEstimated

    Memory budget

    Weights file1.57 GiBcof139/G9v3-3B-mlx-4Bit HF files
    KV cache · 8K0.41 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.97 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 10.8 AA estimate G9v3-3B · Arena unknown · Model page

  23. 23

    Trinity-Large-Thinking

    arcee-ai/Trinity-Large-Thinking

    2026-04-01398B · 13B active

    AA index10.8rank #59
    Arena—
    Best file & memory
    MLX 4-bit209 GiB fileFits · conditional
    211 of 384 GiB+8K ctx · max ≈256K
    Prompt~795 t/sEstimated
    Generate~56 t/sEstimated
    First token · 4K prompt~5.2 sEstimated

    Memory budget

    Weights file209 GiBmlx-community/Trinity-Large-Thinking-4bit · 47 files HF files
    KV cache · 8K1.17 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted211 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 394 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 6.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 10.8 Trinity Large Thinking · Arena unknown · Model page

  24. 24

    North Mini Code

    CohereLabs/North-Mini-Code-1.0

    2026-06-0930B · 3B active

    AA index9.9rank #61
    Arena—
    Best file & memory
    MLX 8-bit31.0 GiB fileFits
    32.5 of 384 GiB8K ctx · max ≈250K
    Prompt~3,445 t/sEstimated
    Generate~123 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file30.98 GiBaaronmwelborn/North-Mini-Code-1.0-8bit · 7 files HF files
    KV cache · 8K0.48 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted32.47 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈250K tokens (native 250K).

    Other MLX files

    • MLX 4-bit 17.2 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.9 North Mini Code · Arena unknown · Model page

  25. 25

    Qwen3-Coder-Next

    Qwen/Qwen3-Coder-Next

    2026-02-0280B · 3B active

    AA index9.2rank #63
    Arena—
    Best file & memory
    MLX 8-bit78.8 GiB fileFits · conditional
    80.0 of 384 GiB+8K ctx · max ≈256K
    Prompt~3,445 t/sEstimated
    Generate~129 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file78.84 GiBmlx-community/Qwen3-Coder-Next-8bit · 17 files HF files
    KV cache · 8K0.19 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted80.03 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 41.8 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.2 Qwen3 Coder Next · Arena unknown · Model page

  26. 26

    Granite 4.2 3B

    ibm-granite/granite-4.2-3b

    2026-08-253B dense

    AA index9.1rank #64
    Arena1,293
    Best file & memory
    MLX 8-bit3.6 GiB fileFits
    5.2 of 384 GiB8K ctx · max ≈128K
    Prompt~16,893 t/sEstimated
    Generate~254 t/sEstimated
    First token · 4K prompt~242 msEstimated

    Memory budget

    Weights file3.62 GiBlmstudio-community/granite-4.2-3b-MLX-8bit HF files
    KV cache · 8K0.63 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted5.25 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 1.9 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.1 Granite 4.2 3B · Arena 1,293 · Model page

  27. 27

    MiniCPM5-1B

    openbmb/MiniCPM5-1B

    2026-05-191.1B dense

    AA index8.8 AA estimaterank #67
    Arena—
    Best file & memory
    MLX 8-bit1.1 GiB fileFits
    2.3 of 384 GiB8K ctx · max ≈128K
    Prompt~46,898 t/sEstimated
    Generate~860 t/sEstimated
    First token · 4K prompt~87 msEstimated

    Memory budget

    Weights file1.07 GiBmlx-community/MiniCPM5-1B-8bit HF files
    KV cache · 8K0.19 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.26 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 0.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.8 AA estimate MiniCPM5-1B (Reasoning) · Arena unknown · Model page

  28. 28

    Nanbeige4.1-3B

    Nanbeige/Nanbeige4.1-3B

    2026-02 (day unknown)3B dense

    AA index8.4 AA estimaterank #69
    Arena—
    Best file & memory
    MLX 8-bit3.9 GiB fileFits
    5.4 of 384 GiB8K ctx · max ≈256K
    Prompt~16,893 t/sEstimated
    Generate~236 t/sEstimated
    First token · 4K prompt~242 msEstimated

    Memory budget

    Weights file3.89 GiBmlx-community/Nanbeige4.1-3B-8bit HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted5.39 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 2.1 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.4 AA estimate Nanbeige4.1-3B · Arena unknown · Model page

  29. 29

    LFM2.5-2.6B

    LiquidAI/LFM2.5-2.6B

    2026-08-042.7B dense

    AA index8.4 AA estimaterank #70
    Arena—
    Best file & memory
    MLX 8-bit2.7 GiB fileFits · conditional
    3.8 of 384 GiB+8K ctx · max ≈128K
    Prompt~18,840 t/sEstimated
    Generate~345 t/sEstimated
    First token · 4K prompt~217 msEstimated

    Memory budget

    Weights file2.67 GiBLiquidAI/LFM2.5-2.6B-MLX-8bit HF files
    KV cache · 8K0.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted3.79 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 1.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.4 AA estimate LFM2.5-2.6B · Arena unknown · Model page

  30. 30

    Falcon-H1R-7B

    tiiuae/Falcon-H1R-7B

    2026-01-057B dense

    AA index7.8 AA estimaterank #71
    Arena—
    Best file & memory
    MLX 8-bit7.5 GiB fileFits · conditional
    8.9 of 384 GiB+8K ctx · max ≈256K
    Prompt~7,240 t/sEstimated
    Generate~123 t/sEstimated
    First token · 4K prompt~566 msEstimated

    Memory budget

    Weights file7.51 GiBmlx-community/Falcon-H1R-7B-8bit · 2 files HF files
    KV cache · 8K0.34 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted8.85 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 4.0 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 7.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.8 AA estimate Falcon-H1R-7B · Arena unknown · Model page

  31. 31

    LFM2.5-8B-A1B

    LiquidAI/LFM2.5-8B-A1B

    2026-05-288.3B · 1.5B active

    AA index7.2 AA estimaterank #74
    Arena—
    Best file & memory
    MLX 8-bit8.4 GiB fileFits · conditional
    9.5 of 384 GiB+8K ctx · max ≈125K
    Prompt~6,890 t/sEstimated
    Generate~252 t/sEstimated
    First token · 4K prompt~594 msEstimated

    Memory budget

    Weights file8.38 GiBLiquidAI/LFM2.5-8B-A1B-MLX-8bit · 2 files HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted9.48 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈125K tokens (native 125K).

    Other MLX files

    • MLX 4-bit 4.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.2 AA estimate LFM2.5-8B-A1B · Arena unknown · Model page

  32. 32

    NVIDIA Nemotron 3 Nano 4B

    nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16

    2026-03-164B dense

    AA index4.8rank #75
    Arena—
    Best file & memory
    MLX 8-bit3.9 GiB fileFits · conditional
    5.1 of 384 GiB+8K ctx · max ≈256K
    Prompt~12,766 t/sEstimated
    Generate~234 t/sEstimated
    First token · 4K prompt~321 msEstimated

    Memory budget

    Weights file3.93 GiBmlx-community/NVIDIA-Nemotron-3-Nano-4B-8bit HF files
    KV cache · 8K0.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.06 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 2.1 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 4.8 NVIDIA Nemotron 3 Nano 4B · Arena unknown · Model page

  33. 33

    LFM2.5-24B-A2B

    LiquidAI/LFM2.5-24B-A2B

    2026-10-1324B · 2B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit23.7 GiB fileFits · conditional
    24.9 of 384 GiB+8K ctx · max ≈128K
    Prompt~5,168 t/sEstimated
    Generate~193 t/sEstimated
    First token · 4K prompt~793 msEstimated

    Memory budget

    Weights file23.73 GiBLiquidAI/LFM2.5-24B-A2B-MLX-8bit · 5 files HF files
    KV cache · 8K0.16 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted24.89 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 12.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 2.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  34. 34

    MiniCPM-V 4.7 35B-A3B

    openbmb/MiniCPM-V-4.7-35B-A3B

    2026-10-1235B · 3B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit37.1 GiB fileFits · conditional
    38.3 of 384 GiB+8K ctx · max ≈256K
    Prompt~3,445 t/sEstimated
    Generate~120 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file37.10 GiBmlx-community/MiniCPM-V-4.7-35B-A3B-8bit · 8 files HF files
    KV cache · 8K0.16 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted38.25 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 21.9 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.2 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  35. 35

    Aleph Alpha Kolibri-1

    Aleph-Alpha/Kolibri-1

    2026-10-0378B · 3.5B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 4-bit41.3 GiB fileFits · conditional
    42.5 of 384 GiB+8K ctx · max ≈256K
    Prompt~2,987 t/sEstimated
    Generate~208 t/sEstimated
    First token · 4K prompt~1.4 sEstimated

    Memory budget

    Weights file41.29 GiBvelaia/Kolibri-1-MLX-4bit · 9 files HF files
    KV cache · 8K0.20 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted42.48 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  36. 36

    MiMo-V2.6-Distill-Qwen-9B

    XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B

    2026-09-229B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit9.7 GiB fileFits · conditional
    11.0 of 384 GiB+8K ctx · max ≈256K
    Prompt~5,631 t/sEstimated
    Generate~94.7 t/sEstimated
    First token · 4K prompt~727 msEstimated

    Memory budget

    Weights file9.71 GiBnativ-community/MiMo-V2.6-Distill-Qwen-9B-MLX-8bit · 2 files HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted10.96 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 5.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 9.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  37. 37

    K2-Horizon-0.9B

    IFM/K2-Horizon-0.9B

    2026-09-03size unknown

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit1.1 GiB fileFits
    2.5 of 384 GiB8K ctx · max ≈128K
    Prompt—Unknown
    Generate~862 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file1.07 GiBmlx-community/K2-Horizon-0.9B-8bit HF files
    KV cache · 8K0.44 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.50 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 0.8 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  38. 38

    Ornith-1.5-9B

    ornith-ai/Ornith-1.5-9B

    2026-08 (day unknown)9B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit8.9 GiB fileFits · conditional
    10.1 of 384 GiB+8K ctx · max ≈256K
    Prompt~5,631 t/sEstimated
    Generate~104 t/sEstimated
    First token · 4K prompt~727 msEstimated

    Memory budget

    Weights file8.86 GiBornith-ai/Ornith-1.5-9B-MLX-8bit · 2 files HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted10.11 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 4.7 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  39. 39

    Poolside Laguna S 2.1

    poolside/Laguna-S-2.1

    2026-07-21118B · 8B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit116 GiB fileFits · conditional
    118 of 384 GiB+8K ctx · max ≈1M
    Prompt~1,292 t/sEstimated
    Generate~48.4 t/sEstimated
    First token · 4K prompt~3.2 sEstimated

    Memory budget

    Weights file116 GiBTensorFold/Laguna-S-2.1-MLX-8bit · 24 files HF files
    KV cache · 8K0.45 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted118 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈1M tokens (native 1M).

    Other MLX files

    • MLX 4-bit 61.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 7.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  40. 40

    Qwen3.6-27B

    Qwen/Qwen3.6-27B

    2026-04-2227B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit27.5 GiB fileFits · conditional
    29.0 of 384 GiB+8K ctx · max ≈256K
    Prompt~1,877 t/sEstimated
    Generate~33.5 t/sEstimated
    First token · 4K prompt~2.2 sEstimated

    Memory budget

    Weights file27.48 GiBmlx-community/Qwen3.6-27B-8bit · 6 files HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted28.98 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 15.0 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 27.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  41. 41

    NVIDIA Nemotron 3 Super 120B-A12B

    nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16

    2026-03-11120B · 12B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit119 GiB fileFits · conditional
    121 of 384 GiB+8K ctx · max ≈256K
    Prompt~861 t/sEstimated
    Generate~32 t/sEstimated
    First token · 4K prompt~4.8 sEstimated

    Memory budget

    Weights file119 GiBmlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-8bit · 27 files HF files
    KV cache · 8K0.06 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted121 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 63.3 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 11.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Sliding-window allocation is not established.
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  42. 42

    Qwen3.5-0.8B

    Qwen/Qwen3.5-0.8B

    2026-03-020.8B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit0.9 GiB fileFits · conditional
    2.0 of 384 GiB+8K ctx · max ≈256K
    Prompt~63,350 t/sEstimated
    Generate~987 t/sEstimated
    First token · 4K prompt~65 msEstimated

    Memory budget

    Weights file0.93 GiBmlx-community/Qwen3.5-0.8B-8bit HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted2.03 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 0.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 0.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  43. 43

    Qwen3.5-2B

    Qwen/Qwen3.5-2B

    2026-03-022B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit2.5 GiB fileFits · conditional
    3.6 of 384 GiB+8K ctx · max ≈256K
    Prompt~25,340 t/sEstimated
    Generate~371 t/sEstimated
    First token · 4K prompt~162 msEstimated

    Memory budget

    Weights file2.48 GiBmlx-community/Qwen3.5-2B-8bit HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted3.57 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 1.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  44. 44

    Qwen3.5-27B

    Qwen/Qwen3.5-27B

    2026-02-2427B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit27.5 GiB fileFits · conditional
    29.0 of 384 GiB+8K ctx · max ≈256K
    Prompt~1,877 t/sEstimated
    Generate~33.5 t/sEstimated
    First token · 4K prompt~2.2 sEstimated

    Memory budget

    Weights file27.48 GiBmlx-community/Qwen3.5-27B-8bit · 6 files HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted28.98 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 15.0 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 27.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  45. 45

    Qwen3.5-35B-A3B

    Qwen/Qwen3.5-35B-A3B

    2026-02-2435B · 3B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit35.1 GiB fileFits · conditional
    36.3 of 384 GiB+8K ctx · max ≈256K
    Prompt~3,445 t/sEstimated
    Generate~127 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file35.13 GiBmlx-community/Qwen3.5-35B-A3B-8bit · 8 files HF files
    KV cache · 8K0.16 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted36.29 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 19.0 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  46. 46

    MiniMax-M2.7

    MiniMaxAI/MiniMax-M2.7

    2026 (month unknown)MoE · size unknown

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit226 GiB fileFits · conditional
    229 of 384 GiB+8K ctx · max ≈200K
    Prompt—Unknown
    Generate—Unknown
    First token · 4K prompt—Unknown

    Memory budget

    Weights file226 GiBmlx-community/MiniMax-M2.7-8bit · 47 files HF files
    KV cache · 8K1.94 GiBBF16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted229 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈200K tokens (native 200K).

    Other MLX files

    • MLX 4-bit 120 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate unknown: no measured report, and bandwidth, active size or backend calibration is missing.
    • First token unknown: needs a prefill value.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

May fit: KV-cache size unknown 18

Weights and runtime fit, but the cache for this attention type is not established. Ranked by AA index.

  1. GLM-5.3-Flash

    zai-org/GLM-5.3-Flash

    2026-08-26320B · 18B active

    AA index41.8rank #5
    Arena1,471
    Best file & memory
    MLX 8-bit311 GiB fileCache unknown
    312 of 384 GiB+8K ctx
    Prompt~574 t/sEstimated
    Generate~21.8 t/sEstimated
    First token · 4K prompt~7.1 sEstimated

    Memory budget

    Weights file311 GiBpipenetwork/GLM-5.3-Flash-MLX-8bit · 32 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted312 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 1M).

    Other MLX files

    • MLX 4-bit 190 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 17.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 41.8 GLM-5.3-Flash · Arena 1,471 · Model page

  2. Qwen3.8-Flash-Next

    Qwen/Qwen3.8-Flash-Next

    2026-08-26180B · 6B active

    AA index39.8rank #8
    Arena—
    Best file & memory
    MLX 4-bit104 GiB fileCache unknown
    105 of 384 GiB+8K ctx
    Prompt~1,723 t/sEstimated
    Generate~110 t/sEstimated
    First token · 4K prompt~2.4 sEstimated

    Memory budget

    Weights file104 GiBmlx-community/Qwen3.8-Flash-Next-4bit · 22 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted105 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 39.8 Qwen3.8-Flash-Next · Arena unknown · Model page

  3. Motif 3

    Motif-Technologies/Motif-3

    2026-08 (day unknown)314B · 13B active

    AA index33.6 AA estimaterank #14
    Arena—
    Best file & memory
    MLX 8-bit312 GiB fileCache unknown
    313 of 384 GiB+8K ctx
    Prompt~783 t/sEstimated
    Generate~29.1 t/sEstimated
    First token · 4K prompt~5.2 sEstimated

    Memory budget

    Weights file312 GiBavlp12/Motif-3-Alis-MLX-8bit · 78 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted313 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 13.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 33.6 AA estimate Motif 3 · Arena unknown · Model page

  4. MiniMax-M3

    MiniMaxAI/MiniMax-M3

    2026-06 (day unknown)428B · 23B active

    AA index29.2rank #17
    Arena1,433
    Best file & memory
    MLX 4-bit225 GiB fileCache unknown
    226 of 384 GiB+8K ctx
    Prompt~449 t/sEstimated
    Generate~31.6 t/sEstimated
    First token · 4K prompt~9.1 sEstimated

    Memory budget

    Weights file225 GiBmlx-community/MiniMax-M3-4bit · 58 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted226 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 1M).

    Other MLX files

    • MLX 8-bit 421 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 12.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 29.2 MiniMax-M3 · Arena 1,433 · Model page

  5. K2-Horizon-MoVA-36B-A4B

    IFM/K2-Horizon-MoVA-36B-A4B

    2026-09-0336B · 4B active

    AA index25.3rank #22
    Arena—
    Best file & memory
    MLX 8-bit37.1 GiB fileCache unknown
    38.1 of 384 GiB+8K ctx
    Prompt~2,584 t/sEstimated
    Generate~92.7 t/sEstimated
    First token · 4K prompt~1.6 sEstimated

    Memory budget

    Weights file37.07 GiBmlx-community/K2-Horizon-MoVA-36B-A4B-8bit · 8 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted38.07 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 512K).

    Other MLX files

    • MLX 4-bit 24.6 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 4.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 25.3 K2 Horizon MoVA 36B A4B · Arena unknown · Model page

  6. Gemma 4 26B A4B (it)

    google/gemma-4-26B-A4B-it

    2026-04-0225B · 3.8B active

    AA index16.7 AA estimaterank #39
    Arena1,434
    Best file & memory
    MLX 8-bit26.0 GiB fileCache unknown
    27.0 of 384 GiB+8K ctx
    Prompt~2,720 t/sEstimated
    Generate~97.2 t/sEstimated
    First token · 4K prompt~1.5 sEstimated

    Memory budget

    Weights file26.03 GiBmlx-community/gemma-4-26B-A4B-it-8bit · 6 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted27.03 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 4-bit 14.3 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 16.7 AA estimate Gemma 4 26B A4B (Reasoning) · Arena 1,434 · Model page

  7. Gemma 4 31B (it)

    google/gemma-4-31B-it

    2026-04-0231B dense

    AA index14.7rank #42
    Arena1,443
    Best file & memory
    MLX 8-bit31.4 GiB fileCache unknown
    32.4 of 384 GiB+8K ctx
    Prompt~1,651 t/sEstimated
    Generate~29.3 t/sEstimated
    First token · 4K prompt~2.5 sEstimated

    Memory budget

    Weights file31.44 GiBmlx-community/gemma-4-31B-it-8bit · 7 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted32.44 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 4-bit 17.1 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 31.4 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 14.7 Gemma 4 31B (Reasoning) · Arena 1,443 · Model page

  8. Gemma 4 12B (it)

    google/gemma-4-12B-it

    2026-06-0312B dense

    AA index14.2 AA estimaterank #44
    Arena—
    Best file & memory
    MLX 8-bit11.8 GiB fileCache unknown
    12.8 of 384 GiB+8K ctx
    Prompt~4,241 t/sEstimated
    Generate~77.7 t/sEstimated
    First token · 4K prompt~966 msEstimated

    Memory budget

    Weights file11.84 GiBmlx-community/gemma-4-12B-it-8bit · 3 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted12.84 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 4-bit 6.3 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 11.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 14.2 AA estimate Gemma 4 12B (Reasoning) · Arena unknown · Model page

  9. LongCat-Flash-Lite

    meituan-longcat/LongCat-Flash-Lite

    2026-01 (day unknown)69B · 3B active

    AA index11.5 AA estimaterank #54
    Arena—
    Best file & memory
    MLX 8-bit67.8 GiB fileCache unknown
    68.8 of 384 GiB+8K ctx
    Prompt~3,445 t/sEstimated
    Generate~128 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file67.84 GiBmlx-community/LongCat-Flash-Lite-8bit · 20 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted68.84 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 320K).

    Other MLX files

    • MLX 4-bit 35.9 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.5 AA estimate LongCat Flash Lite · Arena unknown · Model page

  10. Ling-3.0-tiny

    inclusionAI/Ling-3.0-tiny

    2026-08 (day unknown)7.9B · 1.3B active

    AA index11.1rank #57
    Arena—
    Best file & memory
    MLX 4-bit4.1 GiB fileCache unknown
    5.1 of 384 GiB+8K ctx
    Prompt~7,950 t/sEstimated
    Generate~560 t/sEstimated
    First token · 4K prompt~515 msEstimated

    Memory budget

    Weights file4.14 GiBrapid-mlx/Ling-3.0-tiny-MLX-4bit HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.14 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 0.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.1 Ling 3.0 Tiny · Arena unknown · Model page

  11. Nemotron 3 Nano Omni 30B A3B Reasoning

    nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16

    2026-04-2830B · 3B active

    AA index10.3 AA estimaterank #60
    Arena—
    Best file & memory
    MLX 8-bit33.3 GiB fileCache unknown
    34.3 of 384 GiB+8K ctx
    Prompt~3,445 t/sEstimated
    Generate~115 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file33.32 GiBmlx-community/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-8bit · 8 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted34.32 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 4-bit 18.3 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.3 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 10.3 AA estimate Nemotron 3 Nano Omni 30B A3B Reasoning · Arena unknown · Model page

  12. DiffusionGemma 26B A4B

    google/diffusiongemma-26B-A4B-it

    2026-06-1025B · 3.8B active

    AA index9.5 AA estimaterank #62
    Arena—
    Best file & memory
    MLX 8-bit26.1 GiB fileCache unknown
    27.1 of 384 GiB+8K ctx
    Prompt~2,720 t/sEstimated
    Generate~97.2 t/sEstimated
    First token · 4K prompt~1.5 sEstimated

    Memory budget

    Weights file26.05 GiBmlx-community/diffusiongemma-26B-A4B-it-8bit · 6 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted27.05 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 4-bit 15.4 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.5 AA estimate DiffusionGemma 26B A4B · Arena unknown · Model page

  13. Gemma 4 E4B (it)

    google/gemma-4-E4B-it

    2026-04-028B dense

    AA index8.9rank #66
    Arena—
    Best file & memory
    MLX 8-bit8.3 GiB fileCache unknown
    9.3 of 384 GiB+8K ctx
    Prompt~6,335 t/sEstimated
    Generate~111 t/sEstimated
    First token · 4K prompt~647 msEstimated

    Memory budget

    Weights file8.27 GiBmlx-community/gemma-4-E4B-it-8bit · 2 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted9.27 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Other MLX files

    • MLX 4-bit 4.8 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.3 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.9 Gemma 4 E4B (Reasoning) · Arena unknown · Model page

  14. Gemma 4 E2B

    google/gemma-4-E2B-it

    2026-04-025.1B dense

    AA index7.8 AA estimaterank #72
    Arena—
    Best file & memory
    MLX 8-bit5.5 GiB fileCache unknown
    6.5 of 384 GiB+8K ctx
    Prompt~9,937 t/sEstimated
    Generate~168 t/sEstimated
    First token · 4K prompt~412 msEstimated

    Memory budget

    Weights file5.46 GiBmlx-community/gemma-4-E2B-it-8bit · 2 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted6.46 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Other MLX files

    • MLX 4-bit 3.3 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 16 measured M5 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 5.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.8 AA estimate Gemma 4 E2B (Reasoning) · Arena unknown · Model page

  15. Ling-3.0-flash

    inclusionAI/Ling-3.0-flash

    2026-08-07124B · 5.1B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit123 GiB fileCache unknown
    124 of 384 GiB+8K ctx
    Prompt~2,026 t/sEstimated
    Generate~75.4 t/sEstimated
    First token · 4K prompt~2.0 sEstimated

    Memory budget

    Weights file123 GiBTensorFold/Ling-3.0-flash-MLX-8bit · 27 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted124 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 4-bit 65.2 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 5.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  16. DeepSeek V4 Flash 0731

    deepseek-ai/DeepSeek-V4-Flash-0731

    2026-07-31MoE · size unknown

    AA index—unranked
    Arena—
    Best file & memory
    MLX 4-bit144 GiB fileCache unknown
    145 of 384 GiB+8K ctx
    Prompt—Unknown
    Generate—Unknown
    First token · 4K prompt—Unknown

    Memory budget

    Weights file144 GiBToPo-ToPo/DeepSeek-V4-Flash-0731-mlx-4bit · 33 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted145 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 1M).

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate unknown: no measured report, and bandwidth, active size or backend calibration is missing.
    • First token unknown: needs a prefill value.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  17. Step 3.7 Flash

    stepfun-ai/Step-3.7-Flash

    2026-05-29198B · 11B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit195 GiB fileCache unknown
    196 of 384 GiB+8K ctx
    Prompt~940 t/sEstimated
    Generate~35.3 t/sEstimated
    First token · 4K prompt~4.4 sEstimated

    Memory budget

    Weights file195 GiBmlx-community/Step-3.7-Flash-8bit · 43 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted196 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other MLX files

    • MLX 4-bit 103 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 10.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  18. GLM-4.7-Flash

    zai-org/GLM-4.7-Flash

    2026-01 (day unknown)30B · 3B active

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit29.6 GiB fileCache unknown
    30.6 of 384 GiB+8K ctx
    Prompt~3,445 t/sEstimated
    Generate~129 t/sEstimated
    First token · 4K prompt~1.2 sEstimated

    Memory budget

    Weights file29.64 GiBmlx-community/GLM-4.7-Flash-8bit · 7 files HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted30.64 GiBof 384 GiB usable (75% of 512 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 198K).

    Other MLX files

    • MLX 4-bit 15.7 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from median of 4 measured prefill points for this device (MLX), rescaled to this model.
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

7 models too big for M5 Ultra 80c GPU · 512 GB

Specs & sources

Memory512 GB unifiedsource
Bandwidth1,200 GB/s (derived)source
FP16unknownApple publishes no comparable FP16 throughput or GPU-only TDP in reviewed chip/device technical specifications. Power adapter rating and whole-system maximum continuous power are not GPU TDP.
PowerunknownApple publishes no comparable FP16 throughput or GPU-only TDP in reviewed chip/device technical specifications. Power adapter rating and whole-system maximum continuous power are not GPU TDP.

Found in: Mac Studio (M5 Max or M5 Ultra)

Measured speeds

8 reported observations

On this device (same chip and GPU-core count). Reported by the linked sources; not reproduced by LocalFit.

Open in speed databaseSubmit yours

Doesn't fit? Rent a GPU

  • RunpodPlaceholderCloud GPU pods you can rent for running larger models.
  • Vast.aiPlaceholderGPU rental marketplace with many hosts and GPU types.
  • LambdaPlaceholderCloud GPU instances for AI training and inference.
  • TensorDockPlaceholderGPU cloud marketplace with consumer and datacenter GPUs.

Sponsored placeholder links: LocalFit may earn a commission if these become affiliate links. They never influence fit results, estimates or picks.