Apple M2 Ultra (76-core GPU), 192 GB: which models fit?

RTX 4090 RTX 5090 RTX 3090 RTX 4060 Ti (16 GB)RTX 5070 Ti Radeon RX 7900 XTX M4 Max 128 GB M3 Ultra 512 GB DGX Spark (128 GB)All devices

Apple Silicon Mac

Apple M2 Ultra (76-core GPU), 192 GB

Memory192 GB unifiedBandwidth800 GB/s vendor specPlanned for models144 GiB 75% of unified memoryFilesMLX Apple MLX format

Editorial pick · 192 GB MacsQwen3.8-27B MLX 8-bit All picks

Budget = file + KV cache + 1 GiB runtime + 0.5 GiB spare. Macs: 75% of memory. Details

42 models fit M2 Ultra 76c GPU · 192 GB · 8K context

Ranked by Artificial Analysis Intelligence Index, unranked models last. Arena is shown separately. How ranking works

Intelligence

Artificial Analysis Intelligence Index · top 10 models that fit M2 Ultra 76c GPU · 192 GB · higher is better

Qwen3.8-27B33.7
NVIDIA Nemotron 3.5 Super VL 120B-A12B26.5
Solar Open 2 (250B-A15B)24.7
G9v3-39A5B (preview)21.8
NVIDIA Nemotron 3.5 Bolt 74B-A9B21.4
K2-Horizon-7B20.6
Qwen3.6-35B-A3B18.2
Qwen3.5-122B-A10B17.7
Muse Glimmer 30B17.5
K2-Horizon-3.7B15.6

AA estimate

Generation speed

Tokens per second on M2 Ultra 76c GPU · 192 GB, best file for each of the same models · higher is better

K2-Horizon-3.7B~123
G9v3-39A5B (preview)~93.5
Qwen3.6-35B-A3B~84.5
K2-Horizon-7B~68.9
NVIDIA Nemotron 3.5 Super VL 120B-A12B~39.4
Solar Open 2 (250B-A15B)~32.3
NVIDIA Nemotron 3.5 Bolt 74B-A9B~28
Qwen3.5-122B-A10B~25.5
Qwen3.8-27B~22.3
Muse Glimmer 30B~19.7

MeasuredEstimated (~)

WeightsKV cacheRuntimeFreeMeasuredlinks to its sourceEstimatedshown with ~Unknownnever zero

Qwen3.8-27B

Qwen/Qwen3.8-27B

2026-08-1427B dense

AA index33.7rank #13

Arena1,441

Best file & memory

MLX 8-bit27.5 GiB fileFits · conditional

29.0 of 144 GiB+8K ctx · max ≈256K

Prompt~250 t/sEstimated

Generate~22.3 t/sEstimated

First token · 4K prompt~16 sEstimated

Memory budget

Weights file	27.48 GiB	mlx-community/Qwen3.8-27B-8bit · 6 files HF files
KV cache · 8K	0.50 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	28.98 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 15.0 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 27.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 33.7 Qwen3.8 27B (Xhigh) · Arena 1,441 · Model page

NVIDIA Nemotron 3.5 Super VL 120B-A12B

nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16

2026-10-15120B · 12B active

AA index26.5rank #19

Arena1,409

Best file & memory

MLX 4-bit64.6 GiB fileFits · conditional

65.7 of 144 GiB+8K ctx · max ≈1M

Prompt~347 t/sEstimated

Generate~39.4 t/sEstimated

First token · 4K prompt~12 sEstimated

Memory budget

Weights file	64.65 GiB	Duckets/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit · 14 files HF files
KV cache · 8K	0.06 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	65.71 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈1M tokens (native 1M).

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 6.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Sliding-window allocation is not established.
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 26.5 Nemotron 3.5 Super VL · Arena 1,409 · Model page

Solar Open 2 (250B-A15B)

upstage/Solar-Open2-250B

2026-07-22250B · 15B active

AA index24.7 AA estimaterank #25

Arena—

Best file & memory

MLX 4-bit131 GiB fileFits · conditional

133 of 144 GiB+8K ctx · max ≈241K

Prompt~278 t/sEstimated

Generate~32.3 t/sEstimated

First token · 4K prompt~15 sEstimated

Memory budget

Weights file	131 GiB	TensorFold/Solar-Open2-250B-MLX-4bit · 29 files HF files
KV cache · 8K	0.38 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	133 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈241K tokens (native 1M).

Other MLX files

MLX 8-bit 248 GiB · too big · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 7.9 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Sliding-window allocation is not established.
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 24.7 AA estimate Solar Open2 250B · Arena unknown · Model page

G9v3-39A5B (preview)

ai9stars/G9v3-39A5B

2026-08 (day unknown)39B · 5B active

AA index21.8 AA estimaterank #29

Arena—

Best file & memory

MLX 4-bit21.2 GiB fileFits

22.5 of 144 GiB8K ctx · max ≈128K

Prompt~833 t/sEstimated

Generate~93.5 t/sEstimated

First token · 4K prompt~4.9 sEstimated

Memory budget

Weights file	21.24 GiB	rapid-mlx/G9v3-39A5B-MLX-4bit · 5 files HF files
KV cache · 8K	0.30 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	22.53 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 2.7 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 21.8 AA estimate G9v3-39A5B · Arena unknown · Model page

NVIDIA Nemotron 3.5 Bolt 74B-A9B

nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16

2026-10-1574B · 9B active

AA index21.4rank #32

Arena—

Best file & memory

MLX 8-bit74.7 GiB fileFits · conditional

75.8 of 144 GiB+8K ctx · max ≈256K

Prompt~463 t/sEstimated

Generate~28 t/sEstimated

First token · 4K prompt~8.9 sEstimated

Memory budget

Weights file	74.72 GiB	nativ-community/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-8bit · 17 files HF files
KV cache · 8K	0.06 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	75.79 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 40.3 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 9.1 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Sliding-window allocation is not established.
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 21.4 Nemotron 3.5 Bolt 74B-A9B · Arena unknown · Model page

K2-Horizon-7B

IFM/K2-Horizon-7B

2026-09-03size unknown

AA index20.6rank #33

Arena—

Best file & memory

MLX 8-bit8.9 GiB fileFits

11.0 of 144 GiB8K ctx · max ≈512K

Prompt—Unknown

Generate~68.9 t/sEstimated

First token · 4K prompt—Unknown

Memory budget

Weights file	8.91 GiB	mlx-community/K2-Horizon-7B-8bit · 2 files HF files
KV cache · 8K	1.13 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	11.03 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈512K tokens (native 512K).

Other MLX files

MLX 4-bit 6.7 GiB · fits · files

Where the speeds come from

Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.9 GiB read per token.
First token unknown: needs a prefill value.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 20.6 K2 Horizon 7B · Arena unknown · Model page

Qwen3.6-35B-A3B

Qwen/Qwen3.6-35B-A3B

2026-04-1635B · 3B active

AA index18.2rank #36

Arena—

Best file & memory

MLX 8-bit35.1 GiB fileFits · conditional

36.3 of 144 GiB+8K ctx · max ≈256K

Prompt~1,388 t/sEstimated

Generate~84.5 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	35.13 GiB	mlx-community/Qwen3.6-35B-A3B-8bit · 8 files HF files
KV cache · 8K	0.16 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	36.29 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 19.0 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 18.2 Qwen3.6 35B A3B (Reasoning) · Arena unknown · Model page

Qwen3.5-122B-A10B

Qwen/Qwen3.5-122B-A10B

2026-02-24122B · 10B active

AA index17.7 AA estimaterank #37

Arena1,417

Best file & memory

MLX 8-bit122 GiB fileFits · conditional

123 of 144 GiB+8K ctx · max ≈256K

Prompt~416 t/sEstimated

Generate~25.5 t/sEstimated

First token · 4K prompt~9.8 sEstimated

Memory budget

Weights file	122 GiB	mlx-community/Qwen3.5-122B-A10B-8bit · 25 files HF files
KV cache · 8K	0.19 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	123 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 64.8 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 10.0 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 17.7 AA estimate Qwen3.5 122B A10B (Non-reasoning) · Arena 1,417 · Model page

Muse Glimmer 30B

meta-models/Muse-Glimmer-30B

2026-08-1030B dense

AA index17.5rank #38

Arena1,389

Best file & memory

MLX 8-bit31.1 GiB fileFits

32.3 of 144 GiB8K ctx · max ≈128K

Prompt~228 t/sEstimated

Generate~19.7 t/sEstimated

First token · 4K prompt~18 sEstimated

Memory budget

Weights file	31.08 GiB	mlx-community/Muse-Glimmer-30B-8bit · 153 files HF files
KV cache · 8K	0.18 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	32.26 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Other MLX files

MLX 4-bit 18.1 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 31.1 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 17.5 Muse Glimmer (High) · Arena 1,389 · Model page

K2-Horizon-3.7B

IFM/K2-Horizon-3.7B

2026-09-03size unknown

AA index15.6rank #41

Arena—

Best file & memory

MLX 8-bit5.0 GiB fileFits

7.1 of 144 GiB8K ctx · max ≈512K

Prompt—Unknown

Generate~123 t/sEstimated

First token · 4K prompt—Unknown

Memory budget

Weights file	5.01 GiB	mlx-community/K2-Horizon-3.7B-8bit · 2 files HF files
KV cache · 8K	1.13 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	7.13 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈512K tokens (native 512K).

Other MLX files

MLX 4-bit 3.9 GiB · fits · files

Where the speeds come from

Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 5.0 GiB read per token.
First token unknown: needs a prefill value.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 15.6 K2 Horizon 3.7B · Arena unknown · Model page

Mistral Medium 3.5 (128B dense)

mistralai/Mistral-Medium-3.5-128B

2026-04-28128B dense

AA index14.2rank #43

Arena1,421

Best file & memory

MLX 4-bit72.6 GiB fileFits · conditional

76.4 of 144 GiB+8K ctx · max ≈203K

Prompt~52.7 t/sEstimated

Generate~8.4 t/sEstimated

First token · 4K prompt~78 sEstimated

Memory budget

Weights file	72.65 GiB	mlx-community/Mistral-Medium-3.5-128B-4bit · 15 files HF files
KV cache · 8K	2.75 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	76.40 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈203K tokens (native 256K).

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 72.6 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 14.2 Mistral Medium 3.5 · Arena 1,421 · Model page

Qwen3.5-9B

Qwen/Qwen3.5-9B

2026-03-029B dense

AA index13.3 AA estimaterank #45

Arena—

Best file & memory

MLX 8-bit9.7 GiB fileFits · conditional

11.0 of 144 GiB+8K ctx · max ≈256K

Prompt~750 t/sEstimated

Generate~63.2 t/sEstimated

First token · 4K prompt~5.5 sEstimated

Memory budget

Weights file	9.71 GiB	mlx-community/Qwen3.5-9B-8bit · 2 files HF files
KV cache · 8K	0.25 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	10.96 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 5.5 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 9.7 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 13.3 AA estimate Qwen3.5 9B (Non-reasoning) · Arena unknown · Model page

Qwen3.5-4B

Qwen/Qwen3.5-4B

2026-03-024B dense

AA index13.1 AA estimaterank #48

Arena—

Best file & memory

MLX 8-bit4.8 GiB fileFits · conditional

6.0 of 144 GiB+8K ctx · max ≈256K

Prompt~1,687 t/sEstimated

Generate~128 t/sEstimated

First token · 4K prompt~2.4 sEstimated

Memory budget

Weights file	4.78 GiB	mlx-community/Qwen3.5-4B-8bit HF files
KV cache · 8K	0.25 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	6.03 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 2.8 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 4.8 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 13.1 AA estimate Qwen3.5 4B (Reasoning) · Arena unknown · Model page

NVIDIA Nemotron 3.5 Lightning 30B-A3B

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

2026-08-1130B · 3B active

AA index12.9rank #49

Arena1,329

Best file & memory

MLX 8-bit31.3 GiB fileFits · conditional

32.3 of 144 GiB+8K ctx · max ≈256K

Prompt~1,388 t/sEstimated

Generate~81.4 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	31.26 GiB	mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-8bit · 7 files HF files
KV cache · 8K	0.05 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	32.30 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 16.6 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.1 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 12.9 Nemotron 3.5 Lightning · Arena 1,329 · Model page

IBM Granite 4.2 30B

ibm-granite/granite-4.2-30b

2026-08-2530B dense

AA index12.7rank #50

Arena1,361

Best file & memory

MLX 8-bit29.0 GiB fileFits

32.0 of 144 GiB8K ctx · max ≈128K

Prompt~225 t/sEstimated

Generate~21.2 t/sEstimated

First token · 4K prompt~18 sEstimated

Memory budget

Weights file	28.97 GiB	lmstudio-community/granite-4.2-30b-MLX-8bit · 6 files HF files
KV cache · 8K	2.00 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	31.97 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Other MLX files

MLX 4-bit 15.3 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 29.0 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 12.7 Granite 4.2 30B · Arena 1,361 · Model page

MiniCPM5-2B

openbmb/MiniCPM5-2B

2026-09-072.5B dense

AA index12.5rank #51

Arena—

Best file & memory

MLX 8-bit2.5 GiB fileFits

3.8 of 144 GiB8K ctx · max ≈128K

Prompt~2,682 t/sEstimated

Generate~246 t/sEstimated

First token · 4K prompt~1.5 sEstimated

Memory budget

Weights file	2.49 GiB	mlx-community/MiniCPM5-2B-8bit HF files
KV cache · 8K	0.33 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	3.82 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Other MLX files

MLX 4-bit 1.3 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 12.5 MiniCPM5-2B · Arena unknown · Model page

Nemotron Cascade 2 30B A3B

nvidia/Nemotron-Cascade-2-30B-A3B

2026-03-1930B · 3B active

AA index11.7 AA estimaterank #53

Arena—

Best file & memory

MLX 8-bit31.3 GiB fileFits · conditional

32.3 of 144 GiB+8K ctx · max ≈256K

Prompt~1,388 t/sEstimated

Generate~81.4 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	31.26 GiB	mlx-community/Nemotron-Cascade-2-30B-A3B-8bit · 7 files HF files
KV cache · 8K	0.05 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	32.30 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 16.6 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.1 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 11.7 AA estimate Nemotron Cascade 2 30B A3B · Arena unknown · Model page

Granite 4.2 8B

ibm-granite/granite-4.2-8b

2026-08-258B dense

AA index11.1rank #56

Arena1,320

Best file & memory

MLX 8-bit8.7 GiB fileFits

10.9 of 144 GiB8K ctx · max ≈128K

Prompt~844 t/sEstimated

Generate~70.5 t/sEstimated

First token · 4K prompt~4.9 sEstimated

Memory budget

Weights file	8.70 GiB	lmstudio-community/granite-4.2-8b-MLX-8bit · 2 files HF files
KV cache · 8K	1.25 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	10.95 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Other MLX files

MLX 4-bit 4.6 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.7 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 11.1 Granite 4.2 8B · Arena 1,320 · Model page

G9v3-3B

AI9Stars/G9v3-3B

2026-07 (day unknown)3B dense

AA index10.8 AA estimaterank #58

Arena—

Best file & memory

MLX 4-bit1.6 GiB fileFits

3.0 of 144 GiB8K ctx · max ≈128K

Prompt~2,250 t/sEstimated

Generate~392 t/sEstimated

First token · 4K prompt~1.8 sEstimated

Memory budget

Weights file	1.57 GiB	cof139/G9v3-3B-mlx-4Bit HF files
KV cache · 8K	0.41 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	2.97 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.6 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 10.8 AA estimate G9v3-3B · Arena unknown · Model page

North Mini Code

CohereLabs/North-Mini-Code-1.0

2026-06-0930B · 3B active

AA index9.9rank #61

Arena—

Best file & memory

MLX 8-bit31.0 GiB fileFits

32.5 of 144 GiB8K ctx · max ≈250K

Prompt~1,388 t/sEstimated

Generate~82.1 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	30.98 GiB	aaronmwelborn/North-Mini-Code-1.0-8bit · 7 files HF files
KV cache · 8K	0.48 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	32.47 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈250K tokens (native 250K).

Other MLX files

MLX 4-bit 17.2 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.1 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 9.9 North Mini Code · Arena unknown · Model page

Qwen3-Coder-Next

Qwen/Qwen3-Coder-Next

2026-02-0280B · 3B active

AA index9.2rank #63

Arena—

Best file & memory

MLX 8-bit78.8 GiB fileFits · conditional

80.0 of 144 GiB+8K ctx · max ≈256K

Prompt~1,388 t/sEstimated

Generate~86.1 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	78.84 GiB	mlx-community/Qwen3-Coder-Next-8bit · 17 files HF files
KV cache · 8K	0.19 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	80.03 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 41.8 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 9.2 Qwen3 Coder Next · Arena unknown · Model page

Granite 4.2 3B

ibm-granite/granite-4.2-3b

2026-08-253B dense

AA index9.1rank #64

Arena1,293

Best file & memory

MLX 8-bit3.6 GiB fileFits

5.2 of 144 GiB8K ctx · max ≈128K

Prompt~2,250 t/sEstimated

Generate~169 t/sEstimated

First token · 4K prompt~1.8 sEstimated

Memory budget

Weights file	3.62 GiB	lmstudio-community/granite-4.2-3b-MLX-8bit HF files
KV cache · 8K	0.63 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	5.25 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Other MLX files

MLX 4-bit 1.9 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.6 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 9.1 Granite 4.2 3B · Arena 1,293 · Model page

MiniCPM5-1B

openbmb/MiniCPM5-1B

2026-05-191.1B dense

AA index8.8 AA estimaterank #67

Arena—

Best file & memory

MLX 8-bit1.1 GiB fileFits

2.3 of 144 GiB8K ctx · max ≈128K

Prompt~6,245 t/sEstimated

Generate~573 t/sEstimated

First token · 4K prompt~656 msEstimated

Memory budget

Weights file	1.07 GiB	mlx-community/MiniCPM5-1B-8bit HF files
KV cache · 8K	0.19 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	2.26 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Other MLX files

MLX 4-bit 0.6 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.1 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 8.8 AA estimate MiniCPM5-1B (Reasoning) · Arena unknown · Model page

Nanbeige4.1-3B

Nanbeige/Nanbeige4.1-3B

2026-02 (day unknown)3B dense

AA index8.4 AA estimaterank #69

Arena—

Best file & memory

MLX 8-bit3.9 GiB fileFits

5.4 of 144 GiB8K ctx · max ≈256K

Prompt~2,250 t/sEstimated

Generate~158 t/sEstimated

First token · 4K prompt~1.8 sEstimated

Memory budget

Weights file	3.89 GiB	mlx-community/Nanbeige4.1-3B-8bit HF files
KV cache · 8K	0.50 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	5.39 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 2.1 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.9 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 8.4 AA estimate Nanbeige4.1-3B · Arena unknown · Model page

LFM2.5-2.6B

LiquidAI/LFM2.5-2.6B

2026-08-042.7B dense

AA index8.4 AA estimaterank #70

Arena—

Best file & memory

MLX 8-bit2.7 GiB fileFits · conditional

3.8 of 144 GiB+8K ctx · max ≈128K

Prompt~2,509 t/sEstimated

Generate~230 t/sEstimated

First token · 4K prompt~1.6 sEstimated

Memory budget

Weights file	2.67 GiB	LiquidAI/LFM2.5-2.6B-MLX-8bit HF files
KV cache · 8K	0.13 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	3.79 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Other MLX files

MLX 4-bit 1.5 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.7 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 8.4 AA estimate LFM2.5-2.6B · Arena unknown · Model page

Falcon-H1R-7B

tiiuae/Falcon-H1R-7B

2026-01-057B dense

AA index7.8 AA estimaterank #71

Arena—

Best file & memory

MLX 8-bit7.5 GiB fileFits · conditional

8.9 of 144 GiB+8K ctx · max ≈256K

Prompt~964 t/sEstimated

Generate~81.7 t/sEstimated

First token · 4K prompt~4.2 sEstimated

Memory budget

Weights file	7.51 GiB	mlx-community/Falcon-H1R-7B-8bit · 2 files HF files
KV cache · 8K	0.34 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	8.85 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 4.0 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 7.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 7.8 AA estimate Falcon-H1R-7B · Arena unknown · Model page

LFM2.5-8B-A1B

LiquidAI/LFM2.5-8B-A1B

2026-05-288.3B · 1.5B active

AA index7.2 AA estimaterank #74

Arena—

Best file & memory

MLX 8-bit8.4 GiB fileFits · conditional

9.5 of 144 GiB+8K ctx · max ≈125K

Prompt~2,776 t/sEstimated

Generate~168 t/sEstimated

First token · 4K prompt~1.5 sEstimated

Memory budget

Weights file	8.38 GiB	LiquidAI/LFM2.5-8B-A1B-MLX-8bit · 2 files HF files
KV cache · 8K	0.09 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	9.48 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈125K tokens (native 125K).

Other MLX files

MLX 4-bit 4.5 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 7.2 AA estimate LFM2.5-8B-A1B · Arena unknown · Model page

NVIDIA Nemotron 3 Nano 4B

nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16

2026-03-164B dense

AA index4.8rank #75

Arena—

Best file & memory

MLX 8-bit3.9 GiB fileFits · conditional

5.1 of 144 GiB+8K ctx · max ≈256K

Prompt~1,700 t/sEstimated

Generate~156 t/sEstimated

First token · 4K prompt~2.4 sEstimated

Memory budget

Weights file	3.93 GiB	mlx-community/NVIDIA-Nemotron-3-Nano-4B-8bit HF files
KV cache · 8K	0.13 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	5.06 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 2.1 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.9 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 4.8 NVIDIA Nemotron 3 Nano 4B · Arena unknown · Model page

LFM2.5-24B-A2B

LiquidAI/LFM2.5-24B-A2B

2026-10-1324B · 2B active

AA index—unranked

Arena—

Best file & memory

MLX 8-bit23.7 GiB fileFits · conditional

24.9 of 144 GiB+8K ctx · max ≈128K

Prompt~2,082 t/sEstimated

Generate~129 t/sEstimated

First token · 4K prompt~2.0 sEstimated

Memory budget

Weights file	23.73 GiB	LiquidAI/LFM2.5-24B-A2B-MLX-8bit · 5 files HF files
KV cache · 8K	0.16 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	24.89 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Other MLX files

MLX 4-bit 12.6 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 2.0 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

MiniCPM-V 4.7 35B-A3B

openbmb/MiniCPM-V-4.7-35B-A3B

2026-10-1235B · 3B active

AA index—unranked

Arena—

Best file & memory

MLX 8-bit37.1 GiB fileFits · conditional

38.3 of 144 GiB+8K ctx · max ≈256K

Prompt~1,388 t/sEstimated

Generate~80 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	37.10 GiB	mlx-community/MiniCPM-V-4.7-35B-A3B-8bit · 8 files HF files
KV cache · 8K	0.16 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	38.25 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 21.9 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.2 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

Aleph Alpha Kolibri-1

Aleph-Alpha/Kolibri-1

2026-10-0378B · 3.5B active

AA index—unranked

Arena—

Best file & memory

MLX 4-bit41.3 GiB fileFits · conditional

42.5 of 144 GiB+8K ctx · max ≈256K

Prompt~1,203 t/sEstimated

Generate~139 t/sEstimated

First token · 4K prompt~3.4 sEstimated

Memory budget

Weights file	41.29 GiB	velaia/Kolibri-1-MLX-4bit · 9 files HF files
KV cache · 8K	0.20 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	42.48 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 1.8 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

MiMo-V2.6-Distill-Qwen-9B

XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B

2026-09-229B dense

AA index—unranked

Arena—

Best file & memory

MLX 8-bit9.7 GiB fileFits · conditional

11.0 of 144 GiB+8K ctx · max ≈256K

Prompt~750 t/sEstimated

Generate~63.2 t/sEstimated

First token · 4K prompt~5.5 sEstimated

Memory budget

Weights file	9.71 GiB	nativ-community/MiMo-V2.6-Distill-Qwen-9B-MLX-8bit · 2 files HF files
KV cache · 8K	0.25 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	10.96 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 5.5 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 9.7 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

K2-Horizon-0.9B

IFM/K2-Horizon-0.9B

2026-09-03size unknown

AA index—unranked

Arena—

Best file & memory

MLX 8-bit1.1 GiB fileFits

2.5 of 144 GiB8K ctx · max ≈128K

Prompt—Unknown

Generate~575 t/sEstimated

First token · 4K prompt—Unknown

Memory budget

Weights file	1.07 GiB	mlx-community/K2-Horizon-0.9B-8bit HF files
KV cache · 8K	0.44 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Accounted	2.50 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈128K tokens (native 128K).

Other MLX files

MLX 4-bit 0.8 GiB · fits · files

Where the speeds come from

Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.1 GiB read per token.
First token unknown: needs a prefill value.

Caveats

Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

Ornith-1.5-9B

ornith-ai/Ornith-1.5-9B

2026-08 (day unknown)9B dense

AA index—unranked

Arena—

Best file & memory

MLX 8-bit8.9 GiB fileFits · conditional

10.1 of 144 GiB+8K ctx · max ≈256K

Prompt~750 t/sEstimated

Generate~69.2 t/sEstimated

First token · 4K prompt~5.5 sEstimated

Memory budget

Weights file	8.86 GiB	ornith-ai/Ornith-1.5-9B-MLX-8bit · 2 files HF files
KV cache · 8K	0.25 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	10.11 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 4.7 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.9 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

Poolside Laguna S 2.1

poolside/Laguna-S-2.1

2026-07-21118B · 8B active

AA index—unranked

Arena—

Best file & memory

MLX 8-bit116 GiB fileFits · conditional

118 of 144 GiB+8K ctx · max ≈557K

Prompt~521 t/sEstimated

Generate~32.3 t/sEstimated

First token · 4K prompt~7.9 sEstimated

Memory budget

Weights file	116 GiB	TensorFold/Laguna-S-2.1-MLX-8bit · 24 files HF files
KV cache · 8K	0.45 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	118 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈557K tokens (native 1M).

Other MLX files

MLX 4-bit 61.6 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 7.9 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

Qwen3.6-27B

Qwen/Qwen3.6-27B

2026-04-2227B dense

AA index—unranked

Arena—

Best file & memory

MLX 8-bit27.5 GiB fileFits · conditional

29.0 of 144 GiB+8K ctx · max ≈256K

Prompt~250 t/sEstimated

Generate~22.3 t/sEstimated

First token · 4K prompt~16 sEstimated

Memory budget

Weights file	27.48 GiB	mlx-community/Qwen3.6-27B-8bit · 6 files HF files
KV cache · 8K	0.50 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	28.98 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 15.0 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 27.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

NVIDIA Nemotron 3 Super 120B-A12B

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16

2026-03-11120B · 12B active

AA index—unranked

Arena—

Best file & memory

MLX 8-bit119 GiB fileFits · conditional

121 of 144 GiB+8K ctx · max ≈256K

Prompt~347 t/sEstimated

Generate~21.3 t/sEstimated

First token · 4K prompt~12 sEstimated

Memory budget

Weights file	119 GiB	mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-8bit · 27 files HF files
KV cache · 8K	0.06 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	121 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 63.3 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 11.9 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Sliding-window allocation is not established.
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

Qwen3.5-0.8B

Qwen/Qwen3.5-0.8B

2026-03-020.8B dense

AA index—unranked

Arena—

Best file & memory

MLX 8-bit0.9 GiB fileFits · conditional

2.0 of 144 GiB+8K ctx · max ≈256K

Prompt~8,436 t/sEstimated

Generate~658 t/sEstimated

First token · 4K prompt~486 msEstimated

Memory budget

Weights file	0.93 GiB	mlx-community/Qwen3.5-0.8B-8bit HF files
KV cache · 8K	0.09 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	2.03 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 0.6 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 0.9 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

Qwen3.5-2B

Qwen/Qwen3.5-2B

2026-03-022B dense

AA index—unranked

Arena—

Best file & memory

MLX 8-bit2.5 GiB fileFits · conditional

3.6 of 144 GiB+8K ctx · max ≈256K

Prompt~3,374 t/sEstimated

Generate~247 t/sEstimated

First token · 4K prompt~1.2 sEstimated

Memory budget

Weights file	2.48 GiB	mlx-community/Qwen3.5-2B-8bit HF files
KV cache · 8K	0.09 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	3.57 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 1.6 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

Qwen3.5-27B

Qwen/Qwen3.5-27B

2026-02-2427B dense

AA index—unranked

Arena—

Best file & memory

MLX 8-bit27.5 GiB fileFits · conditional

29.0 of 144 GiB+8K ctx · max ≈256K

Prompt~250 t/sEstimated

Generate~22.3 t/sEstimated

First token · 4K prompt~16 sEstimated

Memory budget

Weights file	27.48 GiB	mlx-community/Qwen3.5-27B-8bit · 6 files HF files
KV cache · 8K	0.50 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	28.98 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 15.0 GiB · fits · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 27.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

Qwen3.5-35B-A3B

Qwen/Qwen3.5-35B-A3B

2026-02-2435B · 3B active

AA index—unranked

Arena—

Best file & memory

MLX 8-bit35.1 GiB fileFits · conditional

36.3 of 144 GiB+8K ctx · max ≈256K

Prompt~1,388 t/sEstimated

Generate~84.5 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	35.13 GiB	mlx-community/Qwen3.5-35B-A3B-8bit · 8 files HF files
KV cache · 8K	0.16 GiB	FP16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	36.29 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈256K tokens (native 256K).

Other MLX files

MLX 4-bit 19.0 GiB · fits · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

MiniMax-M2.7

MiniMaxAI/MiniMax-M2.7

2026 (month unknown)MoE · size unknown

AA index—unranked

Arena—

Best file & memory

MLX 4-bit120 GiB fileFits · conditional

123 of 144 GiB+8K ctx · max ≈94K

Prompt—Unknown

Generate—Unknown

First token · 4K prompt—Unknown

Memory budget

Weights file	120 GiB	mlx-community/MiniMax-M2.7-4bit · 27 files HF files
KV cache · 8K	1.94 GiB	BF16 cache from config.json Estimated
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	123 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: ≈94K tokens (native 200K).

Other MLX files

MLX 8-bit 226 GiB · too big · files

Where the speeds come from

Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
Generate unknown: no measured report, and bandwidth, active size or backend calibration is missing.
First token unknown: needs a prefill value.

Caveats

Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

May fit: KV-cache size unknown 14

Weights and runtime fit, but the cache for this attention type is not established. Ranked by AA index.

Qwen3.8-Flash-Next

Qwen/Qwen3.8-Flash-Next

2026-08-26180B · 6B active

AA index39.8rank #8

Arena—

Best file & memory

MLX 4-bit104 GiB fileCache unknown

105 of 144 GiB+8K ctx

Prompt~694 t/sEstimated

Generate~73.5 t/sEstimated

First token · 4K prompt~5.9 sEstimated

Memory budget

Weights file	104 GiB	mlx-community/Qwen3.8-Flash-Next-4bit · 22 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	105 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 256K).

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 39.8 Qwen3.8-Flash-Next · Arena unknown · Model page

K2-Horizon-MoVA-36B-A4B

IFM/K2-Horizon-MoVA-36B-A4B

2026-09-0336B · 4B active

AA index25.3rank #22

Arena—

Best file & memory

MLX 8-bit37.1 GiB fileCache unknown

38.1 of 144 GiB+8K ctx

Prompt~1,041 t/sEstimated

Generate~61.8 t/sEstimated

First token · 4K prompt~3.9 sEstimated

Memory budget

Weights file	37.07 GiB	mlx-community/K2-Horizon-MoVA-36B-A4B-8bit · 8 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	38.07 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 512K).

Other MLX files

MLX 4-bit 24.6 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 4.1 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 25.3 K2 Horizon MoVA 36B A4B · Arena unknown · Model page

Gemma 4 26B A4B (it)

google/gemma-4-26B-A4B-it

2026-04-0225B · 3.8B active

AA index16.7 AA estimaterank #39

Arena1,434

Best file & memory

MLX 8-bit26.0 GiB fileCache unknown

27.0 of 144 GiB+8K ctx

Prompt~1,096 t/sEstimated

Generate~64.8 t/sEstimated

First token · 4K prompt~3.7 sEstimated

Memory budget

Weights file	26.03 GiB	mlx-community/gemma-4-26B-A4B-it-8bit · 6 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	27.03 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 256K).

Other MLX files

MLX 4-bit 14.3 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.9 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 16.7 AA estimate Gemma 4 26B A4B (Reasoning) · Arena 1,434 · Model page

Gemma 4 31B (it)

google/gemma-4-31B-it

2026-04-0231B dense

AA index14.7rank #42

Arena1,443

Best file & memory

MLX 8-bit31.4 GiB fileCache unknown

32.4 of 144 GiB+8K ctx

Prompt~220 t/sEstimated

Generate~19.5 t/sEstimated

First token · 4K prompt~19 sEstimated

Memory budget

Weights file	31.44 GiB	mlx-community/gemma-4-31B-it-8bit · 7 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	32.44 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 256K).

Other MLX files

MLX 4-bit 17.1 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 31.4 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 14.7 Gemma 4 31B (Reasoning) · Arena 1,443 · Model page

Gemma 4 12B (it)

google/gemma-4-12B-it

2026-06-0312B dense

AA index14.2 AA estimaterank #44

Arena—

Best file & memory

MLX 8-bit11.8 GiB fileCache unknown

12.8 of 144 GiB+8K ctx

Prompt~565 t/sEstimated

Generate~51.8 t/sEstimated

First token · 4K prompt~7.3 sEstimated

Memory budget

Weights file	11.84 GiB	mlx-community/gemma-4-12B-it-8bit · 3 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	12.84 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 256K).

Other MLX files

MLX 4-bit 6.3 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 11.8 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 14.2 AA estimate Gemma 4 12B (Reasoning) · Arena unknown · Model page

LongCat-Flash-Lite

meituan-longcat/LongCat-Flash-Lite

2026-01 (day unknown)69B · 3B active

AA index11.5 AA estimaterank #54

Arena—

Best file & memory

MLX 8-bit67.8 GiB fileCache unknown

68.8 of 144 GiB+8K ctx

Prompt~1,388 t/sEstimated

Generate~85.7 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	67.84 GiB	mlx-community/LongCat-Flash-Lite-8bit · 20 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	68.84 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 320K).

Other MLX files

MLX 4-bit 35.9 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 11.5 AA estimate LongCat Flash Lite · Arena unknown · Model page

Ling-3.0-tiny

inclusionAI/Ling-3.0-tiny

2026-08 (day unknown)7.9B · 1.3B active

AA index11.1rank #57

Arena—

Best file & memory

MLX 4-bit4.1 GiB fileCache unknown

5.1 of 144 GiB+8K ctx

Prompt~3,203 t/sEstimated

Generate~373 t/sEstimated

First token · 4K prompt~1.3 sEstimated

Memory budget

Weights file	4.14 GiB	rapid-mlx/Ling-3.0-tiny-MLX-4bit HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	5.14 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 128K).

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 0.7 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 11.1 Ling 3.0 Tiny · Arena unknown · Model page

Nemotron 3 Nano Omni 30B A3B Reasoning

nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16

2026-04-2830B · 3B active

AA index10.3 AA estimaterank #60

Arena—

Best file & memory

MLX 8-bit33.3 GiB fileCache unknown

34.3 of 144 GiB+8K ctx

Prompt~1,388 t/sEstimated

Generate~76.4 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	33.32 GiB	mlx-community/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-8bit · 8 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	34.32 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 256K).

Other MLX files

MLX 4-bit 18.3 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.3 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 10.3 AA estimate Nemotron 3 Nano Omni 30B A3B Reasoning · Arena unknown · Model page

DiffusionGemma 26B A4B

google/diffusiongemma-26B-A4B-it

2026-06-1025B · 3.8B active

AA index9.5 AA estimaterank #62

Arena—

Best file & memory

MLX 8-bit26.1 GiB fileCache unknown

27.1 of 144 GiB+8K ctx

Prompt~1,096 t/sEstimated

Generate~64.8 t/sEstimated

First token · 4K prompt~3.7 sEstimated

Memory budget

Weights file	26.05 GiB	mlx-community/diffusiongemma-26B-A4B-it-8bit · 6 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	27.05 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 256K).

Other MLX files

MLX 4-bit 15.4 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.9 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 9.5 AA estimate DiffusionGemma 26B A4B · Arena unknown · Model page

Gemma 4 E4B (it)

google/gemma-4-E4B-it

2026-04-028B dense

AA index8.9rank #66

Arena—

Best file & memory

MLX 8-bit8.3 GiB fileCache unknown

9.3 of 144 GiB+8K ctx

Prompt~844 t/sEstimated

Generate~74.1 t/sEstimated

First token · 4K prompt~4.9 sEstimated

Memory budget

Weights file	8.27 GiB	mlx-community/gemma-4-E4B-it-8bit · 2 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	9.27 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 128K).

Other MLX files

MLX 4-bit 4.8 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 8.3 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 8.9 Gemma 4 E4B (Reasoning) · Arena unknown · Model page

Gemma 4 E2B

google/gemma-4-E2B-it

2026-04-025.1B dense

AA index7.8 AA estimaterank #72

Arena—

Best file & memory

MLX 8-bit5.5 GiB fileCache unknown

6.5 of 144 GiB+8K ctx

Prompt~1,323 t/sEstimated

Generate~112 t/sEstimated

First token · 4K prompt~3.1 sEstimated

Memory budget

Weights file	5.46 GiB	mlx-community/gemma-4-E2B-it-8bit · 2 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	6.46 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 128K).

Other MLX files

MLX 4-bit 3.3 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median prefill rate per GPU core of 11 measured M2 points (MLX) × this chip's GPU cores ÷ (2 × active params).
Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 5.5 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA 7.8 AA estimate Gemma 4 E2B (Reasoning) · Arena unknown · Model page

Ling-3.0-flash

inclusionAI/Ling-3.0-flash

2026-08-07124B · 5.1B active

AA index—unranked

Arena—

Best file & memory

MLX 8-bit123 GiB fileCache unknown

124 of 144 GiB+8K ctx

Prompt~816 t/sEstimated

Generate~50.3 t/sEstimated

First token · 4K prompt~5.0 sEstimated

Memory budget

Weights file	123 GiB	TensorFold/Ling-3.0-flash-MLX-8bit · 27 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	124 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 256K).

Other MLX files

MLX 4-bit 65.2 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 5.1 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

Step 3.7 Flash

stepfun-ai/Step-3.7-Flash

2026-05-29198B · 11B active

AA index—unranked

Arena—

Best file & memory

MLX 4-bit103 GiB fileCache unknown

104 of 144 GiB+8K ctx

Prompt~379 t/sEstimated

Generate~44.4 t/sEstimated

First token · 4K prompt~11 sEstimated

Memory budget

Weights file	103 GiB	mlx-community/Step-3.7-Flash-4bit · 22 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	104 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 256K).

Other MLX files

MLX 8-bit 195 GiB · too big · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 5.7 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

GLM-4.7-Flash

zai-org/GLM-4.7-Flash

2026-01 (day unknown)30B · 3B active

AA index—unranked

Arena—

Best file & memory

MLX 8-bit29.6 GiB fileCache unknown

30.6 of 144 GiB+8K ctx

Prompt~1,388 t/sEstimated

Generate~85.9 t/sEstimated

First token · 4K prompt~3.0 sEstimated

Memory budget

Weights file	29.64 GiB	mlx-community/GLM-4.7-Flash-8bit · 7 files HF files
KV cache · 8K	—	Unknown
Runtime allowance	1.00 GiB	planning assumption Estimated
Recurrent state	—	Unknown
Accounted	30.64 GiB	of 144 GiB usable (75% of 192 GB) · 0.5 GiB spare margin required

Max context in free memory: unknown (native 198K).

Other MLX files

MLX 4-bit 15.7 GiB · cache unknown · files

Where the speeds come from

Prompt estimated from median of 3 measured prefill points for this device (MLX), rescaled to this model.
Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 3.0 GiB read per token.
First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

Caveats

KV-cache size could not be established from the config (compressed or backend-specific attention).
Fixed recurrent/convolution state is unknown and not included.
Text-only budget. A downloadable file does not prove your runtime supports this architecture.

AA index unknown · Arena unknown · Model page

15 models too big for M2 Ultra 76c GPU · 192 GB

GLM-5.3-FlashMLX 4-bit needs 191 GiB+ (KV unknown)
Muse Spark Open (816B-A42B)MLX 4-bit needs 498 GiB+ (KV unknown)
Motif 3MLX 8-bit needs 313 GiB+ (KV unknown)
MiniMax-M3MLX 4-bit needs 226 GiB+ (KV unknown)
Nex-N2-Pro (Qwen3.5-397B-A17B based)MLX 8-bit needs 394 GiB
Kimi K2.7 CodeMLX 4-bit needs 598 GiB+ (KV unknown)
Inkling-SmallMLX 4-bit needs 145 GiB
InklingMLX 4-bit needs 513 GiB
NVIDIA Nemotron 3 Ultra 550B-A55BMLX 4-bit needs 290 GiB
Qwen3.5-397B-A17BMLX 4-bit needs 210 GiB
Trinity-Large-ThinkingMLX 4-bit needs 212 GiB
Hy4-previewMLX 4-bit needs 405 GiB+ (KV unknown)
DeepSeek V4 Flash 0731MLX 4-bit needs 145 GiB+ (KV unknown)
GLM-5.2MLX 4-bit needs 391 GiB+ (KV unknown)
Kimi K2.6MLX 4-bit needs 614 GiB+ (KV unknown)

Specs & sources

Memory	192 GB unified	source
Bandwidth	800 GB/s	source
FP16	unknown	Apple publishes no comparable FP16 throughput or GPU-only TDP in reviewed chip/device technical specifications. Power adapter rating and whole-system maximum continuous power are not GPU TDP.
Power	unknown	Apple publishes no comparable FP16 throughput or GPU-only TDP in reviewed chip/device technical specifications. Power adapter rating and whole-system maximum continuous power are not GPU TDP.

Found in: Mac Studio (2023)

Measured speeds

27 reported observations

On this device (same chip and GPU-core count). Reported by the linked sources; not reproduced by LocalFit.

Open in speed database Submit yours

Doesn't fit? Rent a GPU

RunpodPlaceholderCloud GPU pods you can rent for running larger models.
Vast.aiPlaceholderGPU rental marketplace with many hosts and GPU types.
LambdaPlaceholderCloud GPU instances for AI training and inference.
TensorDockPlaceholderGPU cloud marketplace with consumer and datacenter GPUs.

Sponsored placeholder links: LocalFit may earn a commission if these become affiliate links. They never influence fit results, estimates or picks.