NVIDIA B300 SXM: what runs best?

RTX 4090 RTX 5090 RTX 3090 RTX 4060 Ti (16 GB)RTX 5070 Ti Radeon RX 7900 XTX M4 Max 128 GB M3 Ultra 512 GB DGX Spark (128 GB)All devices

NVIDIA · GPU

NVIDIA B300 SXM

Memory288 GB HBM3eBandwidth8,000 GB/s vendor specPlussystem RAM for offload (set below)

GPUs

B300 SXM288 GB · 8,000 GB/s1

Custom GPU

System RAM

InstalledGBTypeSpeedMT/sChannels× 64-bit

≈ 89.6 GB/s theoretical ceiling (MT/s × 8 bytes × channels). DDR5’s two 32-bit subchannels count as one channel.

CPU

PlatformThreads for inference

Used for CPU layers/experts and the -t flag. A reference platform fills its maximum channels and speed; set what you actually populated.

Advanced: reserves, bandwidth, workload, models

Available VRAM per GPUGiB Runtime allowance per deviceGiB Spare margin per deviceGiB Unified memory for models% RAM kept for OS and appsGiB Measured RAM bandwidthGB/s GPU linkLanes Prompt length (first token)tokens Custom contexttokens Parallel sequences

Allow disk-streamed experts (documented cases)Include base checkpointsInclude specialistsInclude community & quant-trained variantsInclude models with unverified release dates

Defaults are editorial planning choices, not measurements: 1 GiB runtime + 0.5 GiB margin per device, 75% of unified memory, 8 GiB RAM for the OS, a vendor “24 GB” treated as 24 GiB (enter your driver’s available VRAM to override), 5 t/s minimum speed (research editorial default). Memory ledger

B300 SXM64 GB DDR5-5600 ×2 ch · 56.0 GiB usable · 89.6 GB/s theoretical8K contextF16 KVany backend≥ 5 t/s

Best plans B300 SXM

Best quality at ≥ 5 t/s

Qwen3.8-Flash-Next

39.8 AA index single mode

Q8_0llama.cpp CUDAFull GPU

~274–791 t/s Estimated extrapolated

Speed extrapolated, not measured on this setup.

Faster tradeoff

K2-Horizon-MoVA-36B-A4B

25.3 AA index single mode

Q8_0llama.cpp CUDAFull GPU

~289–835 t/s Estimated extrapolated

Next point on the quality/speed front; then Qwen3.6-35B-A3B.

Possibly better · memory not established

GLM-5.3

44.8 AA index Max

EXL3ExLlamaV3Full GPU

≤ 597 t/s Ceiling

Only a bandwidth ceiling (≤ 597 t/s) is known: this backend or placement has no calibration. KV memory not established. Also: GLM-5.3-Flash, Ling-3.1-flash.

WeightsKV cacheReserveFreeFits* = a state term unsizedMeasuredlinks to its sourceEstimatedcalibrated or extrapolated range, ~Ceilingupper bound, ≤Unknownnever zero

Meets your 5 t/s floor 48

Best plan per model that reaches your floor by a matching measurement, a calibrated range, or an extrapolated estimate (labelled). Sorted by the selected quality metric, then speed.

1
Qwen3.8-Flash-Next
Qwen/Qwen3.8-Flash-Next
2026-08-26180B MoE · 6B active
AA index39.8single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
153 / 288 GiB
Generate~274–791 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
2
Qwen3.8-27B
Qwen/Qwen3.8-27B
2026-08-1427B dense
AA index33.7Xhigh
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
28.6 / 288 GiB
Generate~158–228 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
3
Beam-1.0-501B-A23B
reflection/Beam-1.0-501B-A23B
2026-10-16501B MoE · 23B active
AA index30.3single mode
Plan & memory
Q5_K_Mllama.cpp CUDAGPU + CPU layers · 44/52 layers on GPUFits
GPU
282 / 288 GiB
RAM
51.0 / 56.0 GiB
Generate~7.3–9.0 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
4
MiniMax-M3
MiniMaxAI/MiniMax-M3
2026-06 (day unknown)428B MoE · 23B active
AA index29.2single mode
Plan & memory
Q4_K_Mllama.cpp CUDAFull GPUFits
GPU
249 / 288 GiB
Generate~103–296 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
5
Nex-N2-Pro
nex-agi/Nex-N2-Pro
2026-06 (day unknown)397B MoE · 17B active
AA index28.2 est.Based on Qwen3.5-3
Plan & memory
Q5_K_Mllama.cpp CUDAFull GPUFits*
GPU
266 / 288 GiB
Generate~123–356 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
6
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16Specialist
nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16
2026-10-15120B MoE · 12B active
AA index26.5single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
121 / 288 GiB
Generate~117–339 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
7
Inkling-Small
thinkingmachines/Inkling-Small
2026-07-30276B MoE · 12B active
AA index25.7single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
264 / 288 GiB
Generate~117–339 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
8
K2-Horizon-MoVA-36B-A4B
IFM/K2-Horizon-MoVA-36B-A4B
2026-09-0336B MoE · 4B active
AA index25.3single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
40.1 / 288 GiB
Generate~289–835 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
9
Hy3
tencent/Hy3
2026-07-06295B MoE · 21B active
AA index25.3single mode
Plan & memory
Q4_K_Mllama.cpp CUDAFull GPUFits
GPU
174 / 288 GiB
Generate~106–305 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
10
NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16
nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16
2026-10-1574B MoE · 9B active
AA index21.4single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
75.0 / 288 GiB
Generate~157–454 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
11
K2-Horizon-7B
IFM/K2-Horizon-7B
2026-09-039B tensors · dense
AA index20.6single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
11.5 / 288 GiB
Generate~447–647 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
12
Qwen3.5-397B-A17B
Qwen/Qwen3.5-397B-A17B
2026-02-16397B MoE · 17B active
AA index18.4Reasoning
Plan & memory
Q5_K_Mllama.cpp CUDAFull GPUFits*
GPU
275 / 288 GiB
Generate~119–344 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
13
Qwen3.6-35B-A3B
Qwen/Qwen3.6-35B-A3B
2026-04-1635B MoE · 3B active
AA index18.2Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
36.0 / 288 GiB
Generate~465–1,345 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
14
Muse-Glimmer-30B
meta-models/Muse-Glimmer-30B
2026-08-1030B dense
AA index17.5High
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
29.3 / 288 GiB
Generate~153–221 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
15
K2-Horizon-3.7B
IFM/K2-Horizon-3.7B
2026-09-035.1B tensors · dense
AA index15.6single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
7.6 / 288 GiB
Generate~760–1,100 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
16
Qwen3.5-122B-A10B
Qwen/Qwen3.5-122B-A10B
2026-02-24122B MoE · 10B active
AA index15.6Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
125 / 288 GiB
Generate~138–398 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
17
Mistral-Medium-3.5-128B
mistralai/Mistral-Medium-3.5-128B
2026-04-28128B dense
AA index14.2single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
128 / 288 GiB
Generate~34–49 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
18
EXAONE-4.5-33B
LGAI-EXAONE/EXAONE-4.5-33B
2026-04-0933B dense
AA index13.2 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
35.5 / 288 GiB
Generate~126–182 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
19
Qwen3.5-4B
Qwen/Qwen3.5-4B
2026-03-024B dense
AA index13.1 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
6.1 / 288 GiB
Generate~957–1,385 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
20
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
2026-08-1130B MoE · 3B active
AA index12.9single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
32.8 / 288 GiB
Generate~447–1,291 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
21
granite-4.2-30b
ibm-granite/granite-4.2-30b
2026-08-2530B dense
AA index12.7single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
32.5 / 288 GiB
Generate~141–205 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
22
MiniCPM5-2B
openbmb/MiniCPM5-2B
2026-09-072.5B dense
AA index12.5single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits
GPU
6.5 / 288 GiB
Generate~873–1,263 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
23
Hypernova-60B-2605
MultiverseComputingCAI/Hypernova-60B-2605
2026-05-0660B MoE · 4.8B active
Also: HyperNova-60B (listed with its dated update); Hypernova-60B-2602 (dated update, not separately evaluated)
AA index11.7 est.High
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
59.9 / 288 GiB
Generate~294–851 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
24
Nemotron-Cascade-2-30B-A3B
nvidia/Nemotron-Cascade-2-30B-A3B
2026-03-1930B MoE · 3B active
AA index11.7 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
32.8 / 288 GiB
Generate~447–1,291 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
25
Qwen3.5-9B
Qwen/Qwen3.5-9B
2026-03-029B dense
AA index11.2Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
10.9 / 288 GiB
Generate~458–663 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
26
granite-4.2-8b
ibm-granite/granite-4.2-8b
2026-08-258B dense
AA index11.1single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
11.5 / 288 GiB
Generate~454–658 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
27
G9v3-3B
ai9stars/G9v3-3B
2026-07 (day unknown)3B dense
AA index10.8 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits
GPU
7.5 / 288 GiB
Generate~734–1,062 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
28
Trinity-Large-Thinking
arcee-ai/Trinity-Large-Thinking
2026-04-01398B MoE · 13B active
AA index10.8single mode
Plan & memory
Q5_K_Mllama.cpp CUDAFull GPUFits
GPU
267 / 288 GiB
Generate~147–425 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
29
granite-4.2-3b
ibm-granite/granite-4.2-3b
2026-08-253B dense
AA index9.1single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
5.8 / 288 GiB
Generate~1,077–1,558 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
30
Tri-21B-Think
trillionlabs/Tri-21B-Think
2026-02 (day unknown)21B dense
AA index9.0 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
23.6 / 288 GiB
Generate~199–288 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
31
LFM2.5-2.6B
LiquidAI/LFM2.5-2.6B
2026-08-042.7B dense
AA index8.4 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits*
GPU
6.7 / 288 GiB
Generate~832–1,204 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
32
LFM2.5-8B-A1B
LiquidAI/LFM2.5-8B-A1B
2026-05-288.3B MoE · 1.5B active
AA index7.2 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits*
GPU
17.4 / 288 GiB
Generate~485–1,403 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
33
Olmo-3.1-32B-ThinkSpecialist
allenai/Olmo-3.1-32B-Think
2025-12-1232B tensors · dense
AA index7.1 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
34.7 / 288 GiB
Generate~129–186 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
34
Qwen3.5-2B
Qwen/Qwen3.5-2B
2026-03-022B dense
AA index6.9 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
3.5 / 288 GiB
Generate~2,137–3,092 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
35
Olmo-3.1-32B-Instruct
allenai/Olmo-3.1-32B-Instruct
2025-12-1232B tensors · dense
AA index6.5 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
34.7 / 288 GiB
Generate~129–186 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
36
Qwen3.5-0.8B
Qwen/Qwen3.5-0.8B
2026-03-020.8B dense
AA index6.1 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
2.4 / 288 GiB
Generate~5,150–7,452 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
37
granite-4.0-h-small
ibm-granite/granite-4.0-h-small
2025-10-0232B tensors · MoE
AA index6.0 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits*
GPU
61.6 / 288 GiB
Generate~85–247 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
38
Phi-4Older
microsoft/phi-4
2024-12-1215B tensors · dense
AA index5.9 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits
GPU
30.4 / 288 GiB
Generate~151–218 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
39
Olmo-3-7B-ThinkSpecialist
allenai/Olmo-3-7B-Think
2025-11-207.3B tensors · dense
AA index5.6 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
11.2 / 288 GiB
Generate~459–665 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
40
Olmo-3-7B-Instruct
allenai/Olmo-3-7B-Instruct
2025-11-207.3B tensors · dense
AA index5.2 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
11.2 / 288 GiB
Generate~459–665 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
41
LFM2.5-1.2B-ThinkingSpecialist
LiquidAI/LFM2.5-1.2B-Thinking
2026-01-201.2B tensors · dense
AA index5.2 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits*
GPU
3.8 / 288 GiB
Generate~1,902–2,752 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
42
LFM2.5-1.2B-Instruct
LiquidAI/LFM2.5-1.2B-Instruct
2026-01-051.2B tensors · dense
AA index5.2 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits*
GPU
3.8 / 288 GiB
Generate~1,902–2,752 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
43
Apertus-70B-Instruct-2509
swiss-ai/Apertus-70B-Instruct-2509
2025-09-0271B tensors · dense
AA index5.1 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
73.9 / 288 GiB
Generate~60–86 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
44
Molmo2-8BSpecialist
allenai/Molmo2-8B
2025-12-118.7B tensors · dense
AA index5.0 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits
GPU
17.9 / 288 GiB
Generate~268–388 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
45
LFM2.5-VL-1.6BSpecialist
LiquidAI/LFM2.5-VL-1.6B
2026-01-051.6B tensors · dense
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits*
GPU
3.8 / 288 GiB
Generate~1,902–2,752 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
46
Apertus-8B-Instruct-2509
swiss-ai/Apertus-8B-Instruct-2509
2025-09-028.1B tensors · dense
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUFits
GPU
17.5 / 288 GiB
Generate~273–396 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
47
NVIDIA-Nemotron-3-Nano-4B-BF16
nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
2026-03-164B dense
AA index4.8single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits*
GPU
5.6 / 288 GiB
Generate~1,058–1,532 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
48
K2-Horizon-0.9B
IFM/K2-Horizon-0.9B
2026-09-031.1B tensors · dense
AA index3.0single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
3.0 / 288 GiB
Generate~3,290–4,761 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown

+ 65 models without a AA index score

LFM2.5-230MFP16 · full gpu · ~10,780 t/s
LFM2.5-350MFP16 · full gpu · ~7,248 t/s
LittleLambFP16 · full gpu · ~5,198 t/s
LittleLamb-ToolCallingFP16 · full gpu · ~5,198 t/s
Jais-3-0.9BQ8_0 · full gpu · ~3,991 t/s
Qwen3-0.6BFP16 · full gpu · ~2,790 t/s
LFM2.5-1.2B-JPFP16 · full gpu · ~2,307 t/s
granite-4.0-h-tinyQ8_0 · full gpu · ~1,930 t/s
granite-4.1-3bQ8_0 · full gpu · ~1,396 t/s
Qwen3-1.7BFP16 · full gpu · ~1,216 t/s
Agents-A1-4BQ8_0 · full gpu · ~1,196 t/s
FrogNano-4B-2609Q8_0 · full gpu · ~1,161 t/s
Qwen3-4BQ8_0 · full gpu · ~1,130 t/s
Mellum2.1-12B-A2.5B-ThinkingQ8_0 · full gpu · ~1,101 t/s
AgentCPM-ReportQ4_K_M · full gpu · ~1,083 t/s
Qwen3.5-35B-A3BQ8_0 · full gpu · ~926 t/s
Jais-3-3.7BQ8_0 · full gpu · ~922 t/s
Kolibri-1Q8_0 · full gpu · ~807 t/s
Qwen3-30B-A3BQ8_0 · full gpu · ~775 t/s
Qwen3-Coder-30B-A3B-InstructQ8_0 · full gpu · ~775 t/s
LFM2.5-24B-A2BFP16 · full gpu · ~754 t/s
PaCoRe-8BQ8_0 · full gpu · ~593 t/s
Qwen3-8BQ8_0 · full gpu · ~593 t/s
AgentCPM-ExploreFP16 · full gpu · ~585 t/s
MiMo-V2.6-Distill-Qwen-9BQ8_0 · full gpu · ~571 t/s
Ornith-1.5-9BQ8_0 · full gpu · ~570 t/s
granite-4.1-8bQ8_0 · full gpu · ~551 t/s
Jais-3-7BQ8_0 · full gpu · ~543 t/s
MiniCPM-V-4.7-35B-A3BFP16 · full gpu · ~510 t/s
Mistral 7B Instruct v0.3FP16 · full gpu · ~367 t/s
IQuest-Coder-V1-7B-Base-Stage1FP16 · full gpu · ~357 t/s
IQuest-Coder-V1-7B-BaseFP16 · full gpu · ~357 t/s
IQuest-Coder-V1-7B-InstructFP16 · full gpu · ~357 t/s
IQuest-Coder-V1-7B-ThinkingFP16 · full gpu · ~357 t/s
MagenticBrainQ8_0 · full gpu · ~337 t/s
DeepSeek-R1-Distill-Llama-8BFP16 · full gpu · ~333 t/s
RLVR-8B-0926FP16 · full gpu · ~325 t/s
Nemotron-Cascade-8BFP16 · full gpu · ~325 t/s
NVIDIA-Nemotron-Nano-9B-v2-JapaneseFP16 · full gpu · ~309 t/s
MiniMax-M2.7Q8_0 · full gpu · ~241 t/s
Mistral NeMo 12B Instruct (2407)FP16 · full gpu · ~219 t/s
Mixtral 8x7B Instruct v0.1Q8_0 · full gpu · ~216 t/s
Mistral Small 3.2 24B Instruct (2506)Q8_0 · full gpu · ~215 t/s
Qwen3.6-27BQ8_0 · full gpu · ~191 t/s
Laguna-S-2.1FP16 · full gpu · ~190 t/s
IQuest-Coder-V1-14B-Base-Stage1FP16 · full gpu · ~188 t/s
IQuest-Coder-V1-14B-BaseFP16 · full gpu · ~188 t/s
IQuest-Coder-V1-14B-InstructFP16 · full gpu · ~188 t/s
IQuest-Coder-V1-14B-ThinkingFP16 · full gpu · ~188 t/s
Qwen3.5-27BQ8_0 · full gpu · ~188 t/s
Qwen3-14BFP16 · full gpu · ~183 t/s
FrogMini-14B-2510FP16 · full gpu · ~183 t/s
granite-4.1-30bQ8_0 · full gpu · ~174 t/s
GLM-4.5-AirQ8_0 · full gpu · ~163 t/s
Olmo-3-32B-ThinkQ8_0 · full gpu · ~156 t/s
Qwen3-32BQ8_0 · full gpu · ~154 t/s
Qwen3-Nemotron-32B-RLBFFQ8_0 · full gpu · ~154 t/s
Jais-3-32BQ8_0 · full gpu · ~145 t/s
Qwen3-Coder-480B-A35B-InstructQ4_K_M · full gpu · ~137 t/s
IQuest-Coder-V1-40B-ThinkingQ8_0 · full gpu · ~127 t/s
Qwen3-235B-A22BQ8_0 · full gpu · ~126 t/s
Qwen3-VL-235B-A22B-ThinkingQ8_0 · full gpu · ~123 t/s
Qwen3-VL-235B-A22B-InstructQ8_0 · full gpu · ~123 t/s
FrogBoss-32B-2510FP16 · full gpu · ~83 t/s
DeepSeek-R1-Distill-Llama-70BFP16 · full gpu · ~39 t/s

Speed not established 7

Feasible plans with only a bandwidth ceiling (backends or placements without calibration). Check the evidence before relying on them.

1
G9v3-39A5B (preview)
ai9stars/G9v3-39A5B
2026-08 (day unknown)39B MoE · 5B active
AA index21.8 est.single mode
Plan & memory
native F32/BF16vLLMFull GPUFits
GPU
74.4 / 288 GiB
Generate≤788 t/sCeilingnot established
First token · 4K prompt—Unknown
2
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16Specialist
nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16
2026-04-2830B MoE · 3B active
AA index10.3 est.single mode
Plan & memory
native F32/BF16vLLMFull GPUFits*
GPU
63.0 / 288 GiB
Generate≤1,207 t/sCeilingnot established
First token · 4K prompt—Unknown
3
North-Mini-Code-1.0Specialist
CohereLabs/North-Mini-Code-1.0
2026-06-0930B MoE · 3B active
AA index9.9single mode
Plan & memory
native BF16vLLMFull GPUFits
GPU
58.8 / 288 GiB
Generate≤1,229 t/sCeilingnot established
First token · 4K prompt—Unknown
4
MiniCPM5-1B
openbmb/MiniCPM5-1B
2026-05-191.1B dense
AA index8.8 est.Reasoning
Plan & memory
native BF16vLLMFull GPUFits
GPU
3.7 / 288 GiB
Generate≤3,537 t/sCeilingnot established
First token · 4K prompt—Unknown
5
Nanbeige4.1-3B
Nanbeige/Nanbeige4.1-3B
2026-02 (day unknown)3B dense
AA index8.4 est.single mode
Plan & memory
native BF16vLLMFull GPUFits
GPU
9.3 / 288 GiB
Generate≤983 t/sCeilingnot established
First token · 4K prompt—Unknown
6
Step3-VL-10B
stepfun-ai/Step3-VL-10B
2026-01 (day unknown)10B dense
AA index7.7 est.single mode
Plan & memory
native BF16vLLMFull GPUFits
GPU
21.6 / 288 GiB
Generate≤382 t/sCeilingnot established
First token · 4K prompt—Unknown
7
NVIDIA-Nemotron-Nano-12B-v2-VL-BF16Specialist
nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16
2025-10-2813B tensors · dense
AA index7.5 est.Reasoning
Plan & memory
native F32/BF16vLLMFull GPUFits*
GPU
26.2 / 288 GiB
Generate≤302 t/sCeilingnot established
First token · 4K prompt—Unknown

+ 9 models without a AA index score

Agents-A1native BF16 · full gpu · ≤ 1,014 t/s
IQuest-Coder-V1-40B-Loop-Thinkingnative BF16 · full gpu · ≤ 99 t/s
K2-Horizon-32Bnative BF16 · full gpu · ≤ 113 t/s
Ming-flash-omni-2.0native BF16 · full gpu · ≤ 524 t/s
MiniCPM-SALAnative BF16 · full gpu · ≤ 419 t/s
Mixtral 8x22B Instruct v0.1native BF16 · full gpu · ≤ 101 t/s
NVIDIA-Nemotron-3-Super-120B-A12B-BF16native F32/BF16 · full gpu · ≤ 323 t/s
Penguin-VL-2Bnative BF16 · full gpu · ≤ 1,665 t/s
Penguin-VL-8Bnative BF16 · full gpu · ≤ 447 t/s

Memory not fully resolved 25

The weights fit, but the KV cache for this attention type (or the chosen cache layout) is not established, so a fit cannot be guaranteed.

1
GLM-5.3
zai-org/GLM-5.3
2026-08-28753B tensors · MoE
AA index44.8Max
Plan & memory
EXL3ExLlamaV3Full GPUKV unknown
GPU
185+ / 288 GiB
Generate≤597 t/sCeilingnot established
First token · 4K prompt—Unknown
2
GLM-5.3-Flash
zai-org/GLM-5.3-Flash
2026-08-26320B MoE · 18B active
AA index41.8single mode
Plan & memory
Q8_0llama.cpp CUDAGPU + CPU layers · 40/45 layers on GPUKV unknown
GPU
284+ / 288 GiB
RAM
35.3 / 56.0 GiB
Generate~8.6–11 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
3
Ling-3.1-flash
inclusionAI/Ling-3.1-flash
2026 (month unknown)560B MoE · 25B active
AA index41.1single mode
Plan & memory
Q4_K_Mllama.cpp CUDAGPU + CPU experts · 10 expert layers on CPUKV unknown
GPU
287+ / 288 GiB
RAM
47.9 / 56.0 GiB
Generate~9.3–14 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
4
DeepSeek-V4.1-Flash
deepseek-ai/DeepSeek-V4.1-Flash
2026-09-10748B MoE · 16B active
Also: DeepSeek-V4.1-Flash-1017 (dated update, not separately evaluated)
AA index39.5Max
Plan & memory
EXL3ExLlamaV3Full GPUKV unknown
GPU
198+ / 288 GiB
Generate≤1,776 t/sCeilingnot established
First token · 4K prompt—Unknown
5
MiMo-V2.6-Flash-MOPD
XiaomiMiMo/MiMo-V2.6-Flash-MOPD
2026-09-27309B MoE · 15B active
Also: MiMo-V2.6-Flash-RL (shares this AA evaluation)
AA index37.9single mode
Plan & memory
native fp8vLLMFull GPUKV unknown
GPU
163+ / 288 GiB
Generate≤953 t/sCeilingnot established
First token · 4K prompt—Unknown
6
Ling-3.0-flash-VL
inclusionAI/Ling-3.0-flash-VL
2026-09-09124B MoE · 5.5B active
AA index24.6single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
125+ / 288 GiB
Generate~257–744 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
7
Ling-3.0-flash-Fin
inclusionAI/Ling-3.0-flash-Fin
2026 (month unknown)124B MoE · 5.1B active
AA index22.6single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
128+ / 288 GiB
Generate~271–783 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
8
gemma-4-26B-A4B-it
google/gemma-4-26B-A4B-it
2026-04-0225B MoE · 3.8B active
AA index16.7 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
26.5+ / 288 GiB
Generate~373–1,078 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
9
gemma-4-31B-it
google/gemma-4-31B-it
2026-04-0231B dense
AA index14.7Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
31.9+ / 288 GiB
Generate~139–202 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
10
gemma-4-12B-it
google/gemma-4-12B-it
2026-06-0312B dense
AA index14.2 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
13.3+ / 288 GiB
Generate~359–520 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
11
LongCat-Flash-Lite
meituan-longcat/LongCat-Flash-Lite
2026-01 (day unknown)69B MoE · 3B active
AA index11.5 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
69.4+ / 288 GiB
Generate~474–1,369 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
12
Mistral-Small-4-119B-2603
mistralai/Mistral-Small-4-119B-2603
2026-03-16119B MoE · 6B active
AA index11.3Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
119+ / 288 GiB
Generate~237–685 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
13
Qwen3-Next-80B-A3B-ThinkingSpecialist
Qwen/Qwen3-Next-80B-A3B-Thinking
2025-09-1181B tensors · MoE
AA index11.2 est.Reasoning
Plan & memory
native BF16vLLMFull GPUKV unknown
GPU
153+ / 288 GiB
Generate≤724 t/sCeilingnot established
First token · 4K prompt—Unknown
14
Ling-3.0-tiny
inclusionAI/Ling-3.0-tiny
2026-08 (day unknown)7.9B MoE · 1.3B active
AA index11.1single mode
Plan & memory
FP16llama.cpp CUDAFull GPUKV unknown
GPU
16.2+ / 288 GiB
Generate~581–1,680 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
15
Qwen3-Next-80B-A3B-InstructOlder
Qwen/Qwen3-Next-80B-A3B-Instruct
2025-09-1181B tensors · MoE
AA index9.6 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
80.5+ / 288 GiB
Generate~262–758 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
16
diffusiongemma-26B-A4B-it
google/diffusiongemma-26B-A4B-it
2026-06-1025B MoE · 3.8B active
AA index9.5 est.single mode
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
26.5+ / 288 GiB
Generate~373–1,078 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
17
Llama 4 Maverick 17B-128E InstructOlderGated
meta-llama/Llama-4-Maverick-17B-128E-Instruct
2025-04-05402B tensors
AA index9.3single mode
Plan & memory
Q5_K_Mllama.cpp CUDAFull GPUKV unknown
GPU
266+ / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
18
Qwen3-Coder-NextSpecialist
Qwen/Qwen3-Coder-Next
2026-02-0280B MoE · 3B active
AA index9.2single mode
Plan & memory
FP16llama.cpp CUDAFull GPUKV unknown
GPU
150+ / 288 GiB
Generate~253–730 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
19
gemma-4-E4B-it
google/gemma-4-E4B-it
2026-04-028B dense
AA index8.9Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
9.0+ / 288 GiB
Generate~567–820 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
20
Falcon-H1R-7B
tiiuae/Falcon-H1R-7B
2026-01-057B dense
AA index7.8 est.single mode
Plan & memory
native BF16vLLMFull GPUKV unknown
GPU
15.6+ / 288 GiB
Generate≤527 t/sCeilingnot established
First token · 4K prompt—Unknown
21
gemma-4-E2B-it
google/gemma-4-E2B-it
2026-04-025.1B dense
AA index7.8 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAFull GPUKV unknown
GPU
6.1+ / 288 GiB
Generate~916–1,326 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
22
Llama 3.3 70B InstructOlderGated
meta-llama/Llama-3.3-70B-Instruct
2024-12-0671B tensors
AA index7.7 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUKV unknown
GPU
133+ / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
23
Llama 4 Scout 17B-16E InstructOlderGated
meta-llama/Llama-4-Scout-17B-16E-Instruct
2025-04-05109B tensors
AA index6.5single mode
Plan & memory
native BF16vLLMFull GPUKV unknown
GPU
204+ / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
24
Phi-4-mini-instructOlder
microsoft/Phi-4-mini-instruct
2025-02-263.8B tensors · dense
AA index6.3 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUKV unknown
GPU
8.7+ / 288 GiB
Generate~593–858 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
25
tiny-aya-globalGated
CohereLabs/tiny-aya-global
2026-02-173.3B tensors
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp CUDAFull GPUKV unknown
GPU
7.7+ / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown

+ 32 models without a AA index score

DeepSeek-R1-Distill-Qwen-1.5BQ8_0 · full gpu · ~2,914 t/s
NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16Q5_K_M · full gpu · ~1,503 t/s
GLM-4.7-FlashQ8_0 · full gpu · ~966 t/s
Ling-3.0-flashQ8_0 · full gpu · ~552 t/s
DeepSeek-R1-Distill-Qwen-7BFP16 · full gpu · ~362 t/s
Qwen2.5-7B-InstructFP16 · full gpu · ~362 t/s
DeepSeek-R1-Distill-Qwen-14BFP16 · full gpu · ~187 t/s
Qwen2.5-14B-InstructFP16 · full gpu · ~187 t/s
Qwen2.5-Coder-7B-InstructFP16 · full gpu · ~181 t/s
DeepSeek-R1-Distill-Qwen-32BQ8_0 · full gpu · ~159 t/s
Qwen2.5-Coder-32B-InstructFP16 · full gpu · ~84 t/s
Qwen2.5-32B-InstructFP16 · full gpu · ~84 t/s
Qwen2.5-72B-InstructFP16 · full gpu · ~38 t/s
Step-3.7-FlashQ8_0 · full gpu · ~26 t/s
DeepSeek-V3AWQ · full gpu · ≤ 112,951 t/s
DeepSeek-V4-Flash-0731native fp8 · full gpu
Gemma 3 12B ITFP16 · full gpu
Gemma 3 1B ITnative BF16 · full gpu
Gemma 3 27B ITnative BF16 · full gpu
Gemma 3 4B ITFP16 · full gpu
gpt-oss-puzzle-88Bnative mxfp4 · full gpu · ≤ 160 t/s
Llama 2 7Bnative F32/F16 · full gpu
Llama 3.1 70B Instructnative BF16 · full gpu
Llama 3.1 8B Instructnative BF16 · full gpu
Llama 3.2 1B InstructFP16 · full gpu
Llama 3.2 3B InstructFP16 · full gpu
Ming-flash-omni-Previewnative BF16/U8 · full gpu
Nemotron-Elastic-12Bnative BF16 · full gpu
tiny-aya-earthFP16 · full gpu
tiny-aya-fireFP16 · full gpu
tiny-aya-waterFP16 · full gpu
Youtu-Parsing-Omninative BF16 · full gpu · ≤ 750 t/s

Speed unknown 7

No measurement, calibration or bandwidth figure for this plan.

1
gpt-oss-120bOlder
openai/gpt-oss-120b
2025-08-05117B tensors · MoE
AA index11.6High
Plan & memory
Q8_0llama.cpp CUDAFull GPUFits
GPU
60.8 / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
2
gpt-oss-20bOlder
openai/gpt-oss-20b
2025-08-0521B tensors · MoE
AA index9.0High
Plan & memory
native mxfp4vLLMFull GPUFits
GPU
14.5 / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
3
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
2025-12-1532B tensors · MoE
AA index8.9Reasoning
Plan & memory
native F32/BF16vLLMFull GPUFits*
GPU
60.4 / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
4
granite-4.0-h-1b
ibm-granite/granite-4.0-h-1b
2025-10-281.5B tensors · MoE
AA index5.2 est.single mode
Plan & memory
native BF16vLLMFull GPUFits*
GPU
4.3 / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
5
granite-4.0-micro
ibm-granite/granite-4.0-micro
2025-10-023.4B tensors · MoE
AA index5.1 est.single mode
Plan & memory
native BF16vLLMFull GPUFits
GPU
8.5 / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
6
granite-4.0-350m
ibm-granite/granite-4.0-350m
2025-10-280.4B tensors · MoE
AA index4.8 est.single mode
Plan & memory
native BF16vLLMFull GPUFits
GPU
2.4 / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
7
granite-4.0-h-350m
ibm-granite/granite-4.0-h-350m
2025-10-280.3B tensors · MoE
AA index4.8 est.single mode
Plan & memory
native BF16vLLMFull GPUFits*
GPU
2.2 / 288 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown

+ 7 models without a AA index score

granite-4.0-1bnative BF16 · full gpu
granite-4.0-h-micronative BF16 · full gpu
Jais-3-MoVA-36B-A4Bnative F32 · full gpu
Nemotron-3-Super-120B-A12B-BF16-MTPv2native F32/BF16 · full gpu
NVIDIA-Nemotron-Labs-3-Competitive-Coding-550B-A55B-NVFP4native modelopt · gpu streams weights from ram
NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-BF16native F32/BF16 · full gpu
NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16native BF16/F32 · full gpu

Below 5 t/s 0

Fits, but the best evidence (measured, estimated or the bandwidth ceiling) is under your floor. Lower the floor or add memory bandwidth.

1 model without a AA index score

Agnes-3.0-Qwennative BF16/I64 · gpu streams weights from ram · ≤ 1.7 t/s

36 models with no feasible plan on this setup

MiMo-V2.6-Pro-MOPDnative fp8 527 GiB
Step-5-Previewnative fp8 600 GiB
Kimi-K3native F32/BF16 1,454 GiB
Qwen3.8-2.4T-A95Bnative BF16 4,556 GiB
Muse-Spark-Open-816B-A42BQ4_K_M 465 GiB exceeds memory
DeepSeek-V4-Pro-0813native fp8 831 GiB
Motif-3native BF16 586 GiB
K2-Horizon-375B-A23Bnative BF16 706 GiB
Kimi-K2.7-Codenative BF16/F32 554 GiB
InklingQ8_0 798 GiB
Solar-Open2-250Bnative BF16/F32 466 GiB
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16native BF16/F32 1,044 GiB
A.X-K2native fp8 647 GiB
K-EXAONE-2.0-750B-A37Bnative F32/BF16 1,396 GiB
LongCat-2.0native BF16/F32 3,308 GiB
Ring-2.6-1Tnative quantized 970 GiB
command-a-plus-05-2026-bf16native BF16 407 GiB
sarvam-105bnative F32 395 GiB
Agnes-2.5-Pro-Alphanative BF16 739 GiB
Apertus-v1.1-0.5B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.1-1.5B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.1-4B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.5-8Bno backend/format combination
BaguettotronContext 8K exceeds this model's configured maximum (4K)
DeepSeek-R1Q4_K_M 377 GiB
GLM-5.2Q4_K_M 434 GiB
Hy3-previewnative BF16/F32 557 GiB
Hy4-previewnative BF16/F32 1,453 GiB
Jais-3-375B-A23Bnative BF16/F32 706 GiB
Kimi-K2.6native F32/I32 554 GiB
MonadContext 8K exceeds this model's configured maximum (2K)
NVIDIA-Nemotron-Labs-Teacher-Chatnative F32/BF16 1,044 GiB
NVIDIA-Nemotron-Labs-Teacher-Competition-Codingnative BF16/F32 1,044 GiB
NVIDIA-Nemotron-Labs-Teacher-Instruction-Followingnative BF16/F32 1,044 GiB
NVIDIA-Nemotron-Labs-Teacher-STEMnative BF16/F32 1,044 GiB
Apertus-v1.5-70Bno verified artifact for these backends

Specs & sources

Memory	288 GB HBM3e	source
Bandwidth	8,000 GB/s	source
FP16	unknown	No unambiguous comparable value verified; never substitute AI TOPS, BF16, or whole-system power.
Power	unknown	No unambiguous comparable value verified; never substitute AI TOPS, BF16, or whole-system power.

Current HGX reference architecture says up to 8TB/s per GPU. Use source generation; do not substitute old physical-versus-usable capacity figures from other systems.

Reported speeds

0 observations

No reported observation resolved to this exact device yet.

Open in speed database Submit yours

Doesn't fit? Rent a GPU

RunpodPlaceholderCloud GPU pods you can rent for running larger models.
Vast.aiPlaceholderGPU rental marketplace with many hosts and GPU types.
LambdaPlaceholderCloud GPU instances for AI training and inference.
TensorDockPlaceholderGPU cloud marketplace with consumer and datacenter GPUs.

Sponsored placeholder links: LocalFit may earn a commission if these become affiliate links. They never influence fit results, estimates or picks.

NVIDIA B300 SXM

Best plans B300 SXM

Meets your 5 t/s floor 48

NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16Specialist

Olmo-3.1-32B-ThinkSpecialist

Phi-4Older

Olmo-3-7B-ThinkSpecialist

LFM2.5-1.2B-ThinkingSpecialist

Molmo2-8BSpecialist

LFM2.5-VL-1.6BSpecialist

Speed not established 7

Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16Specialist

North-Mini-Code-1.0Specialist

NVIDIA-Nemotron-Nano-12B-v2-VL-BF16Specialist

Memory not fully resolved 25

Qwen3-Next-80B-A3B-ThinkingSpecialist

Qwen3-Next-80B-A3B-InstructOlder

Llama 4 Maverick 17B-128E InstructOlderGated

Qwen3-Coder-NextSpecialist

Llama 3.3 70B InstructOlderGated

Llama 4 Scout 17B-16E InstructOlderGated

Phi-4-mini-instructOlder

tiny-aya-globalGated

Speed unknown 7

gpt-oss-120bOlder

gpt-oss-20bOlder

Below 5 t/s 0

Specs & sources

Reported speeds

Doesn't fit? Rent a GPU