NVIDIA GB10 OEM system 64GB: what runs best?

RTX 4090 RTX 5090 RTX 3090 RTX 4060 Ti (16 GB)RTX 5070 Ti Radeon RX 7900 XTX M4 Max 128 GB M3 Ultra 512 GB DGX Spark (128 GB)All devices

NVIDIA · Unified-memory system

NVIDIA GB10 OEM system 64GB

Memory64 GB LPDDR5X coherent unifiedBandwidth273 GB/s vendor specPlanned for models75% of unified memory (editable)

GPUs

Custom GPU

Unified memory

Chip / systemGPU coresMemory

Advanced: reserves, bandwidth, workload, models

Available VRAM per GPUGiB Runtime allowance per deviceGiB Spare margin per deviceGiB Unified memory for models% RAM kept for OS and appsGiB Measured RAM bandwidthGB/s GPU linkLanes Prompt length (first token)tokens Custom contexttokens Parallel sequences

Allow disk-streamed experts (documented cases)Include base checkpointsInclude specialistsInclude community & quant-trained variantsInclude models with unverified release dates

Defaults are editorial planning choices, not measurements: 1 GiB runtime + 0.5 GiB margin per device, 75% of unified memory, 8 GiB RAM for the OS, a vendor “24 GB” treated as 24 GiB (enter your driver’s available VRAM to override), 5 t/s minimum speed (research editorial default). Memory ledger

NVIDIA GB10 OEM system 64GB · 75% (48.0 GiB) planned for models · 273 GB/s8K contextF16 KVany backend≥ 5 t/s

Best plans NVIDIA GB10 OEM system 64GB

Best quality at ≥ 5 t/s

Qwen3.8-27B

33.7 AA index Xhigh

Q8_0llama.cpp CUDAUnified memory

~5.4–7.8 t/s Estimated

Faster tradeoff

K2-Horizon-MoVA-36B-A4B

25.3 AA index single mode

Q8_0llama.cpp CUDAUnified memory

~9.9–28 t/s Estimated extrapolated

Next point on the quality/speed front; then Qwen3.6-35B-A3B.

Possibly better · speed not established

No higher-scoring model has a feasible plan here.

WeightsKV cacheReserveFreeFits* = a state term unsizedMeasuredlinks to its sourceEstimatedcalibrated or extrapolated range, ~Ceilingupper bound, ≤Unknownnever zero

Meets your 5 t/s floor 35

Best plan per model that reaches your floor by a matching measurement, a calibrated range, or an extrapolated estimate (labelled). Sorted by the selected quality metric, then speed.

1
Qwen3.8-27B
Qwen/Qwen3.8-27B
2026-08-1427B dense
AA index33.7Xhigh
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
28.6 / 48.0 GiB
Generate~5.4–7.8 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
2
K2-Horizon-MoVA-36B-A4B
IFM/K2-Horizon-MoVA-36B-A4B
2026-09-0336B MoE · 4B active
AA index25.3single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
40.1 / 48.0 GiB
Generate~9.9–28 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
3
K2-Horizon-7B
IFM/K2-Horizon-7B
2026-09-039B tensors · dense
AA index20.6single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
11.5 / 48.0 GiB
Generate~15–22 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
4
Qwen3.6-35B-A3B
Qwen/Qwen3.6-35B-A3B
2026-04-1635B MoE · 3B active
AA index18.2Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
36.0 / 48.0 GiB
Generate~16–46 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
5
Muse-Glimmer-30B
meta-models/Muse-Glimmer-30B
2026-08-1030B dense
AA index17.5High
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
29.3 / 48.0 GiB
Generate~5.2–7.6 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
6
K2-Horizon-3.7B
IFM/K2-Horizon-3.7B
2026-09-035.1B tensors · dense
AA index15.6single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
7.6 / 48.0 GiB
Generate~26–38 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
7
EXAONE-4.5-33B
LGAI-EXAONE/EXAONE-4.5-33B
2026-04-0933B dense
AA index13.2 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
35.5 / 48.0 GiB
Generate~4.3–6.2 t/sEstimated≈ meets 5 t/s
First token · 4K prompt—Unknown
8
Qwen3.5-4B
Qwen/Qwen3.5-4B
2026-03-024B dense
AA index13.1 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
6.1 / 48.0 GiB
Generate~33–47 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
9
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
2026-08-1130B MoE · 3B active
AA index12.9single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
32.8 / 48.0 GiB
Generate~15–44 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
10
granite-4.2-30b
ibm-granite/granite-4.2-30b
2026-08-2530B dense
AA index12.7single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
32.5 / 48.0 GiB
Generate~4.8–7.0 t/sEstimated≈ meets 5 t/s
First token · 4K prompt—Unknown
11
MiniCPM5-2B
openbmb/MiniCPM5-2B
2026-09-072.5B dense
AA index12.5single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits
Unified
6.5 / 48.0 GiB
Generate~30–43 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
12
Hypernova-60B-2605
MultiverseComputingCAI/Hypernova-60B-2605
2026-05-0660B MoE · 4.8B active
Also: HyperNova-60B (listed with its dated update); Hypernova-60B-2602 (dated update, not separately evaluated)
AA index11.7 est.High
Plan & memory
Q5_K_Mllama.cpp CUDAUnified memoryFits
Unified
43.0 / 48.0 GiB
Generate~14–40 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
13
Nemotron-Cascade-2-30B-A3B
nvidia/Nemotron-Cascade-2-30B-A3B
2026-03-1930B MoE · 3B active
AA index11.7 est.single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
32.8 / 48.0 GiB
Generate~15–44 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
14
Qwen3.5-9B
Qwen/Qwen3.5-9B
2026-03-029B dense
AA index11.2Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
10.9 / 48.0 GiB
Generate~16–23 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
15
granite-4.2-8b
ibm-granite/granite-4.2-8b
2026-08-258B dense
AA index11.1single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
11.5 / 48.0 GiB
Generate~16–22 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
16
G9v3-3B
ai9stars/G9v3-3B
2026-07 (day unknown)3B dense
AA index10.8 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits
Unified
7.5 / 48.0 GiB
Generate~25–36 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
17
granite-4.2-3b
ibm-granite/granite-4.2-3b
2026-08-253B dense
AA index9.1single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
5.8 / 48.0 GiB
Generate~37–53 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
18
Tri-21B-Think
trillionlabs/Tri-21B-Think
2026-02 (day unknown)21B dense
AA index9.0 est.single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
23.6 / 48.0 GiB
Generate~6.8–9.8 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
19
LFM2.5-2.6B
LiquidAI/LFM2.5-2.6B
2026-08-042.7B dense
AA index8.4 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits*
Unified
6.7 / 48.0 GiB
Generate~28–41 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
20
LFM2.5-8B-A1B
LiquidAI/LFM2.5-8B-A1B
2026-05-288.3B MoE · 1.5B active
AA index7.2 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits*
Unified
17.4 / 48.0 GiB
Generate~17–48 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
21
Olmo-3.1-32B-ThinkSpecialist
allenai/Olmo-3.1-32B-Think
2025-12-1232B tensors · dense
AA index7.1 est.single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
34.7 / 48.0 GiB
Generate~4.4–6.4 t/sEstimated≈ meets 5 t/s
First token · 4K prompt—Unknown
22
Qwen3.5-2B
Qwen/Qwen3.5-2B
2026-03-022B dense
AA index6.9 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
3.5 / 48.0 GiB
Generate~73–106 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
23
Olmo-3.1-32B-Instruct
allenai/Olmo-3.1-32B-Instruct
2025-12-1232B tensors · dense
AA index6.5 est.single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
34.7 / 48.0 GiB
Generate~4.4–6.4 t/sEstimated≈ meets 5 t/s
First token · 4K prompt—Unknown
24
Qwen3.5-0.8B
Qwen/Qwen3.5-0.8B
2026-03-020.8B dense
AA index6.1 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
2.4 / 48.0 GiB
Generate~176–254 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
25
granite-4.0-h-small
ibm-granite/granite-4.0-h-small
2025-10-0232B tensors · MoE
AA index6.0 est.single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
33.5 / 48.0 GiB
Generate~5.5–16 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
26
Phi-4Older
microsoft/phi-4
2024-12-1215B tensors · dense
AA index5.9 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits
Unified
30.4 / 48.0 GiB
Generate~5.2–7.5 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
27
Olmo-3-7B-ThinkSpecialist
allenai/Olmo-3-7B-Think
2025-11-207.3B tensors · dense
AA index5.6 est.single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
11.2 / 48.0 GiB
Generate~16–23 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
28
Olmo-3-7B-Instruct
allenai/Olmo-3-7B-Instruct
2025-11-207.3B tensors · dense
AA index5.2 est.single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
11.2 / 48.0 GiB
Generate~16–23 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
29
LFM2.5-1.2B-ThinkingSpecialist
LiquidAI/LFM2.5-1.2B-Thinking
2026-01-201.2B tensors · dense
AA index5.2 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits*
Unified
3.8 / 48.0 GiB
Generate~65–94 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
30
LFM2.5-1.2B-Instruct
LiquidAI/LFM2.5-1.2B-Instruct
2026-01-051.2B tensors · dense
AA index5.2 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits*
Unified
3.8 / 48.0 GiB
Generate~65–94 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
31
Molmo2-8BSpecialist
allenai/Molmo2-8B
2025-12-118.7B tensors · dense
AA index5.0 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits
Unified
17.9 / 48.0 GiB
Generate~9.1–13 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
32
LFM2.5-VL-1.6BSpecialist
LiquidAI/LFM2.5-VL-1.6B
2026-01-051.6B tensors · dense
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits*
Unified
3.8 / 48.0 GiB
Generate~65–94 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
33
Apertus-8B-Instruct-2509
swiss-ai/Apertus-8B-Instruct-2509
2025-09-028.1B tensors · dense
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryFits
Unified
17.5 / 48.0 GiB
Generate~9.3–13 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
34
NVIDIA-Nemotron-3-Nano-4B-BF16
nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
2026-03-164B dense
AA index4.8single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits*
Unified
5.6 / 48.0 GiB
Generate~36–52 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
35
K2-Horizon-0.9B
IFM/K2-Horizon-0.9B
2026-09-031.1B tensors · dense
AA index3.0single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryFits
Unified
3.0 / 48.0 GiB
Generate~112–162 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown

+ 57 models without a AA index score

LFM2.5-230MFP16 · unified memory · ~368 t/s
LFM2.5-350MFP16 · unified memory · ~247 t/s
LittleLambFP16 · unified memory · ~177 t/s
LittleLamb-ToolCallingFP16 · unified memory · ~177 t/s
Jais-3-0.9BQ8_0 · unified memory · ~136 t/s
Qwen3-0.6BFP16 · unified memory · ~95 t/s
LFM2.5-1.2B-JPFP16 · unified memory · ~79 t/s
granite-4.0-h-tinyQ8_0 · unified memory · ~66 t/s
granite-4.1-3bQ8_0 · unified memory · ~48 t/s
Kolibri-1Q4_K_M · unified memory · ~47 t/s
Qwen3-1.7BFP16 · unified memory · ~42 t/s
Agents-A1-4BQ8_0 · unified memory · ~41 t/s
FrogNano-4B-2609Q8_0 · unified memory · ~40 t/s
Qwen3-4BQ8_0 · unified memory · ~39 t/s
Mellum2.1-12B-A2.5B-ThinkingQ8_0 · unified memory · ~38 t/s
AgentCPM-ReportQ4_K_M · unified memory · ~37 t/s
MiniCPM-V-4.7-35B-A3BQ8_0 · unified memory · ~32 t/s
Qwen3.5-35B-A3BQ8_0 · unified memory · ~32 t/s
Jais-3-3.7BQ8_0 · unified memory · ~31 t/s
Qwen3-30B-A3BQ8_0 · unified memory · ~26 t/s
Qwen3-Coder-30B-A3B-InstructQ8_0 · unified memory · ~26 t/s
LFM2.5-24B-A2BFP16 · unified memory · ~26 t/s
PaCoRe-8BQ8_0 · unified memory · ~20 t/s
Qwen3-8BQ8_0 · unified memory · ~20 t/s
AgentCPM-ExploreFP16 · unified memory · ~20 t/s
MiMo-V2.6-Distill-Qwen-9BQ8_0 · unified memory · ~20 t/s
Ornith-1.5-9BQ8_0 · unified memory · ~19 t/s
granite-4.1-8bQ8_0 · unified memory · ~19 t/s
Jais-3-7BQ8_0 · unified memory · ~19 t/s
Mistral 7B Instruct v0.3FP16 · unified memory · ~13 t/s
IQuest-Coder-V1-7B-Base-Stage1FP16 · unified memory · ~12 t/s
IQuest-Coder-V1-7B-BaseFP16 · unified memory · ~12 t/s
IQuest-Coder-V1-7B-InstructFP16 · unified memory · ~12 t/s
IQuest-Coder-V1-7B-ThinkingFP16 · unified memory · ~12 t/s
MagenticBrainQ8_0 · unified memory · ~12 t/s
DeepSeek-R1-Distill-Llama-8BFP16 · unified memory · ~11 t/s
Mixtral 8x7B Instruct v0.1Q5_K_M · unified memory · ~11 t/s
RLVR-8B-0926FP16 · unified memory · ~11 t/s
Nemotron-Cascade-8BFP16 · unified memory · ~11 t/s
NVIDIA-Nemotron-Nano-9B-v2-JapaneseFP16 · unified memory · ~11 t/s
Mistral NeMo 12B Instruct (2407)FP16 · unified memory · ~7.5 t/s
Mistral Small 3.2 24B Instruct (2506)Q8_0 · unified memory · ~7.3 t/s
Qwen3.6-27BQ8_0 · unified memory · ~6.5 t/s
IQuest-Coder-V1-14B-Base-Stage1FP16 · unified memory · ~6.4 t/s
IQuest-Coder-V1-14B-BaseFP16 · unified memory · ~6.4 t/s
IQuest-Coder-V1-14B-InstructFP16 · unified memory · ~6.4 t/s
IQuest-Coder-V1-14B-ThinkingFP16 · unified memory · ~6.4 t/s
Qwen3.5-27BQ8_0 · unified memory · ~6.4 t/s
IQuest-Coder-V1-40B-ThinkingQ5_K_M · unified memory · ~6.4 t/s
Qwen3-14BFP16 · unified memory · ~6.2 t/s
FrogMini-14B-2510FP16 · unified memory · ~6.2 t/s
granite-4.1-30bQ8_0 · unified memory · ~5.9 t/s
Olmo-3-32B-ThinkQ8_0 · unified memory · ~5.3 t/s
Qwen3-32BQ8_0 · unified memory · ~5.2 t/s
FrogBoss-32B-2510Q8_0 · unified memory · ~5.2 t/s
Qwen3-Nemotron-32B-RLBFFQ8_0 · unified memory · ~5.2 t/s
Jais-3-32BQ8_0 · unified memory · ~5.1 t/s

Speed not established 4

Feasible plans with only a bandwidth ceiling (backends or placements without calibration). Check the evidence before relying on them.

1
MiniCPM5-1B
openbmb/MiniCPM5-1B
2026-05-191.1B dense
AA index8.8 est.Reasoning
Plan & memory
native BF16vLLMUnified memoryFits
Unified
3.7 / 48.0 GiB
Generate≤121 t/sCeilingnot established
First token · 4K prompt—Unknown
2
Nanbeige4.1-3B
Nanbeige/Nanbeige4.1-3B
2026-02 (day unknown)3B dense
AA index8.4 est.single mode
Plan & memory
native BF16vLLMUnified memoryFits
Unified
9.3 / 48.0 GiB
Generate≤34 t/sCeilingnot established
First token · 4K prompt—Unknown
3
Step3-VL-10B
stepfun-ai/Step3-VL-10B
2026-01 (day unknown)10B dense
AA index7.7 est.single mode
Plan & memory
native BF16vLLMUnified memoryFits
Unified
21.6 / 48.0 GiB
Generate≤13 t/sCeilingnot established
First token · 4K prompt—Unknown
4
NVIDIA-Nemotron-Nano-12B-v2-VL-BF16Specialist
nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16
2025-10-2813B tensors · dense
AA index7.5 est.Reasoning
Plan & memory
native F32/BF16vLLMUnified memoryFits*
Unified
26.2 / 48.0 GiB
Generate≤10 t/sCeilingnot established
First token · 4K prompt—Unknown

+ 3 models without a AA index score

MiniCPM-SALAnative BF16 · unified memory · ≤ 14 t/s
Penguin-VL-2Bnative BF16 · unified memory · ≤ 57 t/s
Penguin-VL-8Bnative BF16 · unified memory · ≤ 15 t/s

Memory not fully resolved 14

The weights fit, but the KV cache for this attention type (or the chosen cache layout) is not established, so a fit cannot be guaranteed.

1
gemma-4-26B-A4B-it
google/gemma-4-26B-A4B-it
2026-04-0225B MoE · 3.8B active
AA index16.7 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryKV unknown
Unified
26.5+ / 48.0 GiB
Generate~13–37 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
2
gemma-4-31B-it
google/gemma-4-31B-it
2026-04-0231B dense
AA index14.7Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryKV unknown
Unified
31.9+ / 48.0 GiB
Generate~4.8–6.9 t/sEstimated≈ meets 5 t/s
First token · 4K prompt—Unknown
3
gemma-4-12B-it
google/gemma-4-12B-it
2026-06-0312B dense
AA index14.2 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryKV unknown
Unified
13.3+ / 48.0 GiB
Generate~12–18 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
4
LongCat-Flash-Lite
meituan-longcat/LongCat-Flash-Lite
2026-01 (day unknown)69B MoE · 3B active
AA index11.5 est.single mode
Plan & memory
Q5_K_Mllama.cpp CUDAUnified memoryKV unknown
Unified
46.2+ / 48.0 GiB
Generate~25–71 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
5
Ling-3.0-tiny
inclusionAI/Ling-3.0-tiny
2026-08 (day unknown)7.9B MoE · 1.3B active
AA index11.1single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryKV unknown
Unified
16.2+ / 48.0 GiB
Generate~20–57 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
6
Qwen3-Next-80B-A3B-InstructOlder
Qwen/Qwen3-Next-80B-A3B-Instruct
2025-09-1181B tensors · MoE
AA index9.6 est.single mode
Plan & memory
Q4_K_Mllama.cpp CUDAUnified memoryKV unknown
Unified
46.7+ / 48.0 GiB
Generate~16–45 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
7
diffusiongemma-26B-A4B-it
google/diffusiongemma-26B-A4B-it
2026-06-1025B MoE · 3.8B active
AA index9.5 est.single mode
Plan & memory
Q8_0llama.cpp CUDAUnified memoryKV unknown
Unified
26.5+ / 48.0 GiB
Generate~13–37 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
8
Qwen3-Coder-NextSpecialist
Qwen/Qwen3-Coder-Next
2026-02-0280B MoE · 3B active
AA index9.2single mode
Plan & memory
Q4_K_Mllama.cpp CUDAUnified memoryKV unknown
Unified
46.9+ / 48.0 GiB
Generate~28–82 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
9
gemma-4-E4B-it
google/gemma-4-E4B-it
2026-04-028B dense
AA index8.9Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryKV unknown
Unified
9.0+ / 48.0 GiB
Generate~19–28 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
10
Falcon-H1R-7B
tiiuae/Falcon-H1R-7B
2026-01-057B dense
AA index7.8 est.single mode
Plan & memory
native BF16vLLMUnified memoryKV unknown
Unified
15.6+ / 48.0 GiB
Generate≤18 t/sCeilingnot established
First token · 4K prompt—Unknown
11
gemma-4-E2B-it
google/gemma-4-E2B-it
2026-04-025.1B dense
AA index7.8 est.Reasoning
Plan & memory
Q8_0llama.cpp CUDAUnified memoryKV unknown
Unified
6.1+ / 48.0 GiB
Generate~31–45 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
12
Llama 3.3 70B InstructOlderGated
meta-llama/Llama-3.3-70B-Instruct
2024-12-0671B tensors
AA index7.7 est.single mode
Plan & memory
Q4_K_Mllama.cpp CUDAUnified memoryKV unknown
Unified
41.1+ / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
13
Phi-4-mini-instructOlder
microsoft/Phi-4-mini-instruct
2025-02-263.8B tensors · dense
AA index6.3 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryKV unknown
Unified
8.7+ / 48.0 GiB
Generate~20–29 t/sEstimatedmeets 5 t/s
First token · 4K prompt—Unknown
14
tiny-aya-globalGated
CohereLabs/tiny-aya-global
2026-02-173.3B tensors
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp CUDAUnified memoryKV unknown
Unified
7.7+ / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown

+ 27 models without a AA index score

DeepSeek-R1-Distill-Qwen-1.5BQ8_0 · unified memory · ~99 t/s
NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16Q5_K_M · unified memory · ~51 t/s
GLM-4.7-FlashQ8_0 · unified memory · ~33 t/s
DeepSeek-R1-Distill-Qwen-7BFP16 · unified memory · ~12 t/s
Qwen2.5-7B-InstructFP16 · unified memory · ~12 t/s
DeepSeek-R1-Distill-Qwen-14BFP16 · unified memory · ~6.4 t/s
Qwen2.5-14B-InstructFP16 · unified memory · ~6.4 t/s
Qwen2.5-Coder-7B-InstructFP16 · unified memory · ~6.2 t/s
DeepSeek-R1-Distill-Qwen-32BQ8_0 · unified memory · ~5.4 t/s
Qwen2.5-32B-InstructQ8_0 · unified memory · ~5.4 t/s
DeepSeek-V3AWQ · unified memory · ≤ 3,854 t/s
Gemma 3 12B ITFP16 · unified memory
Gemma 3 1B ITnative BF16 · unified memory
Gemma 3 27B ITQ8_0 · unified memory
Gemma 3 4B ITFP16 · unified memory
Llama 2 7Bnative F32/F16 · unified memory
Llama 3.1 70B InstructQ4_K_M · unified memory
Llama 3.1 8B Instructnative BF16 · unified memory
Llama 3.2 1B InstructFP16 · unified memory
Llama 3.2 3B InstructFP16 · unified memory
Nemotron-Elastic-12Bnative BF16 · unified memory
Qwen2.5-72B-InstructAWQ · unified memory · ≤ 6.6 t/s
Qwen2.5-Coder-32B-InstructAWQ · unified memory · ≤ 14 t/s
tiny-aya-earthFP16 · unified memory
tiny-aya-fireFP16 · unified memory
tiny-aya-waterFP16 · unified memory
Youtu-Parsing-Omninative BF16 · unified memory · ≤ 26 t/s

Speed unknown 5

No measurement, calibration or bandwidth figure for this plan.

1
gpt-oss-20bOlder
openai/gpt-oss-20b
2025-08-0521B tensors · MoE
AA index9.0High
Plan & memory
native mxfp4vLLMUnified memoryFits
Unified
14.5 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
2
granite-4.0-h-1b
ibm-granite/granite-4.0-h-1b
2025-10-281.5B tensors · MoE
AA index5.2 est.single mode
Plan & memory
native BF16vLLMUnified memoryFits*
Unified
4.3 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
3
granite-4.0-micro
ibm-granite/granite-4.0-micro
2025-10-023.4B tensors · MoE
AA index5.1 est.single mode
Plan & memory
native BF16vLLMUnified memoryFits
Unified
8.5 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
4
granite-4.0-350m
ibm-granite/granite-4.0-350m
2025-10-280.4B tensors · MoE
AA index4.8 est.single mode
Plan & memory
native BF16vLLMUnified memoryFits
Unified
2.4 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
5
granite-4.0-h-350m
ibm-granite/granite-4.0-h-350m
2025-10-280.3B tensors · MoE
AA index4.8 est.single mode
Plan & memory
native BF16vLLMUnified memoryFits*
Unified
2.2 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown

+ 5 models without a AA index score

granite-4.0-1bnative BF16 · unified memory
granite-4.0-h-micronative BF16 · unified memory
Jais-3-MoVA-36B-A4BQ8_0 · unified memory
Nemotron-3-Super-120B-A12B-BF16-MTPv2native F32/BF16 · unified memory
NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-BF16Q8_0 · unified memory

Below 5 t/s 1

Fits, but the best evidence (measured, estimated or the bandwidth ceiling) is under your floor. Lower the floor or add memory bandwidth.

1
Apertus-70B-Instruct-2509
swiss-ai/Apertus-70B-Instruct-2509
2025-09-0271B tensors · dense
AA index5.1 est.single mode
Plan & memory
Q4_K_Mllama.cpp CUDAUnified memoryFits
Unified
44.7 / 48.0 GiB
Generate~3.4–5.0 t/sEstimatedbelow 5 t/s
First token · 4K prompt—Unknown

+ 1 model without a AA index score

DeepSeek-R1-Distill-Llama-70BQ4_K_M · unified memory · ~4.3 t/s

85 models with no feasible plan on this setup

MiMo-V2.6-Pro-MOPDnative fp8 527 GiB exceeds memory
GLM-5.3native fp8 704 GiB exceeds memory
Step-5-Previewnative fp8 600 GiB exceeds memory
Kimi-K3native F32/BF16 1,454 GiB exceeds memory
GLM-5.3-Flashnative fp8 306 GiB exceeds memory
Ling-3.1-flashQ4_K_M 333 GiB exceeds memory
Qwen3.8-2.4T-A95Bnative BF16 4,556 GiB exceeds memory
Qwen3.8-Flash-NextQ4_K_M 111 GiB exceeds memory
DeepSeek-V4.1-FlashQ4_K_M 414 GiB exceeds memory
Muse-Spark-Open-816B-A42BQ4_K_M 465 GiB exceeds memory
MiMo-V2.6-Flash-MOPDnative fp8 161 GiB exceeds memory
DeepSeek-V4-Pro-0813native fp8 831 GiB exceeds memory
Motif-3native BF16 586 GiB exceeds memory
K2-Horizon-375B-A23Bnative BF16 706 GiB exceeds memory
Beam-1.0-501B-A23BQ5_K_M 331 GiB exceeds memory
MiniMax-M3Q4_K_M 247 GiB exceeds memory
Nex-N2-ProQ4_K_M 225 GiB exceeds memory
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16Q4_K_M 80.1 GiB exceeds memory
Kimi-K2.7-Codenative BF16/F32 554 GiB exceeds memory
Inkling-SmallQ4_K_M 151 GiB exceeds memory
Hy3Q4_K_M 170 GiB exceeds memory
InklingQ8_0 798 GiB exceeds memory
Solar-Open2-250Bnative BF16/F32 466 GiB exceeds memory
Ling-3.0-flash-VLQ4_K_M 73.3 GiB exceeds memory
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16native BF16/F32 1,044 GiB exceeds memory
Ling-3.0-flash-FinQ4_K_M 72.5 GiB exceeds memory
G9v3-39A5B (preview)native F32/BF16 72.6 GiB exceeds memory
A.X-K2native fp8 647 GiB exceeds memory
NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16Q8_0 73.4 GiB exceeds memory
K-EXAONE-2.0-750B-A37Bnative F32/BF16 1,396 GiB exceeds memory
LongCat-2.0native BF16/F32 3,308 GiB exceeds memory
Qwen3.5-397B-A17BGPTQ 220 GiB exceeds memory
Ring-2.6-1Tnative quantized 970 GiB exceeds memory
Qwen3.5-122B-A10BQ4_K_M 72.3 GiB exceeds memory
Mistral-Medium-3.5-128BQ4_K_M 69.8 GiB exceeds memory
command-a-plus-05-2026-bf16native BF16 407 GiB exceeds memory
gpt-oss-120bQ4_K_M 58.5 GiB exceeds memory
Mistral-Small-4-119B-2603Q4_K_M 67.6 GiB exceeds memory
Qwen3-Next-80B-A3B-Thinkingnative BF16 151 GiB exceeds memory
Trinity-Large-ThinkingQ4_K_M 225 GiB exceeds memory
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16native F32/BF16 61.5 GiB exceeds memory
North-Mini-Code-1.0native BF16 56.8 GiB exceeds memory
Llama 4 Maverick 17B-128E InstructQ4_K_M 226 GiB exceeds memory
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16native F32/BF16 58.8 GiB exceeds memory
sarvam-105bnative F32 395 GiB exceeds memory
Llama 4 Scout 17B-16E InstructQ4_K_M 62.9 GiB exceeds memory
Agents-A1native BF16 65.4 GiB exceeds memory
Agnes-2.5-Pro-Alphanative BF16 739 GiB exceeds memory
Agnes-3.0-Qwennative BF16/I64 335 GiB exceeds memory
Apertus-v1.1-0.5B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.1-1.5B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.1-4B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.5-8Bno backend/format combination
BaguettotronContext 8K exceeds this model's configured maximum (4K)
DeepSeek-R1Q4_K_M 377 GiB exceeds memory
DeepSeek-V4-Flash-0731native fp8 155 GiB exceeds memory
GLM-4.5-AirQ4_K_M 68.0 GiB exceeds memory
GLM-5.2Q4_K_M 434 GiB exceeds memory
gpt-oss-puzzle-88Bnative mxfp4 46.6 GiB exceeds memory
Hy3-previewnative BF16/F32 557 GiB exceeds memory
Hy4-previewnative BF16/F32 1,453 GiB exceeds memory
IQuest-Coder-V1-40B-Loop-Thinkingnative BF16 74.1 GiB exceeds memory
Jais-3-375B-A23Bnative BF16/F32 706 GiB exceeds memory
K2-Horizon-32Bnative BF16 64.8 GiB exceeds memory
Kimi-K2.6native F32/I32 554 GiB exceeds memory
Laguna-S-2.1Q5_K_M 81.8 GiB exceeds memory
Ling-3.0-flashQ4_K_M 72.9 GiB exceeds memory
Ming-flash-omni-2.0native BF16 194 GiB exceeds memory
Ming-flash-omni-Previewnative BF16/U8 194 GiB exceeds memory
MiniMax-M2.7Q4_K_M 131 GiB exceeds memory
Mixtral 8x22B Instruct v0.1AWQ 68.6 GiB exceeds memory
MonadContext 8K exceeds this model's configured maximum (2K)
NVIDIA-Nemotron-3-Super-120B-A12B-BF16native F32/BF16 230 GiB exceeds memory
NVIDIA-Nemotron-Labs-3-Competitive-Coding-550B-A55B-NVFP4native modelopt 328 GiB exceeds memory
NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16native BF16/F32 146 GiB exceeds memory
NVIDIA-Nemotron-Labs-Teacher-Chatnative F32/BF16 1,044 GiB exceeds memory
NVIDIA-Nemotron-Labs-Teacher-Competition-Codingnative BF16/F32 1,044 GiB exceeds memory
NVIDIA-Nemotron-Labs-Teacher-Instruction-Followingnative BF16/F32 1,044 GiB exceeds memory
NVIDIA-Nemotron-Labs-Teacher-STEMnative BF16/F32 1,044 GiB exceeds memory
Qwen3-235B-A22BGPTQ 116 GiB exceeds memory
Qwen3-Coder-480B-A35B-InstructQ4_K_M 270 GiB exceeds memory
Qwen3-VL-235B-A22B-InstructQ4_K_M 132 GiB exceeds memory
Qwen3-VL-235B-A22B-ThinkingQ4_K_M 132 GiB exceeds memory
Step-3.7-FlashQ4_K_M 114 GiB exceeds memory
Apertus-v1.5-70Bno verified artifact for these backends

Specs & sources

Memory	64 GB LPDDR5X coherent unified	source
Bandwidth	273 GB/s	source
FP16	unknown	No unambiguous comparable value verified; never substitute AI TOPS, BF16, or whole-system power.
Power	unknown	No unambiguous comparable value verified; never substitute AI TOPS, BF16, or whole-system power.

64GB configuration is exclusively through participating OEM partners, not NVIDIA-branded DGX Spark 128GB. 140W is CPU+GPU SoC TD P; 240W is power supply, neither GPU-only TDP.

Reported speeds

0 observations

No reported observation resolved to this exact device yet.

Open in speed database Submit yours

Doesn't fit? Rent a GPU

RunpodPlaceholderCloud GPU pods you can rent for running larger models.
Vast.aiPlaceholderGPU rental marketplace with many hosts and GPU types.
LambdaPlaceholderCloud GPU instances for AI training and inference.
TensorDockPlaceholderGPU cloud marketplace with consumer and datacenter GPUs.

Sponsored placeholder links: LocalFit may earn a commission if these become affiliate links. They never influence fit results, estimates or picks.