Intel Data Center GPU Max 1100: what runs best?

RTX 4090 RTX 5090 RTX 3090 RTX 4060 Ti (16 GB)RTX 5070 Ti Radeon RX 7900 XTX M4 Max 128 GB M3 Ultra 512 GB DGX Spark (128 GB)All devices

Intel · GPU

Intel Data Center GPU Max 1100

Memory48 GB HBM2eBandwidth1,228.8 GB/s vendor specPlussystem RAM for offload (set below)

GPUs

Data Center GPU Max 110048 GB · 1,228.8 GB/s1

Custom GPU

System RAM

InstalledGBTypeSpeedMT/sChannels× 64-bit

≈ 89.6 GB/s theoretical ceiling (MT/s × 8 bytes × channels). DDR5’s two 32-bit subchannels count as one channel.

CPU

PlatformThreads for inference

Used for CPU layers/experts and the -t flag. A reference platform fills its maximum channels and speed; set what you actually populated.

Advanced: reserves, bandwidth, workload, models

Available VRAM per GPUGiB Runtime allowance per deviceGiB Spare margin per deviceGiB Unified memory for models% RAM kept for OS and appsGiB Measured RAM bandwidthGB/s GPU linkLanes Prompt length (first token)tokens Custom contexttokens Parallel sequences

Allow disk-streamed experts (documented cases)Include base checkpointsInclude specialistsInclude community & quant-trained variantsInclude models with unverified release dates

Defaults are editorial planning choices, not measurements: 1 GiB runtime + 0.5 GiB margin per device, 75% of unified memory, 8 GiB RAM for the OS, a vendor “24 GB” treated as 24 GiB (enter your driver’s available VRAM to override), 5 t/s minimum speed (research editorial default). Memory ledger

Data Center GPU Max 110064 GB DDR5-5600 ×2 ch · 56.0 GiB usable · 89.6 GB/s theoretical8K contextF16 KVany backend≥ 5 t/s

Best plans Data Center GPU Max 1100

Best quality at ≥ 5 t/s

Qwen3.8-27B

33.7 AA index Xhigh

Q8_0llama.cpp SYCLFull GPU

~23–36 t/s Estimated extrapolated

Speed extrapolated, not measured on this setup.

Faster tradeoff

K2-Horizon-MoVA-36B-A4B

25.3 AA index single mode

Q8_0llama.cpp SYCLFull GPU

~44–128 t/s Estimated extrapolated

Next point on the quality/speed front; then Qwen3.6-35B-A3B.

Possibly better · speed not established

No higher-scoring model has a feasible plan here.

WeightsKV cacheReserveFreeFits* = a state term unsizedMeasuredlinks to its sourceEstimatedcalibrated or extrapolated range, ~Ceilingupper bound, ≤Unknownnever zero

Meets your 5 t/s floor 37

Best plan per model that reaches your floor by a matching measurement, a calibrated range, or an extrapolated estimate (labelled). Sorted by the selected quality metric, then speed.

1
Qwen3.8-27B
Qwen/Qwen3.8-27B
2026-08-1427B dense
AA index33.7Xhigh
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
28.6 / 48.0 GiB
Generate~23–36 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
2
K2-Horizon-MoVA-36B-A4B
IFM/K2-Horizon-MoVA-36B-A4B
2026-09-0336B MoE · 4B active
AA index25.3single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
40.1 / 48.0 GiB
Generate~44–128 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
3
K2-Horizon-7B
IFM/K2-Horizon-7B
2026-09-039B tensors · dense
AA index20.6single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
11.5 / 48.0 GiB
Generate~64–102 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
4
Qwen3.6-35B-A3B
Qwen/Qwen3.6-35B-A3B
2026-04-1635B MoE · 3B active
AA index18.2Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
36.0 / 48.0 GiB
Generate~71–207 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
5
Muse-Glimmer-30B
meta-models/Muse-Glimmer-30B
2026-08-1030B dense
AA index17.5High
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
29.3 / 48.0 GiB
Generate~22–35 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
6
K2-Horizon-3.7B
IFM/K2-Horizon-3.7B
2026-09-035.1B tensors · dense
AA index15.6single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
7.6 / 48.0 GiB
Generate~109–173 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
7
Qwen3.5-122B-A10B
Qwen/Qwen3.5-122B-A10B
2026-02-24122B MoE · 10B active
AA index15.6Reasoning
Plan & memory
Q4_K_Mllama.cpp SYCLGPU + CPU experts · 19 expert layers on CPUFits*
GPU
46.8 / 48.0 GiB
RAM
27.2 / 56.0 GiB
Generate~7.4–12 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
8
EXAONE-4.5-33B
LGAI-EXAONE/EXAONE-4.5-33B
2026-04-0933B dense
AA index13.2 est.Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
35.5 / 48.0 GiB
Generate~18–29 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
9
Qwen3.5-4B
Qwen/Qwen3.5-4B
2026-03-024B dense
AA index13.1 est.Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
6.1 / 48.0 GiB
Generate~137–218 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
10
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
2026-08-1130B MoE · 3B active
AA index12.9single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
32.8 / 48.0 GiB
Generate~69–198 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
11
granite-4.2-30b
ibm-granite/granite-4.2-30b
2026-08-2530B dense
AA index12.7single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
32.5 / 48.0 GiB
Generate~20–32 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
12
MiniCPM5-2B
openbmb/MiniCPM5-2B
2026-09-072.5B dense
AA index12.5single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits
GPU
6.5 / 48.0 GiB
Generate~125–199 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
13
Hypernova-60B-2605
MultiverseComputingCAI/Hypernova-60B-2605
2026-05-0660B MoE · 4.8B active
Also: HyperNova-60B (listed with its dated update); Hypernova-60B-2602 (dated update, not separately evaluated)
AA index11.7 est.High
Plan & memory
Q5_K_Mllama.cpp SYCLFull GPUFits
GPU
43.0 / 48.0 GiB
Generate~63–182 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
14
Nemotron-Cascade-2-30B-A3B
nvidia/Nemotron-Cascade-2-30B-A3B
2026-03-1930B MoE · 3B active
AA index11.7 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
32.8 / 48.0 GiB
Generate~69–198 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
15
Qwen3.5-9B
Qwen/Qwen3.5-9B
2026-03-029B dense
AA index11.2Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
10.9 / 48.0 GiB
Generate~66–105 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
16
granite-4.2-8b
ibm-granite/granite-4.2-8b
2026-08-258B dense
AA index11.1single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
11.5 / 48.0 GiB
Generate~65–104 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
17
G9v3-3B
ai9stars/G9v3-3B
2026-07 (day unknown)3B dense
AA index10.8 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits
GPU
7.5 / 48.0 GiB
Generate~105–168 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
18
granite-4.2-3b
ibm-granite/granite-4.2-3b
2026-08-253B dense
AA index9.1single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
5.8 / 48.0 GiB
Generate~154–246 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
19
Tri-21B-Think
trillionlabs/Tri-21B-Think
2026-02 (day unknown)21B dense
AA index9.0 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
23.6 / 48.0 GiB
Generate~29–45 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
20
LFM2.5-2.6B
LiquidAI/LFM2.5-2.6B
2026-08-042.7B dense
AA index8.4 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits*
GPU
6.7 / 48.0 GiB
Generate~119–190 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
21
LFM2.5-8B-A1B
LiquidAI/LFM2.5-8B-A1B
2026-05-288.3B MoE · 1.5B active
AA index7.2 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits*
GPU
17.4 / 48.0 GiB
Generate~75–216 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
22
Olmo-3.1-32B-ThinkSpecialist
allenai/Olmo-3.1-32B-Think
2025-12-1232B tensors · dense
AA index7.1 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
34.7 / 48.0 GiB
Generate~18–29 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
23
Qwen3.5-2B
Qwen/Qwen3.5-2B
2026-03-022B dense
AA index6.9 est.Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
3.5 / 48.0 GiB
Generate~306–488 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
24
Olmo-3.1-32B-Instruct
allenai/Olmo-3.1-32B-Instruct
2025-12-1232B tensors · dense
AA index6.5 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
34.7 / 48.0 GiB
Generate~18–29 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
25
Qwen3.5-0.8B
Qwen/Qwen3.5-0.8B
2026-03-020.8B dense
AA index6.1 est.Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
2.4 / 48.0 GiB
Generate~739–1,176 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
26
granite-4.0-h-small
ibm-granite/granite-4.0-h-small
2025-10-0232B tensors · MoE
AA index6.0 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
33.5 / 48.0 GiB
Generate~25–71 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
27
Phi-4Older
microsoft/phi-4
2024-12-1215B tensors · dense
AA index5.9 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits
GPU
30.4 / 48.0 GiB
Generate~22–34 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
28
Olmo-3-7B-ThinkSpecialist
allenai/Olmo-3-7B-Think
2025-11-207.3B tensors · dense
AA index5.6 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
11.2 / 48.0 GiB
Generate~66–105 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
29
Olmo-3-7B-Instruct
allenai/Olmo-3-7B-Instruct
2025-11-207.3B tensors · dense
AA index5.2 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
11.2 / 48.0 GiB
Generate~66–105 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
30
LFM2.5-1.2B-ThinkingSpecialist
LiquidAI/LFM2.5-1.2B-Thinking
2026-01-201.2B tensors · dense
AA index5.2 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits*
GPU
3.8 / 48.0 GiB
Generate~273–434 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
31
LFM2.5-1.2B-Instruct
LiquidAI/LFM2.5-1.2B-Instruct
2026-01-051.2B tensors · dense
AA index5.2 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits*
GPU
3.8 / 48.0 GiB
Generate~273–434 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
32
Apertus-70B-Instruct-2509
swiss-ai/Apertus-70B-Instruct-2509
2025-09-0271B tensors · dense
AA index5.1 est.single mode
Plan & memory
Q4_K_Mllama.cpp SYCLFull GPUFits
GPU
44.7 / 48.0 GiB
Generate~14–23 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
33
Molmo2-8BSpecialist
allenai/Molmo2-8B
2025-12-118.7B tensors · dense
AA index5.0 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits
GPU
17.9 / 48.0 GiB
Generate~38–61 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
34
LFM2.5-VL-1.6BSpecialist
LiquidAI/LFM2.5-VL-1.6B
2026-01-051.6B tensors · dense
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits*
GPU
3.8 / 48.0 GiB
Generate~273–434 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
35
Apertus-8B-Instruct-2509
swiss-ai/Apertus-8B-Instruct-2509
2025-09-028.1B tensors · dense
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUFits
GPU
17.5 / 48.0 GiB
Generate~39–62 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
36
NVIDIA-Nemotron-3-Nano-4B-BF16
nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
2026-03-164B dense
AA index4.8single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
5.6 / 48.0 GiB
Generate~152–242 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
37
K2-Horizon-0.9B
IFM/K2-Horizon-0.9B
2026-09-031.1B tensors · dense
AA index3.0single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
3.0 / 48.0 GiB
Generate~472–751 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown

+ 60 models without a AA index score

LFM2.5-230MFP16 · full gpu · ~1,667 t/s
LFM2.5-350MFP16 · full gpu · ~1,121 t/s
LittleLambFP16 · full gpu · ~804 t/s
LittleLamb-ToolCallingFP16 · full gpu · ~804 t/s
Jais-3-0.9BQ8_0 · full gpu · ~617 t/s
Qwen3-0.6BFP16 · full gpu · ~432 t/s
LFM2.5-1.2B-JPFP16 · full gpu · ~357 t/s
granite-4.0-h-tinyQ8_0 · full gpu · ~297 t/s
granite-4.1-3bQ8_0 · full gpu · ~216 t/s
Kolibri-1Q4_K_M · full gpu · ~212 t/s
Qwen3-1.7BFP16 · full gpu · ~188 t/s
Agents-A1-4BQ8_0 · full gpu · ~185 t/s
FrogNano-4B-2609Q8_0 · full gpu · ~180 t/s
Qwen3-4BQ8_0 · full gpu · ~175 t/s
Mellum2.1-12B-A2.5B-ThinkingQ8_0 · full gpu · ~169 t/s
AgentCPM-ReportQ4_K_M · full gpu · ~167 t/s
MiniCPM-V-4.7-35B-A3BQ8_0 · full gpu · ~146 t/s
Jais-3-3.7BQ8_0 · full gpu · ~143 t/s
Qwen3.5-35B-A3BQ8_0 · full gpu · ~142 t/s
Qwen3-30B-A3BQ8_0 · full gpu · ~119 t/s
Qwen3-Coder-30B-A3B-InstructQ8_0 · full gpu · ~119 t/s
LFM2.5-24B-A2BFP16 · full gpu · ~116 t/s
PaCoRe-8BQ8_0 · full gpu · ~92 t/s
Qwen3-8BQ8_0 · full gpu · ~92 t/s
AgentCPM-ExploreFP16 · full gpu · ~91 t/s
MiMo-V2.6-Distill-Qwen-9BQ8_0 · full gpu · ~88 t/s
Ornith-1.5-9BQ8_0 · full gpu · ~88 t/s
granite-4.1-8bQ8_0 · full gpu · ~85 t/s
Jais-3-7BQ8_0 · full gpu · ~84 t/s
Mistral 7B Instruct v0.3FP16 · full gpu · ~57 t/s
IQuest-Coder-V1-7B-Base-Stage1FP16 · full gpu · ~55 t/s
IQuest-Coder-V1-7B-BaseFP16 · full gpu · ~55 t/s
IQuest-Coder-V1-7B-InstructFP16 · full gpu · ~55 t/s
IQuest-Coder-V1-7B-ThinkingFP16 · full gpu · ~55 t/s
MagenticBrainQ8_0 · full gpu · ~52 t/s
DeepSeek-R1-Distill-Llama-8BFP16 · full gpu · ~51 t/s
RLVR-8B-0926FP16 · full gpu · ~50 t/s
Nemotron-Cascade-8BFP16 · full gpu · ~50 t/s
Mixtral 8x7B Instruct v0.1Q5_K_M · full gpu · ~50 t/s
NVIDIA-Nemotron-Nano-9B-v2-JapaneseFP16 · full gpu · ~48 t/s
Mistral NeMo 12B Instruct (2407)FP16 · full gpu · ~34 t/s
Mistral Small 3.2 24B Instruct (2506)Q8_0 · full gpu · ~33 t/s
Qwen3.6-27BQ8_0 · full gpu · ~30 t/s
IQuest-Coder-V1-14B-Base-Stage1FP16 · full gpu · ~29 t/s
IQuest-Coder-V1-14B-BaseFP16 · full gpu · ~29 t/s
IQuest-Coder-V1-14B-InstructFP16 · full gpu · ~29 t/s
IQuest-Coder-V1-14B-ThinkingFP16 · full gpu · ~29 t/s
Qwen3.5-27BQ8_0 · full gpu · ~29 t/s
Qwen3-14BFP16 · full gpu · ~28 t/s
FrogMini-14B-2510FP16 · full gpu · ~28 t/s
granite-4.1-30bQ8_0 · full gpu · ~27 t/s
Olmo-3-32B-ThinkQ8_0 · full gpu · ~24 t/s
Qwen3-32BQ8_0 · full gpu · ~24 t/s
FrogBoss-32B-2510Q8_0 · full gpu · ~24 t/s
Qwen3-Nemotron-32B-RLBFFQ8_0 · full gpu · ~24 t/s
Jais-3-32BQ8_0 · full gpu · ~22 t/s
IQuest-Coder-V1-40B-ThinkingQ8_0 · full gpu · ~20 t/s
DeepSeek-R1-Distill-Llama-70BQ4_K_M · full gpu · ~19 t/s
Laguna-S-2.1Q5_K_M · gpu + cpu experts · ~6.0 t/s
GLM-4.5-AirQ4_K_M · gpu + cpu experts · ~5.6 t/s

Memory not fully resolved 16

The weights fit, but the KV cache for this attention type (or the chosen cache layout) is not established, so a fit cannot be guaranteed.

1
Ling-3.0-flash-VL
inclusionAI/Ling-3.0-flash-VL
2026-09-09124B MoE · 5.5B active
AA index24.6single mode
Plan & memory
Q4_K_Mllama.cpp SYCLGPU + CPU experts · 15 expert layers on CPUKV unknown
GPU
48.0+ / 48.0 GiB
RAM
26.8 / 56.0 GiB
Generate~15–23 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
2
Ling-3.0-flash-Fin
inclusionAI/Ling-3.0-flash-Fin
2026 (month unknown)124B MoE · 5.1B active
AA index22.6single mode
Plan & memory
Q4_K_Mllama.cpp SYCLGPU + CPU experts · 15 expert layers on CPUKV unknown
GPU
47.5+ / 48.0 GiB
RAM
26.5 / 56.0 GiB
Generate~15–24 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
3
gemma-4-26B-A4B-it
google/gemma-4-26B-A4B-it
2026-04-0225B MoE · 3.8B active
AA index16.7 est.Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUKV unknown
GPU
26.5+ / 48.0 GiB
Generate~57–166 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
4
gemma-4-31B-it
google/gemma-4-31B-it
2026-04-0231B dense
AA index14.7Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUKV unknown
GPU
31.9+ / 48.0 GiB
Generate~20–32 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
5
gemma-4-12B-it
google/gemma-4-12B-it
2026-06-0312B dense
AA index14.2 est.Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUKV unknown
GPU
13.3+ / 48.0 GiB
Generate~52–82 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
6
LongCat-Flash-Lite
meituan-longcat/LongCat-Flash-Lite
2026-01 (day unknown)69B MoE · 3B active
AA index11.5 est.single mode
Plan & memory
Q5_K_Mllama.cpp SYCLFull GPUKV unknown
GPU
46.2+ / 48.0 GiB
Generate~110–319 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
7
Mistral-Small-4-119B-2603
mistralai/Mistral-Small-4-119B-2603
2026-03-16119B MoE · 6B active
AA index11.3Reasoning
Plan & memory
Q4_K_Mllama.cpp SYCLGPU + CPU experts · 12 expert layers on CPUKV unknown
GPU
47.2+ / 48.0 GiB
RAM
22.0 / 56.0 GiB
Generate~9.5–15 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
8
Ling-3.0-tiny
inclusionAI/Ling-3.0-tiny
2026-08 (day unknown)7.9B MoE · 1.3B active
AA index11.1single mode
Plan & memory
FP16llama.cpp SYCLFull GPUKV unknown
GPU
16.2+ / 48.0 GiB
Generate~89–258 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
9
Qwen3-Next-80B-A3B-InstructOlder
Qwen/Qwen3-Next-80B-A3B-Instruct
2025-09-1181B tensors · MoE
AA index9.6 est.single mode
Plan & memory
Q4_K_Mllama.cpp SYCLFull GPUKV unknown
GPU
46.7+ / 48.0 GiB
Generate~70–204 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
10
diffusiongemma-26B-A4B-it
google/diffusiongemma-26B-A4B-it
2026-06-1025B MoE · 3.8B active
AA index9.5 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUKV unknown
GPU
26.5+ / 48.0 GiB
Generate~57–166 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
11
Qwen3-Coder-NextSpecialist
Qwen/Qwen3-Coder-Next
2026-02-0280B MoE · 3B active
AA index9.2single mode
Plan & memory
Q4_K_Mllama.cpp SYCLFull GPUKV unknown
GPU
46.9+ / 48.0 GiB
Generate~127–367 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
12
gemma-4-E4B-it
google/gemma-4-E4B-it
2026-04-028B dense
AA index8.9Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUKV unknown
GPU
9.0+ / 48.0 GiB
Generate~81–129 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
13
gemma-4-E2B-it
google/gemma-4-E2B-it
2026-04-025.1B dense
AA index7.8 est.Reasoning
Plan & memory
Q8_0llama.cpp SYCLFull GPUKV unknown
GPU
6.1+ / 48.0 GiB
Generate~131–209 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
14
Llama 3.3 70B InstructOlderGated
meta-llama/Llama-3.3-70B-Instruct
2024-12-0671B tensors
AA index7.7 est.single mode
Plan & memory
Q4_K_Mllama.cpp SYCLFull GPUKV unknown
GPU
41.1+ / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
15
Phi-4-mini-instructOlder
microsoft/Phi-4-mini-instruct
2025-02-263.8B tensors · dense
AA index6.3 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUKV unknown
GPU
8.7+ / 48.0 GiB
Generate~85–135 t/sEstimatedextrapolated≈ meets 5 t/s
First token · 4K prompt—Unknown
16
tiny-aya-globalGated
CohereLabs/tiny-aya-global
2026-02-173.3B tensors
AA index4.8 est.single mode
Plan & memory
FP16llama.cpp SYCLFull GPUKV unknown
GPU
7.7+ / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown

+ 25 models without a AA index score

DeepSeek-R1-Distill-Qwen-1.5BQ8_0 · full gpu · ~451 t/s
NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16Q5_K_M · full gpu · ~232 t/s
GLM-4.7-FlashQ8_0 · full gpu · ~148 t/s
DeepSeek-R1-Distill-Qwen-7BFP16 · full gpu · ~56 t/s
Qwen2.5-7B-InstructFP16 · full gpu · ~56 t/s
DeepSeek-R1-Distill-Qwen-14BFP16 · full gpu · ~29 t/s
Qwen2.5-14B-InstructFP16 · full gpu · ~29 t/s
Qwen2.5-Coder-7B-InstructFP16 · full gpu · ~28 t/s
DeepSeek-R1-Distill-Qwen-32BQ8_0 · full gpu · ~25 t/s
Qwen2.5-32B-InstructQ8_0 · full gpu · ~25 t/s
Ling-3.0-flashQ4_K_M · gpu + cpu experts · ~20 t/s
Qwen2.5-72B-InstructQ4_K_M · full gpu · ~19 t/s
Qwen2.5-Coder-32B-InstructQ5_K_M · full gpu · ~18 t/s
Gemma 3 12B ITFP16 · full gpu
Gemma 3 1B ITQ8_0 · full gpu
Gemma 3 27B ITQ8_0 · full gpu
Gemma 3 4B ITFP16 · full gpu
Llama 3.1 70B InstructQ4_K_M · full gpu
Llama 3.1 8B InstructQ8_0 · full gpu
Llama 3.2 1B InstructFP16 · full gpu
Llama 3.2 3B InstructFP16 · full gpu
Nemotron-Elastic-12BQ8_0 · full gpu
tiny-aya-earthFP16 · full gpu
tiny-aya-fireFP16 · full gpu
tiny-aya-waterFP16 · full gpu

Speed unknown 6

No measurement, calibration or bandwidth figure for this plan.

1
gpt-oss-120bOlder
openai/gpt-oss-120b
2025-08-05117B tensors · MoE
AA index11.6High
Plan & memory
Q8_0llama.cpp SYCLGPU + CPU layers · 28/36 layers on GPUFits
GPU
47.6 / 48.0 GiB
RAM
13.2 / 56.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
2
gpt-oss-20bOlder
openai/gpt-oss-20b
2025-08-0521B tensors · MoE
AA index9.0High
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
13.0 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
3
granite-4.0-h-1b
ibm-granite/granite-4.0-h-1b
2025-10-281.5B tensors · MoE
AA index5.2 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
3.0 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
4
granite-4.0-micro
ibm-granite/granite-4.0-micro
2025-10-023.4B tensors · MoE
AA index5.1 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
5.5 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
5
granite-4.0-350m
ibm-granite/granite-4.0-350m
2025-10-280.4B tensors · MoE
AA index4.8 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits
GPU
2.1 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown
6
granite-4.0-h-350m
ibm-granite/granite-4.0-h-350m
2025-10-280.3B tensors · MoE
AA index4.8 est.single mode
Plan & memory
Q8_0llama.cpp SYCLFull GPUFits*
GPU
1.9 / 48.0 GiB
Generate—Unknownspeed unknown
First token · 4K prompt—Unknown

+ 4 models without a AA index score

granite-4.0-1bQ8_0 · full gpu
granite-4.0-h-microQ8_0 · full gpu
Jais-3-MoVA-36B-A4BQ8_0 · full gpu
NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-BF16Q8_0 · full gpu

Below 5 t/s 1

Fits, but the best evidence (measured, estimated or the bandwidth ceiling) is under your floor. Lower the floor or add memory bandwidth.

1
Mistral-Medium-3.5-128B
mistralai/Mistral-Medium-3.5-128B
2026-04-28128B dense
AA index14.2single mode
Plan & memory
Q4_K_Mllama.cpp SYCLGPU + CPU layers · 56/88 layers on GPUFits
GPU
47.6 / 48.0 GiB
RAM
26.4 / 56.0 GiB
Generate~0.7–0.8 t/sEstimatedextrapolatedbelow 5 t/s
First token · 4K prompt—Unknown

88 models with no feasible plan on this setup

MiMo-V2.6-Pro-MOPDno backend/format combination
GLM-5.3Q8_0 746 GiB
Step-5-Previewno backend/format combination
Kimi-K3no backend/format combination
GLM-5.3-FlashQ8_0 318 GiB
Ling-3.1-flashQ4_K_M 333 GiB
Qwen3.8-2.4T-A95Bno backend/format combination
Qwen3.8-Flash-NextQ4_K_M 111 GiB
DeepSeek-V4.1-FlashQ4_K_M 414 GiB
Muse-Spark-Open-816B-A42BQ4_K_M 465 GiB exceeds memory
MiMo-V2.6-Flash-MOPDno backend/format combination
DeepSeek-V4-Pro-0813no backend/format combination
Motif-3no backend/format combination
K2-Horizon-375B-A23Bno backend/format combination
Beam-1.0-501B-A23BQ5_K_M 331 GiB
MiniMax-M3Q4_K_M 247 GiB
Nex-N2-ProQ4_K_M 225 GiB
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16No usable expert/non-expert split: the config does not describe it, or it disagrees with the publisher active count (see model page)
Kimi-K2.7-Codeno backend/format combination
Inkling-SmallQ4_K_M 151 GiB
Hy3Q4_K_M 170 GiB
InklingQ8_0 798 GiB
Solar-Open2-250Bno backend/format combination
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16no backend/format combination
G9v3-39A5B (preview)no backend/format combination
A.X-K2no backend/format combination
NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16No usable expert/non-expert split: the config does not describe it, or it disagrees with the publisher active count (see model page)
K-EXAONE-2.0-750B-A37Bno backend/format combination
LongCat-2.0no backend/format combination
Qwen3.5-397B-A17BQ4_K_M 227 GiB
Ring-2.6-1Tno backend/format combination
command-a-plus-05-2026-bf16no backend/format combination
Qwen3-Next-80B-A3B-Thinkingno backend/format combination
Trinity-Large-ThinkingQ4_K_M 225 GiB
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16no backend/format combination
North-Mini-Code-1.0no backend/format combination
Llama 4 Maverick 17B-128E InstructQ4_K_M 226 GiB exceeds memory
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16no backend/format combination
MiniCPM5-1Bno backend/format combination
sarvam-105bno backend/format combination
Nanbeige4.1-3Bno backend/format combination
Falcon-H1R-7Bno backend/format combination
Step3-VL-10Bno backend/format combination
NVIDIA-Nemotron-Nano-12B-v2-VL-BF16no backend/format combination
Llama 4 Scout 17B-16E InstructQ4_K_M 62.9 GiB exceeds memory
Agents-A1no backend/format combination
Agnes-2.5-Pro-Alphano backend/format combination
Agnes-3.0-Qwenno backend/format combination
Apertus-v1.1-0.5B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.1-1.5B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.1-4B-InstructContext 8K exceeds this model's configured maximum (4K)
Apertus-v1.5-8Bno backend/format combination
BaguettotronContext 8K exceeds this model's configured maximum (4K)
DeepSeek-R1Q4_K_M 377 GiB
DeepSeek-V3Q4_K_M 377 GiB
DeepSeek-V4-Flash-0731no backend/format combination
GLM-5.2Q4_K_M 434 GiB
gpt-oss-puzzle-88Bno backend/format combination
Hy3-previewno backend/format combination
Hy4-previewno backend/format combination
IQuest-Coder-V1-40B-Loop-Thinkingno backend/format combination
Jais-3-375B-A23Bno backend/format combination
K2-Horizon-32Bno backend/format combination
Kimi-K2.6no backend/format combination
Llama 2 7Bno backend/format combination
Ming-flash-omni-2.0no backend/format combination
Ming-flash-omni-Previewno backend/format combination
MiniCPM-SALAno backend/format combination
MiniMax-M2.7Q4_K_M 131 GiB
Mixtral 8x22B Instruct v0.1no backend/format combination
MonadContext 8K exceeds this model's configured maximum (2K)
Nemotron-3-Super-120B-A12B-BF16-MTPv2no backend/format combination
NVIDIA-Nemotron-3-Super-120B-A12B-BF16no backend/format combination
NVIDIA-Nemotron-Labs-3-Competitive-Coding-550B-A55B-NVFP4no backend/format combination
NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16no backend/format combination
NVIDIA-Nemotron-Labs-Teacher-Chatno backend/format combination
NVIDIA-Nemotron-Labs-Teacher-Competition-Codingno backend/format combination
NVIDIA-Nemotron-Labs-Teacher-Instruction-Followingno backend/format combination
NVIDIA-Nemotron-Labs-Teacher-STEMno backend/format combination
Penguin-VL-2Bno backend/format combination
Penguin-VL-8Bno backend/format combination
Qwen3-235B-A22BQ4_K_M 132 GiB
Qwen3-Coder-480B-A35B-InstructQ4_K_M 270 GiB
Qwen3-VL-235B-A22B-InstructQ4_K_M 132 GiB
Qwen3-VL-235B-A22B-ThinkingQ4_K_M 132 GiB
Step-3.7-FlashQ4_K_M 114 GiB
Youtu-Parsing-Omnino backend/format combination
Apertus-v1.5-70Bno verified artifact for these backends

Specs & sources

Memory	48 GB HBM2e	source
Bandwidth	1,228.8 GB/s	source
FP16	unknown	No unambiguous comparable value verified; never substitute AI TOPS, BF16, or whole-system power.
Power	300 W	source

Reported speeds

0 observations

No reported observation resolved to this exact device yet.

Open in speed database Submit yours

Doesn't fit? Rent a GPU

RunpodPlaceholderCloud GPU pods you can rent for running larger models.
Vast.aiPlaceholderGPU rental marketplace with many hosts and GPU types.
LambdaPlaceholderCloud GPU instances for AI training and inference.
TensorDockPlaceholderGPU cloud marketplace with consumer and datacenter GPUs.

Sponsored placeholder links: LocalFit may earn a commission if these become affiliate links. They never influence fit results, estimates or picks.