LocalFit
Submit a benchmark

AMD Radeon RX 7900 GRE: which models fit?

RTX 4090RTX 5090RTX 3090RTX 4060 Ti (16 GB)RTX 5070 TiRadeon RX 7900 XTXM4 Max 128 GBM3 Ultra 512 GBDGX Spark (128 GB)All devices

AMD · GPU

AMD Radeon RX 7900 GRE

Memory16 GB GDDR6Bandwidth576 GB/s vendor specPlanned for models16.0 GiB all VRAM usableFilesGGUF llama.cpp format

Editorial pick · 16 GB GPUsK2-Horizon-7B Q8_0 All picks

Context

Budget = file + KV cache + 1 GiB runtime + 0.5 GiB spare. Macs: 75% of memory. Details

23 models fit Radeon RX 7900 GRE · 8K context

Ranked by Artificial Analysis Intelligence Index, unranked models last. Arena is shown separately. How ranking works

Intelligence

Artificial Analysis Intelligence Index · top 10 models that fit Radeon RX 7900 GRE · higher is better

  1. K2-Horizon-7B20.6
  2. K2-Horizon-3.7B15.6
  3. Qwen3.5-9B13.3
  4. Qwen3.5-4B13.1
  5. MiniCPM5-2B12.5
  6. Granite 4.2 8B11.1
  7. G9v3-3B10.8
  8. Granite 4.2 3B9.1
  9. Tri-21B-Think9.0
  10. MiniCPM5-1B8.8

AA estimate

Generation speed

Tokens per second on Radeon RX 7900 GRE, best file for each of the same models · higher is better

  1. MiniCPM5-1B~279
  2. MiniCPM5-2B~120
  3. G9v3-3B~101
  4. Granite 4.2 3B~82.7
  5. Qwen3.5-4B~71.8
  6. K2-Horizon-3.7B~59.7
  7. Granite 4.2 8B~34.4
  8. Qwen3.5-9B~33.8
  9. K2-Horizon-7B~33.6
  10. Tri-21B-Think~25.6

MeasuredEstimated (~)

WeightsKV cacheRuntimeFreeMeasuredlinks to its sourceEstimatedshown with ~Unknownnever zero
  1. 1

    K2-Horizon-7B

    IFM/K2-Horizon-7B

    2026-09-03size unknown

    AA index20.6rank #33
    Arena—
    Best file & memory
    Q8_08.9 GiB fileFits
    11.0 of 16.0 GiB8K ctx · max ≈40K
    Prompt—Unknown
    Generate~33.6 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file8.92 GiBIFM/K2-Horizon-7B-GGUF HF files
    KV cache · 8K1.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted11.04 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈40K tokens (native 512K).

    Other GGUF files

    • Q4_K_M 5.2 GiB · fits · files
    • Q5_K_M 6.0 GiB · fits · files
    • FP16 16.8 GiB · too big · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 8.9 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 20.6 K2 Horizon 7B · Arena unknown · Model page

  2. 2

    K2-Horizon-3.7B

    IFM/K2-Horizon-3.7B

    2026-09-03size unknown

    AA index15.6rank #41
    Arena—
    Best file & memory
    Q8_05.0 GiB fileFits
    7.1 of 16.0 GiB8K ctx · max ≈67K
    Prompt—Unknown
    Generate~59.7 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file5.02 GiBIFM/K2-Horizon-3.7B-GGUF HF files
    KV cache · 8K1.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted7.14 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈67K tokens (native 512K).

    Other GGUF files

    • Q4_K_M 2.9 GiB · fits · files
    • Q5_K_M 3.4 GiB · fits · files
    • FP16 9.4 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 5.0 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 15.6 K2 Horizon 3.7B · Arena unknown · Model page

  3. 3

    Qwen3.5-9B

    Qwen/Qwen3.5-9B

    2026-03-029B dense

    AA index13.3 AA estimaterank #45
    Arena—
    Best file & memory
    Q8_08.9 GiB fileFits · conditional
    10.1 of 16.0 GiB+8K ctx · max ≈180K
    Prompt~1,773 t/sEstimated
    Generate~33.8 t/sEstimated
    First token · 4K prompt~2.3 sEstimated

    Memory budget

    Weights file8.87 GiBunsloth/Qwen3.5-9B-GGUF HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted10.12 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈180K tokens (native 256K).

    Other GGUF files

    • Q4_K_M 5.3 GiB · fits · files
    • Q5_K_M 6.1 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 8.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 13.3 AA estimate Qwen3.5 9B (Non-reasoning) · Arena unknown · Model page

  4. 4

    Qwen3.5-4B

    Qwen/Qwen3.5-4B

    2026-03-024B dense

    AA index13.1 AA estimaterank #48
    Arena—
    Best file & memory
    Q8_04.2 GiB fileFits · conditional
    5.4 of 16.0 GiB+8K ctx · max ≈256K
    Prompt~3,988 t/sEstimated
    Generate~71.8 t/sEstimated
    First token · 4K prompt~1.0 sEstimated

    Memory budget

    Weights file4.17 GiBunsloth/Qwen3.5-4B-GGUF HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.42 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other GGUF files

    • Q4_K_M 2.6 GiB · fits · files
    • Q5_K_M 2.9 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 4.2 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 13.1 AA estimate Qwen3.5 4B (Reasoning) · Arena unknown · Model page

  5. 5

    MiniCPM5-2B

    openbmb/MiniCPM5-2B

    2026-09-072.5B dense

    AA index12.5rank #51
    Arena—
    Best file & memory
    Q8_02.5 GiB fileFits
    3.8 of 16.0 GiB8K ctx · max ≈128K
    Prompt~6,339 t/sEstimated
    Generate~120 t/sEstimated
    First token · 4K prompt~646 msEstimated

    Memory budget

    Weights file2.50 GiBopenbmb/MiniCPM5-2B-GGUF HF files
    KV cache · 8K0.33 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted3.82 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other GGUF files

    • Q4_K_M 1.5 GiB · fits · files
    • Q5_K_M 1.8 GiB · fits · files
    • FP16 4.7 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 2.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 12.5 MiniCPM5-2B · Arena unknown · Model page

  6. 6

    Granite 4.2 8B

    ibm-granite/granite-4.2-8b

    2026-08-258B dense

    AA index11.1rank #56
    Arena1,320
    Best file & memory
    Q8_08.7 GiB fileFits
    11.0 of 16.0 GiB8K ctx · max ≈37K
    Prompt~1,994 t/sEstimated
    Generate~34.4 t/sEstimated
    First token · 4K prompt~2.1 sEstimated

    Memory budget

    Weights file8.70 GiBibm-granite/granite-4.2-8b-GGUF HF files
    KV cache · 8K1.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted10.95 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈37K tokens (native 128K).

    Other GGUF files

    • Q4_K_M 5.0 GiB · fits · files
    • Q5_K_M 5.8 GiB · fits · files
    • FP16 16.4 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 8.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.1 Granite 4.2 8B · Arena 1,320 · Model page

  7. 7

    G9v3-3B

    AI9Stars/G9v3-3B

    2026-07 (day unknown)3B dense

    AA index10.8 AA estimaterank #58
    Arena—
    Best file & memory
    Q8_03.0 GiB fileFits
    4.4 of 16.0 GiB8K ctx · max ≈128K
    Prompt~5,318 t/sEstimated
    Generate~101 t/sEstimated
    First token · 4K prompt~770 msEstimated

    Memory budget

    Weights file2.96 GiBmradermacher/G9v3-3B-GGUF HF files
    KV cache · 8K0.41 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted4.37 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other GGUF files

    • Q4_K_M 1.7 GiB · fits · files
    • Q5_K_M 2.0 GiB · fits · files
    • FP16 5.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 3.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 10.8 AA estimate G9v3-3B · Arena unknown · Model page

  8. 8

    Granite 4.2 3B

    ibm-granite/granite-4.2-3b

    2026-08-253B dense

    AA index9.1rank #64
    Arena1,293
    Best file & memory
    Q8_03.6 GiB fileFits
    5.3 of 16.0 GiB8K ctx · max ≈128K
    Prompt~5,318 t/sEstimated
    Generate~82.7 t/sEstimated
    First token · 4K prompt~770 msEstimated

    Memory budget

    Weights file3.63 GiBibm-granite/granite-4.2-3b-GGUF HF files
    KV cache · 8K0.63 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted5.25 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other GGUF files

    • Q4_K_M 2.1 GiB · fits · files
    • Q5_K_M 2.4 GiB · fits · files
    • FP16 6.8 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 3.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.1 Granite 4.2 3B · Arena 1,293 · Model page

  9. 9

    Tri-21B-Think

    trillionlabs/Tri-21B-Think

    2026-02 (day unknown)21B dense

    AA index9.0 AA estimaterank #65
    Arena—
    Best file & memory
    Q4_K_M11.7 GiB fileFits
    14.3 of 16.0 GiB8K ctx · max ≈14K
    Prompt~770 t/sEstimated
    Generate~25.6 t/sEstimated
    First token · 4K prompt~5.3 sEstimated

    Memory budget

    Weights file11.72 GiBmykor/Tri-21B-Think-GGUF HF files
    KV cache · 8K1.56 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted14.29 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈14K tokens (native 32K).

    Other GGUF files

    • Q5_K_M 13.7 GiB · tight · files
    • Q8_0 20.5 GiB · too big · files
    • FP16 38.6 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 11.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.0 AA estimate Tri-21B-Think · Arena unknown · Model page

  10. 10

    MiniCPM5-1B

    openbmb/MiniCPM5-1B

    2026-05-191.1B dense

    AA index8.8 AA estimaterank #67
    Arena—
    Best file & memory
    Q8_01.1 GiB fileFits
    2.3 of 16.0 GiB8K ctx · max ≈128K
    Prompt~14,762 t/sEstimated
    Generate~279 t/sEstimated
    First token · 4K prompt~277 msEstimated

    Memory budget

    Weights file1.07 GiBopenbmb/MiniCPM5-1B-GGUF HF files
    KV cache · 8K0.19 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.26 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other GGUF files

    • Q4_K_M 0.6 GiB · fits · files
    • Q5_K_M 0.7 GiB · fits · files
    • FP16 2.0 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.8 AA estimate MiniCPM5-1B (Reasoning) · Arena unknown · Model page

  11. 11

    Nanbeige4.1-3B

    Nanbeige/Nanbeige4.1-3B

    2026-02 (day unknown)3B dense

    AA index8.4 AA estimaterank #69
    Arena—
    Best file & memory
    Q8_03.9 GiB fileFits
    5.4 of 16.0 GiB8K ctx · max ≈170K
    Prompt~5,318 t/sEstimated
    Generate~76.9 t/sEstimated
    First token · 4K prompt~770 msEstimated

    Memory budget

    Weights file3.90 GiBmradermacher/Nanbeige4.1-3B-GGUF HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted5.40 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈170K tokens (native 256K).

    Other GGUF files

    • Q4_K_M 2.3 GiB · fits · files
    • Q5_K_M 2.6 GiB · fits · files
    • FP16 7.3 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.4 AA estimate Nanbeige4.1-3B · Arena unknown · Model page

  12. 12

    LFM2.5-2.6B

    LiquidAI/LFM2.5-2.6B

    2026-08-042.7B dense

    AA index8.4 AA estimaterank #70
    Arena—
    Best file & memory
    Q8_02.7 GiB fileFits · conditional
    3.8 of 16.0 GiB+8K ctx · max ≈128K
    Prompt~5,930 t/sEstimated
    Generate~112 t/sEstimated
    First token · 4K prompt~691 msEstimated

    Memory budget

    Weights file2.68 GiBLiquidAI/LFM2.5-2.6B-GGUF HF files
    KV cache · 8K0.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted3.80 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other GGUF files

    • Q4_K_M 1.6 GiB · fits · files
    • Q5_K_M 1.8 GiB · fits · files
    • FP16 5.0 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 2.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.4 AA estimate LFM2.5-2.6B · Arena unknown · Model page

  13. 13

    Falcon-H1R-7B

    tiiuae/Falcon-H1R-7B

    2026-01-057B dense

    AA index7.8 AA estimaterank #71
    Arena—
    Best file & memory
    Q8_07.5 GiB fileFits · conditional
    8.9 of 16.0 GiB+8K ctx · max ≈163K
    Prompt~2,279 t/sEstimated
    Generate~39.9 t/sEstimated
    First token · 4K prompt~1.8 sEstimated

    Memory budget

    Weights file7.51 GiBtiiuae/Falcon-H1R-7B-GGUF HF files
    KV cache · 8K0.34 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted8.86 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈163K tokens (native 256K).

    Other GGUF files

    • Q4_K_M 4.3 GiB · fits · files
    • Q5_K_M 5.0 GiB · fits · files
    • FP16 14.1 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 7.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.8 AA estimate Falcon-H1R-7B · Arena unknown · Model page

  14. 14

    Step3 VL 10B

    stepfun-ai/Step3-VL-10B

    2026-01 (day unknown)10B dense

    AA index7.7 AA estimaterank #73
    Arena—
    Best file & memory
    Q8_08.1 GiB fileFits
    10.2 of 16.0 GiB8K ctx · max ≈45K
    Prompt~1,595 t/sEstimated
    Generate~36.9 t/sEstimated
    First token · 4K prompt~2.6 sEstimated

    Memory budget

    Weights file8.11 GiBJamePeng2023/Step3-VL-10B-GGUF HF files
    KV cache · 8K1.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted10.24 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈45K tokens (native 64K).

    Other GGUF files

    • Q4_K_M 4.7 GiB · fits · files
    • Q5_K_M 5.4 GiB · fits · files
    • FP16 15.3 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 8.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.7 AA estimate Step3 VL 10B · Arena unknown · Model page

  15. 15

    LFM2.5-8B-A1B

    LiquidAI/LFM2.5-8B-A1B

    2026-05-288.3B · 1.5B active

    AA index7.2 AA estimaterank #74
    Arena—
    Best file & memory
    Q8_08.4 GiB fileFits · conditional
    9.5 of 16.0 GiB+8K ctx · max ≈125K
    Prompt~3,211 t/sEstimated
    Generate~197 t/sEstimated
    First token · 4K prompt~1.3 sEstimated

    Memory budget

    Weights file8.39 GiBLiquidAI/LFM2.5-8B-A1B-GGUF HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted9.49 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈125K tokens (native 125K).

    Other GGUF files

    • Q4_K_M 4.8 GiB · fits · files
    • Q5_K_M 5.6 GiB · fits · files
    • FP16 15.8 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.1047 (n=7) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5564 (n=119) × memory bandwidth ÷ 1.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.2 AA estimate LFM2.5-8B-A1B · Arena unknown · Model page

  16. 16

    NVIDIA Nemotron 3 Nano 4B

    nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16

    2026-03-164B dense

    AA index4.8rank #75
    Arena—
    Best file & memory
    Q8_03.9 GiB fileFits · conditional
    5.1 of 16.0 GiB+8K ctx · max ≈256K
    Prompt~4,018 t/sEstimated
    Generate~76 t/sEstimated
    First token · 4K prompt~1.0 sEstimated

    Memory budget

    Weights file3.94 GiBunsloth/NVIDIA-Nemotron-3-Nano-4B-GGUF HF files
    KV cache · 8K0.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.07 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other GGUF files

    • Q4_K_M 2.6 GiB · fits · files
    • Q5_K_M 2.9 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 4.8 NVIDIA Nemotron 3 Nano 4B · Arena unknown · Model page

  17. 17

    LFM2.5-24B-A2B

    LiquidAI/LFM2.5-24B-A2B

    2026-10-1324B · 2B active

    AA index—unranked
    Arena—
    Best file & memory
    Q4_K_M13.5 GiB fileFits · conditional
    14.7 of 16.0 GiB+8K ctx · max ≈50K
    Prompt~2,408 t/sEstimated
    Generate~265 t/sEstimated
    First token · 4K prompt~1.7 sEstimated

    Memory budget

    Weights file13.53 GiBLiquidAI/LFM2.5-24B-A2B-GGUF HF files
    KV cache · 8K0.16 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted14.68 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈50K tokens (native 128K).

    Other GGUF files

    • Q5_K_M 15.9 GiB · too big · files
    • Q8_0 23.7 GiB · too big · files
    • FP16 44.7 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.1047 (n=7) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5564 (n=119) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  18. 18

    Mellum2.1-12B-A2.5B-Thinking

    JetBrains/Mellum2.1-12B-A2.5B-Thinking

    2026-10-0812B · 2.5B active

    AA index—unranked
    Arena—
    Best file & memory
    Q8_012.0 GiB fileFits
    13.2 of 16.0 GiB8K ctx · max ≈128K
    Prompt~1,926 t/sEstimated
    Generate~119 t/sEstimated
    First token · 4K prompt~2.1 sEstimated

    Memory budget

    Weights file12.04 GiBJetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF HF files
    KV cache · 8K0.15 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted13.19 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other GGUF files

    • Q4_K_M 7.5 GiB · fits · files
    • Q5_K_M 9.3 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.1047 (n=7) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5564 (n=119) × memory bandwidth ÷ 2.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  19. 19

    MiMo-V2.6-Distill-Qwen-9B

    XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B

    2026-09-229B dense

    AA index—unranked
    Arena—
    Best file & memory
    Q8_08.9 GiB fileFits · conditional
    10.1 of 16.0 GiB+8K ctx · max ≈180K
    Prompt~1,773 t/sEstimated
    Generate~33.7 t/sEstimated
    First token · 4K prompt~2.3 sEstimated

    Memory budget

    Weights file8.89 GiBbartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted10.14 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈180K tokens (native 256K).

    Other GGUF files

    • Q4_K_M 5.4 GiB · fits · files
    • Q5_K_M 6.4 GiB · fits · files
    • FP16 16.7 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 8.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  20. 20

    K2-Horizon-0.9B

    IFM/K2-Horizon-0.9B

    2026-09-03size unknown

    AA index—unranked
    Arena—
    Best file & memory
    Q8_01.1 GiB fileFits
    2.5 of 16.0 GiB8K ctx · max ≈128K
    Prompt—Unknown
    Generate~280 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file1.07 GiBIFM/K2-Horizon-0.9B-GGUF HF files
    KV cache · 8K0.44 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.51 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other GGUF files

    • Q4_K_M 0.6 GiB · fits · files
    • Q5_K_M 0.7 GiB · fits · files
    • FP16 2.0 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  21. 21

    Ornith-1.5-9B

    ornith-ai/Ornith-1.5-9B

    2026-08 (day unknown)9B dense

    AA index—unranked
    Arena—
    Best file & memory
    Q8_09.1 GiB fileFits · conditional
    10.4 of 16.0 GiB+8K ctx · max ≈172K
    Prompt~1,773 t/sEstimated
    Generate~32.9 t/sEstimated
    First token · 4K prompt~2.3 sEstimated

    Memory budget

    Weights file9.11 GiBornith-ai/Ornith-1.5-9B-GGUF HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted10.36 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈172K tokens (native 256K).

    Other GGUF files

    • Q4_K_M 5.4 GiB · fits · files
    • Q5_K_M 6.2 GiB · fits · files
    • FP16 16.7 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 9.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  22. 22

    Qwen3.5-0.8B

    Qwen/Qwen3.5-0.8B

    2026-03-020.8B dense

    AA index—unranked
    Arena—
    Best file & memory
    Q8_00.8 GiB fileFits · conditional
    1.8 of 16.0 GiB+8K ctx · max ≈256K
    Prompt~19,941 t/sEstimated
    Generate~396 t/sEstimated
    First token · 4K prompt~205 msEstimated

    Memory budget

    Weights file0.76 GiBunsloth/Qwen3.5-0.8B-GGUF HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted1.85 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other GGUF files

    • Q4_K_M 0.5 GiB · fits · files
    • Q5_K_M 0.5 GiB · fits · files
    • FP16 1.4 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 0.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  23. 23

    Qwen3.5-2B

    Qwen/Qwen3.5-2B

    2026-03-022B dense

    AA index—unranked
    Arena—
    Best file & memory
    Q8_01.9 GiB fileFits · conditional
    3.0 of 16.0 GiB+8K ctx · max ≈256K
    Prompt~7,976 t/sEstimated
    Generate~160 t/sEstimated
    First token · 4K prompt~514 msEstimated

    Memory budget

    Weights file1.87 GiBunsloth/Qwen3.5-2B-GGUF HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted2.97 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other GGUF files

    • Q4_K_M 1.2 GiB · fits · files
    • Q5_K_M 1.3 GiB · fits · files
    • FP16 3.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 1.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

May fit: KV-cache size unknown 4

Weights and runtime fit, but the cache for this attention type is not established. Ranked by AA index.

  1. Gemma 4 12B (it)

    google/gemma-4-12B-it

    2026-06-0312B dense

    AA index14.2 AA estimaterank #44
    Arena—
    Best file & memory
    Q8_011.8 GiB fileCache unknown
    12.8 of 16.0 GiB+8K ctx
    Prompt~1,335 t/sEstimated
    Generate~25.4 t/sEstimated
    First token · 4K prompt~3.1 sEstimated

    Memory budget

    Weights file11.80 GiBbartowski/gemma-4-12B-it-GGUF HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted12.80 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 256K).

    Other GGUF files

    • Q4_K_M 7.1 GiB · cache unknown · files
    • Q5_K_M 8.2 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 11.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 14.2 AA estimate Gemma 4 12B (Reasoning) · Arena unknown · Model page

  2. Ling-3.0-tiny

    inclusionAI/Ling-3.0-tiny

    2026-08 (day unknown)7.9B · 1.3B active

    AA index11.1rank #57
    Arena—
    Best file & memory
    Q8_07.8 GiB fileCache unknown
    8.8 of 16.0 GiB+8K ctx
    Prompt~3,705 t/sEstimated
    Generate~232 t/sEstimated
    First token · 4K prompt~1.1 sEstimated

    Memory budget

    Weights file7.83 GiBinclusionAI/Ling-3.0-tiny-GGUF HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted8.83 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Other GGUF files

    • Q4_K_M 4.5 GiB · cache unknown · files
    • Q5_K_M 5.2 GiB · cache unknown · files
    • FP16 14.7 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.1047 (n=7) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5564 (n=119) × memory bandwidth ÷ 1.3 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.1 Ling 3.0 Tiny · Arena unknown · Model page

  3. Gemma 4 E4B (it)

    google/gemma-4-E4B-it

    2026-04-028B dense

    AA index8.9rank #66
    Arena—
    Best file & memory
    Q8_07.6 GiB fileCache unknown
    8.6 of 16.0 GiB+8K ctx
    Prompt~1,994 t/sEstimated
    Generate~39.3 t/sEstimated
    First token · 4K prompt~2.1 sEstimated

    Memory budget

    Weights file7.63 GiBunsloth/gemma-4-E4B-it-GGUF HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted8.63 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Other GGUF files

    • Q4_K_M 4.6 GiB · cache unknown · files
    • Q5_K_M 5.1 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 7.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.9 Gemma 4 E4B (Reasoning) · Arena unknown · Model page

  4. Gemma 4 E2B

    google/gemma-4-E2B-it

    2026-04-025.1B dense

    AA index7.8 AA estimaterank #72
    Arena—
    Best file & memory
    Q8_04.7 GiB fileCache unknown
    5.7 of 16.0 GiB+8K ctx
    Prompt~3,128 t/sEstimated
    Generate~63.7 t/sEstimated
    First token · 4K prompt~1.3 sEstimated

    Memory budget

    Weights file4.70 GiBunsloth/gemma-4-E2B-it-GGUF HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.70 GiBof 16.0 GiB usable (100% of 16 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Other GGUF files

    • Q4_K_M 2.9 GiB · cache unknown · files
    • Q5_K_M 3.1 GiB · cache unknown · files

    Where the speeds come from

    • Prompt estimated from llama.cpp ROCm median efficiency 0.3468 (n=116) × vendor FP16 TFLOPS ÷ (2 × active params).
    • Generate estimated from llama.cpp ROCm median efficiency 0.5586 (n=117) × memory bandwidth ÷ 4.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.8 AA estimate Gemma 4 E2B (Reasoning) · Arena unknown · Model page

58 models too big for Radeon RX 7900 GRE

Specs & sources

Memory16 GB GDDR6source
Bandwidth576 GB/ssource
FP1692 TFLOPSsource
Power260 Wsource

FP16: Peak Half Precision (FP16 Matrix) Performance (dense); accumulation precision unspecified. Power: Typical Board Power (Desktop).

Doesn't fit? Rent a GPU

  • RunpodPlaceholderCloud GPU pods you can rent for running larger models.
  • Vast.aiPlaceholderGPU rental marketplace with many hosts and GPU types.
  • LambdaPlaceholderCloud GPU instances for AI training and inference.
  • TensorDockPlaceholderGPU cloud marketplace with consumer and datacenter GPUs.

Sponsored placeholder links: LocalFit may earn a commission if these become affiliate links. They never influence fit results, estimates or picks.