LocalFit
Submit a benchmark

Apple M3 (8-core GPU), 8 GB: which models fit?

RTX 4090RTX 5090RTX 3090RTX 4060 Ti (16 GB)RTX 5070 TiRadeon RX 7900 XTXM4 Max 128 GBM3 Ultra 512 GBDGX Spark (128 GB)All devices

Apple Silicon Mac

Apple M3 (8-core GPU), 8 GB

Memory8 GB unifiedBandwidth100 GB/s vendor specPlanned for models6.0 GiB 75% of unified memoryFilesMLX Apple MLX format

Editorial pick · 8 GB MacsQwen3.5-4B MLX 4-bit All picks

Context
Files

Budget = file + KV cache + 1 GiB runtime + 0.5 GiB spare. Macs: 75% of memory. Details

12 models fit M3 8c GPU · 8 GB · 8K context

Ranked by Artificial Analysis Intelligence Index, unranked models last. Arena is shown separately. How ranking works

Intelligence

Artificial Analysis Intelligence Index · top 9 models that fit M3 8c GPU · 8 GB · higher is better

  1. Qwen3.5-4B13.1
  2. MiniCPM5-2B12.5
  3. G9v3-3B10.8
  4. Granite 4.2 3B9.1
  5. MiniCPM5-1B8.8
  6. Nanbeige4.1-3B8.4
  7. LFM2.5-2.6B8.4
  8. Falcon-H1R-7B7.8
  9. NVIDIA Nemotron 3 Nano 4B4.8

AA estimate

Generation speed

Tokens per second on M3 8c GPU · 8 GB, best file for each of the same models · higher is better

  1. MiniCPM5-1B~71.7
  2. G9v3-3B~48.9
  3. MiniCPM5-2B~30.8
  4. LFM2.5-2.6B~28.7
  5. Qwen3.5-4B~27.1
  6. Granite 4.2 3B~21.2
  7. Nanbeige4.1-3B~19.7
  8. NVIDIA Nemotron 3 Nano 4B~19.5
  9. Falcon-H1R-7B~19.3

MeasuredEstimated (~)

WeightsKV cacheRuntimeFreeMeasuredlinks to its sourceEstimatedshown with ~Unknownnever zero
  1. 1

    Qwen3.5-4B

    Qwen/Qwen3.5-4B

    2026-03-024B dense

    AA index13.1 AA estimaterank #48
    Arena—
    Best file & memory
    MLX 4-bit2.8 GiB fileFits · conditional
    4.1 of 6.0 GiB+8K ctx · max ≈54K
    Prompt~296 t/sEstimated
    Generate~27.1 t/sEstimated
    First token · 4K prompt~14 sEstimated

    Memory budget

    Weights file2.83 GiBmlx-community/Qwen3.5-4B-4bit HF files
    KV cache · 8K0.25 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted4.08 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈54K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 4.8 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.8 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 13.1 AA estimate Qwen3.5 4B (Reasoning) · Arena unknown · Model page

  2. 2

    MiniCPM5-2B

    openbmb/MiniCPM5-2B

    2026-09-072.5B dense

    AA index12.5rank #51
    Arena—
    Best file & memory
    MLX 8-bit2.5 GiB fileFits
    3.8 of 6.0 GiB8K ctx · max ≈49K
    Prompt~471 t/sEstimated
    Generate~30.8 t/sEstimated
    First token · 4K prompt~8.7 sEstimated

    Memory budget

    Weights file2.49 GiBmlx-community/MiniCPM5-2B-8bit HF files
    KV cache · 8K0.33 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted3.82 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈49K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 1.3 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 12.5 MiniCPM5-2B · Arena unknown · Model page

  3. 3

    G9v3-3B

    AI9Stars/G9v3-3B

    2026-07 (day unknown)3B dense

    AA index10.8 AA estimaterank #58
    Arena—
    Best file & memory
    MLX 4-bit1.6 GiB fileFits
    3.0 of 6.0 GiB8K ctx · max ≈58K
    Prompt~395 t/sEstimated
    Generate~48.9 t/sEstimated
    First token · 4K prompt~10 sEstimated

    Memory budget

    Weights file1.57 GiBcof139/G9v3-3B-mlx-4Bit HF files
    KV cache · 8K0.41 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.97 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈58K tokens (native 128K).

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 10.8 AA estimate G9v3-3B · Arena unknown · Model page

  4. 4

    Granite 4.2 3B

    ibm-granite/granite-4.2-3b

    2026-08-253B dense

    AA index9.1rank #64
    Arena1,293
    Best file & memory
    MLX 8-bit3.6 GiB fileFits
    5.2 of 6.0 GiB8K ctx · max ≈11K
    Prompt~395 t/sEstimated
    Generate~21.2 t/sEstimated
    First token · 4K prompt~10 sEstimated

    Memory budget

    Weights file3.62 GiBlmstudio-community/granite-4.2-3b-MLX-8bit HF files
    KV cache · 8K0.63 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted5.25 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈11K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 1.9 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.6 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 9.1 Granite 4.2 3B · Arena 1,293 · Model page

  5. 5

    MiniCPM5-1B

    openbmb/MiniCPM5-1B

    2026-05-191.1B dense

    AA index8.8 AA estimaterank #67
    Arena—
    Best file & memory
    MLX 8-bit1.1 GiB fileFits
    2.3 of 6.0 GiB8K ctx · max ≈128K
    Prompt~1,097 t/sEstimated
    Generate~71.7 t/sEstimated
    First token · 4K prompt~3.7 sEstimated

    Memory budget

    Weights file1.07 GiBmlx-community/MiniCPM5-1B-8bit HF files
    KV cache · 8K0.19 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.26 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈128K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 0.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.8 AA estimate MiniCPM5-1B (Reasoning) · Arena unknown · Model page

  6. 6

    Nanbeige4.1-3B

    Nanbeige/Nanbeige4.1-3B

    2026-02 (day unknown)3B dense

    AA index8.4 AA estimaterank #69
    Arena—
    Best file & memory
    MLX 8-bit3.9 GiB fileFits
    5.4 of 6.0 GiB8K ctx · max ≈10K
    Prompt~395 t/sEstimated
    Generate~19.7 t/sEstimated
    First token · 4K prompt~10 sEstimated

    Memory budget

    Weights file3.89 GiBmlx-community/Nanbeige4.1-3B-8bit HF files
    KV cache · 8K0.50 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted5.39 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈10K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 2.1 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.4 AA estimate Nanbeige4.1-3B · Arena unknown · Model page

  7. 7

    LFM2.5-2.6B

    LiquidAI/LFM2.5-2.6B

    2026-08-042.7B dense

    AA index8.4 AA estimaterank #70
    Arena—
    Best file & memory
    MLX 8-bit2.7 GiB fileFits · conditional
    3.8 of 6.0 GiB+8K ctx · max ≈117K
    Prompt~441 t/sEstimated
    Generate~28.7 t/sEstimated
    First token · 4K prompt~9.3 sEstimated

    Memory budget

    Weights file2.67 GiBLiquidAI/LFM2.5-2.6B-MLX-8bit HF files
    KV cache · 8K0.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted3.79 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈117K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 1.5 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 8.4 AA estimate LFM2.5-2.6B · Arena unknown · Model page

  8. 8

    Falcon-H1R-7B

    tiiuae/Falcon-H1R-7B

    2026-01-057B dense

    AA index7.8 AA estimaterank #71
    Arena—
    Best file & memory
    MLX 4-bit4.0 GiB fileFits · conditional
    5.3 of 6.0 GiB+8K ctx · max ≈12K
    Prompt~169 t/sEstimated
    Generate~19.3 t/sEstimated
    First token · 4K prompt~24 sEstimated

    Memory budget

    Weights file3.98 GiBmlx-community/Falcon-H1R-7B-4bit HF files
    KV cache · 8K0.34 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.32 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈12K tokens (native 256K).

    Other MLX files

    • MLX 8-bit 7.5 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 4.0 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.8 AA estimate Falcon-H1R-7B · Arena unknown · Model page

  9. 9

    NVIDIA Nemotron 3 Nano 4B

    nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16

    2026-03-164B dense

    AA index4.8rank #75
    Arena—
    Best file & memory
    MLX 8-bit3.9 GiB fileFits · conditional
    5.1 of 6.0 GiB+8K ctx · max ≈36K
    Prompt~299 t/sEstimated
    Generate~19.5 t/sEstimated
    First token · 4K prompt~14 sEstimated

    Memory budget

    Weights file3.93 GiBmlx-community/NVIDIA-Nemotron-3-Nano-4B-8bit HF files
    KV cache · 8K0.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.06 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈36K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 2.1 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 4.8 NVIDIA Nemotron 3 Nano 4B · Arena unknown · Model page

  10. 10

    K2-Horizon-0.9B

    IFM/K2-Horizon-0.9B

    2026-09-03size unknown

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit1.1 GiB fileFits
    2.5 of 6.0 GiB8K ctx · max ≈63K
    Prompt—Unknown
    Generate~71.8 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file1.07 GiBmlx-community/K2-Horizon-0.9B-8bit HF files
    KV cache · 8K0.44 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted2.50 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈63K tokens (native 128K).

    Other MLX files

    • MLX 4-bit 0.8 GiB · fits · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 1.1 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  11. 11

    Qwen3.5-0.8B

    Qwen/Qwen3.5-0.8B

    2026-03-020.8B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit0.9 GiB fileFits · conditional
    2.0 of 6.0 GiB+8K ctx · max ≈256K
    Prompt~1,482 t/sEstimated
    Generate~82.2 t/sEstimated
    First token · 4K prompt~2.8 sEstimated

    Memory budget

    Weights file0.93 GiBmlx-community/Qwen3.5-0.8B-8bit HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted2.03 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈256K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 0.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 0.9 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

  12. 12

    Qwen3.5-2B

    Qwen/Qwen3.5-2B

    2026-03-022B dense

    AA index—unranked
    Arena—
    Best file & memory
    MLX 8-bit2.5 GiB fileFits · conditional
    3.6 of 6.0 GiB+8K ctx · max ≈172K
    Prompt~593 t/sEstimated
    Generate~30.9 t/sEstimated
    First token · 4K prompt~6.9 sEstimated

    Memory budget

    Weights file2.48 GiBmlx-community/Qwen3.5-2B-8bit HF files
    KV cache · 8K0.09 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted3.57 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈172K tokens (native 256K).

    Other MLX files

    • MLX 4-bit 1.6 GiB · fits · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 2.5 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA index unknown · Arena unknown · Model page

May fit: KV-cache size unknown 2

Weights and runtime fit, but the cache for this attention type is not established. Ranked by AA index.

  1. Ling-3.0-tiny

    inclusionAI/Ling-3.0-tiny

    2026-08 (day unknown)7.9B · 1.3B active

    AA index11.1rank #57
    Arena—
    Best file & memory
    MLX 4-bit4.1 GiB fileCache unknown
    5.1 of 6.0 GiB+8K ctx
    Prompt~619 t/sEstimated
    Generate~46.7 t/sEstimated
    First token · 4K prompt~6.6 sEstimated

    Memory budget

    Weights file4.14 GiBrapid-mlx/Ling-3.0-tiny-MLX-4bit HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted5.14 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 52 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.3416 (n=94) × memory bandwidth ÷ 0.7 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 11.1 Ling 3.0 Tiny · Arena unknown · Model page

  2. Gemma 4 E2B

    google/gemma-4-E2B-it

    2026-04-025.1B dense

    AA index7.8 AA estimaterank #72
    Arena—
    Best file & memory
    MLX 4-bit3.3 GiB fileCache unknown
    4.3 of 6.0 GiB+8K ctx
    Prompt~232 t/sEstimated
    Generate~23.2 t/sEstimated
    First token · 4K prompt~18 sEstimated

    Memory budget

    Weights file3.31 GiBmlx-community/gemma-4-E2B-it-4bit HF files
    KV cache · 8K—Unknown
    Runtime allowance1.00 GiBplanning assumption Estimated
    Recurrent state—Unknown
    Accounted4.31 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: unknown (native 128K).

    Other MLX files

    • MLX 8-bit 5.5 GiB · too big · files

    Where the speeds come from

    • Prompt estimated from median prefill rate per GPU core of 8 measured M3 points (MLX) × this chip's GPU cores ÷ (2 × active params).
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.3 GiB read per token.
    • First token 4K-token prompt ÷ estimated prefill speed. Excludes model load and queueing.

    Caveats

    • KV-cache size could not be established from the config (compressed or backend-specific attention).
    • Fixed recurrent/convolution state is unknown and not included.
    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 7.8 AA estimate Gemma 4 E2B (Reasoning) · Arena unknown · Model page

Tight: only below 8K context 1

  1. K2-Horizon-3.7B

    IFM/K2-Horizon-3.7B

    2026-09-03size unknown

    AA index15.6rank #41
    Arena—
    Best file & memory
    MLX 4-bit3.9 GiB fileTight
    6.0 of 6.0 GiB8K ctx
    Prompt—Unknown
    Generate~19.9 t/sEstimated
    First token · 4K prompt—Unknown

    Memory budget

    Weights file3.85 GiBmlx-community/K2-Horizon-3.7B-4bit HF files
    KV cache · 8K1.13 GiBFP16 cache from config.json Estimated
    Runtime allowance1.00 GiBplanning assumption Estimated
    Accounted5.98 GiBof 6.0 GiB usable (75% of 8 GB) · 0.5 GiB spare margin required

    Max context in free memory: ≈5K tokens (native 512K).

    Other MLX files

    • MLX 8-bit 5.0 GiB · too big · files

    Where the speeds come from

    • Prompt unknown: no measured report, and no comparable vendor FP16 figure or device calibration.
    • Generate estimated from MLX median efficiency 0.8231 (n=60) × memory bandwidth ÷ 3.9 GiB read per token.
    • First token unknown: needs a prefill value.

    Caveats

    • Text-only budget. A downloadable file does not prove your runtime supports this architecture.

    AA 15.6 K2 Horizon 3.7B · Arena unknown · Model page

56 models too big for M3 8c GPU · 8 GB

Specs & sources

Memory8 GB unifiedsource
Bandwidth100 GB/ssource
FP16unknownApple publishes no comparable FP16 throughput or GPU-only TDP in reviewed chip/device technical specifications. Power adapter rating and whole-system maximum continuous power are not GPU TDP.
PowerunknownApple publishes no comparable FP16 throughput or GPU-only TDP in reviewed chip/device technical specifications. Power adapter rating and whole-system maximum continuous power are not GPU TDP.

Found in: MacBook Air (13-inch, M3, 2024)

Doesn't fit? Rent a GPU

  • RunpodPlaceholderCloud GPU pods you can rent for running larger models.
  • Vast.aiPlaceholderGPU rental marketplace with many hosts and GPU types.
  • LambdaPlaceholderCloud GPU instances for AI training and inference.
  • TensorDockPlaceholderGPU cloud marketplace with consumer and datacenter GPUs.

Sponsored placeholder links: LocalFit may earn a commission if these become affiliate links. They never influence fit results, estimates or picks.