Llama 3.1 70B Instruct Legacy

Released2024-07-23Size71B denseLicenseLlama 3.1 Community License

Files and memory at 8K context

No verified quantized file for the requested formats.

Sizes are exact file bytes (text weights; projectors listed separately). “+” = recurrent state or other terms unknown. Smallest fit adds 0.5 GiB spare; Macs use 75% of memory. Legacy model: sizes on device pages are estimated from parameter count.

Quality

Artificial Analysis Intelligence Index

Unknown

Arena text rating

Unknown

Scores describe the evaluated hosted variant, not each quantized file. AA and Arena are never averaged.

Reported speeds 12

Hardware	Quant	Backend	Prompt t/s	Gen t/s	Date	Source
Mac Studio M1 Ultra (64 core GPU, 128GB)	Q4_K_M	llama.cpp Metal	74.2	9.8	2026-02-07	ai-benchmarks #48
Mac Studio M1 Ultra (64 core GPU, 128GB)	Q4_K_M	llama.cpp Metal	71.9	—	2026-02-07	ai-benchmarks #48
NVIDIA DGX Spark	Q4_K_M	Ollama	—	4.4	2025-10-23	ollama.com
NVIDIA DGX Spark	Q4_K_M	Ollama	—	4.6	2025-10-13	docs.google.com
RTX Pro 6000 Blackwell Edition	Q4_K_M	Ollama	—	32.1	2025-10-13	docs.google.com
Framework Desktop Mainboard (AMD Ryzen AI Max+ 395, 128 GB)	Q4_K_M	llama.cpp Vulkan	34.2	5.0	2025-08-07	ai-benchmarks #21
Framework Desktop Mainboard (AMD Ryzen AI Max+ 395, 128 GB)	Q4_K_M	llama.cpp Vulkan	32.5	—	2025-08-07	ai-benchmarks #21
M4 Max, 40 GPU cores (MBP 16)	Q8_0	llama.cpp Metal	81.7	6.4	2024-11-18	llama.cpp #4167
Mac Studio M3 Ultra 512GB	Q4_K_M	llama.cpp Metal	160	14.1	2025-12-08	ai-benchmarks #42
Mac Studio M3 Ultra 512GB	Q4_K_M	llama.cpp Metal	151	—	2025-12-08	ai-benchmarks #42
NVIDIA DGX Spark	FP8	SGLang	804	2.7	2025-10-13	docs.google.com
RTX Pro 6000 Blackwell Edition	FP8	SGLang	4,696	20.6	2025-10-13	docs.google.com

Reported by the linked sources, not reproduced by LocalFit. All rows in the speed database