MaziyarPanahi / llama-3.1 family / text-generation
llama-3.1-70b-instruct
meta/llama-3.1-70b-instruct
Run locally Smallest indexed GGUF / needs Ollama
ollama run hf.co/meta/llama-3.1-70b-instruct/Meta-Llama-3.1-70B-Instruct-IQ1_M.gguf Benchmarks
Where it scores.
General Provisional
62.4
Coding Provisional
56.1
Math Provisional
56.1
Reasoning Provisional
56.1
Normalized 0–100 across indexed models.
Runnable variants
Pick a size.
| Precision | Format | Size | Params | File |
|---|---|---|---|---|
| IQ1_M / smallest | GGUF | 16.8 GB | — | Meta-Llama-3.1-70B-Instruct-IQ1_M.gguf |
| IQ2_XXS | GGUF | 19.1 GB | — | Meta-Llama-3.1-70B-Instruct-IQ2_XXS.gguf |
| IQ2_XS | GGUF | 21.1 GB | — | Meta-Llama-3.1-70B-Instruct-IQ2_XS.gguf |
| IQ2_S | GGUF | 22.2 GB | — | Meta-Llama-3.1-70B-Instruct-IQ2_S.gguf |
| IQ2_M | GGUF | 24.1 GB | — | Meta-Llama-3.1-70B-Instruct-IQ2_M.gguf |
| Q2_K | GGUF | 26.4 GB | — | Meta-Llama-3.1-70B-Instruct-Q2_K.gguf |
| Q2_K_L | GGUF | 27.4 GB | — | Meta-Llama-3.1-70B-Instruct-Q2_K_L.gguf |
| IQ3_XS | GGUF | 29.3 GB | — | Meta-Llama-3.1-70B-Instruct-IQ3_XS.gguf |
| Q3_K_S | GGUF | 30.9 GB | — | Meta-Llama-3.1-70B-Instruct-Q3_K_S.gguf |
| IQ3_M | GGUF | 31.9 GB | — | Meta-Llama-3.1-70B-Instruct-IQ3_M.gguf |
| Q3_K_M | GGUF | 34.3 GB | — | Meta-Llama-3.1-70B-Instruct-Q3_K_M.gguf |
| Q3_K_L | GGUF | 37.1 GB | — | Meta-Llama-3.1-70B-Instruct-Q3_K_L.gguf |
| IQ4_XS | GGUF | 37.9 GB | — | Meta-Llama-3.1-70B-Instruct-IQ4_XS.gguf |
| Q3_K_XL | GGUF | 38.1 GB | — | Meta-Llama-3.1-70B-Instruct-Q3_K_XL.gguf |
| Q4_K_S | GGUF | 40.3 GB | — | Meta-Llama-3.1-70B-Instruct-Q4_K_S.gguf |
| Q4_K_M | GGUF | 42.5 GB | — | Meta-Llama-3.1-70B-Instruct-Q4_K_M.gguf |
| Q4_K_L | GGUF | 43.3 GB | — | Meta-Llama-3.1-70B-Instruct-Q4_K_L.gguf |
| Q5_K_S | GGUF | 48.7 GB | — | Meta-Llama-3.1-70B-Instruct-Q5_K_S.gguf |
| Q5_K_M | GGUF | 49.9 GB | — | Meta-Llama-3.1-70B-Instruct-Q5_K_M/Meta-Llama-3.1-70B-Instruct-Q5_K_M-00001-of-00002.gguf |
| Q5_K_L | GGUF | 50.6 GB | — | Meta-Llama-3.1-70B-Instruct-Q5_K_L/Meta-Llama-3.1-70B-Instruct-Q5_K_L-00001-of-00002.gguf |
| Q6_K | GGUF | 57.9 GB | — | Meta-Llama-3.1-70B-Instruct-Q6_K/Meta-Llama-3.1-70B-Instruct-Q6_K-00001-of-00002.gguf |
| Q6_K_L | GGUF | 58.4 GB | — | Meta-Llama-3.1-70B-Instruct-Q6_K_L/Meta-Llama-3.1-70B-Instruct-Q6_K_L-00001-of-00002.gguf |
| Q8_0 | GGUF | 75.0 GB | — | Meta-Llama-3.1-70B-Instruct-Q8_0/Meta-Llama-3.1-70B-Instruct-Q8_0-00001-of-00002.gguf |
Sizes are file sizes, not peak runtime memory.
Raw benchmarks
The inputs.
| Source | Task | Raw | Normalized |
|---|---|---|---|
| Arena | ArenaScore | 1293.4 | 74.2 |
| ZeroEval | GPQA | 41.7 | 41.7 |
From the Hugging Face model card
About this model.
From the Hugging Face model card