mradermacher / hermes-3 family

hermes-3-70b

nous-research/hermes-3-70b

GGUF Provisional scores Updated 20 Sept 2024

Run locally Smallest indexed GGUF / needs Ollama

ollama run hf.co/nous-research/hermes-3-70b/Hermes-3-Llama-3.1-70B.IQ1_S.gguf

Benchmarks

Where it scores.

Methodology →

General Provisional

66.1

Coding Provisional

66.1

Math Provisional

66.1

Reasoning Provisional

66.1

Normalized 0–100 across indexed models.

Runnable variants

Pick a size.

25 indexed

Precision	Format	Size	Params	File
IQ1_S / smallest	GGUF	15.3 GB	—	Hermes-3-Llama-3.1-70B.IQ1_S.gguf
IQ1_M	GGUF	16.8 GB	—	Hermes-3-Llama-3.1-70B.IQ1_M.gguf
IQ2_XXS	GGUF	19.1 GB	—	Hermes-3-Llama-3.1-70B.IQ2_XXS.gguf
IQ2_XS	GGUF	21.1 GB	—	Hermes-3-Llama-3.1-70B.IQ2_XS.gguf
IQ2_S	GGUF	22.2 GB	—	Hermes-3-Llama-3.1-70B.IQ2_S.gguf
IQ2_M	GGUF	24.1 GB	—	Hermes-3-Llama-3.1-70B.IQ2_M.gguf
Q2_K_S	GGUF	24.5 GB	—	Hermes-3-Llama-3.1-70B.Q2_K_S.gguf
Q2_K	GGUF	26.4 GB	—	Hermes-3-Llama-3.1-70B.Q2_K.gguf
IQ3_XXS	GGUF	27.5 GB	—	Hermes-3-Llama-3.1-70B.IQ3_XXS.gguf
IQ3_XS	GGUF	29.3 GB	—	Hermes-3-Llama-3.1-70B.IQ3_XS.gguf
IQ3_S	GGUF	30.9 GB	—	Hermes-3-Llama-3.1-70B.IQ3_S.gguf
Q3_K_S	GGUF	30.9 GB	—	Hermes-3-Llama-3.1-70B.Q3_K_S.gguf
IQ3_M	GGUF	31.9 GB	—	Hermes-3-Llama-3.1-70B.IQ3_M.gguf
Q3_K	GGUF	34.3 GB	—	Hermes-3-Llama-3.1-70B.Q3_K.gguf
Q3_K_L	GGUF	37.1 GB	—	Hermes-3-Llama-3.1-70B.Q3_K_L.gguf
IQ4_XS	GGUF	37.9 GB	—	Hermes-3-Llama-3.1-70B.IQ4_XS.gguf
IQ4_NL	GGUF	40.1 GB	—	Hermes-3-Llama-3.1-70B.IQ4_NL.gguf
Q4_K_S	GGUF	40.3 GB	—	Hermes-3-Llama-3.1-70B.Q4_K_S.gguf
Q4_K	GGUF	42.5 GB	—	Hermes-3-Llama-3.1-70B.Q4_K.gguf
Q5_K_S	GGUF	48.7 GB	—	Hermes-3-Llama-3.1-70B.Q5_K_S/Hermes-3-Llama-3.1-70B.Q5_K_S-00001-of-00003.gguf
Q5_K	GGUF	49.9 GB	—	Hermes-3-Llama-3.1-70B.Q5_K/Hermes-3-Llama-3.1-70B.Q5_K-00001-of-00003.gguf
Q6_K	GGUF	57.9 GB	—	Hermes-3-Llama-3.1-70B.Q6_K/Hermes-3-Llama-3.1-70B.Q6_K-00001-of-00003.gguf
Q8_0	GGUF	75.0 GB	—	Hermes-3-Llama-3.1-70B.Q8_0/Hermes-3-Llama-3.1-70B.Q8_0-00001-of-00004.gguf
BF16	GGUF	141.1 GB	—	Hermes-3-Llama-3.1-70B.BF16/Hermes-3-Llama-3.1-70B.BF16-00001-of-00006.gguf
FP16	GGUF	141.1 GB	—	Hermes-3-Llama-3.1-70B.FP16/Hermes-3-Llama-3.1-70B.FP16-00001-of-00006.gguf

Sizes are file sizes, not peak runtime memory.

Raw benchmarks

The inputs.

1 records

Source	Task	Raw	Normalized
ZeroEval	GPQA	66.1	66.1

From the Hugging Face model card

About this model.

Read it on Hugging Face ↗