01
Collect.
Chatbot Arena and ZeroEval.
Method / fit instrument
How the fit is computed.
Two benchmark sources, one normalized score, one derived memory budget.
02
Normalize.
Raw scores become one 0–100 scale per use case.
03
Fit.
Only variants at or below your memory budget are ranked.
01
Start with live benchmarks.
Scores come from two actively maintained sources. Chatbot Arena captures blind human preference; ZeroEval runs ten automated evaluations across reasoning, math, coding, knowledge, and multimodal tasks.
| Source | Signal |
|---|---|
| Chatbot Arena | Blind pairwise votes from real users |
| ZeroEval | GPQA, HLE, AIME2025, FrontierMath, SWE-bench Verified, SciCode, MMMLU, SimpleQA, MMMU, MMMUPro |
02
Normalize to one scale.
Every benchmark is min-max normalized to 0–100 before combination:
normalized = ((raw − min) / (max − min)) × 100
The unified score for a use case is a weighted average of benchmark-group means. Missing groups are dropped and the remaining weights are renormalized, so a model is not penalized for an evaluation it has not received yet.
S = (w₁ × b̄₁ + w₂ × b̄₂ + … + wₙ × b̄ₙ) / (w₁ + w₂ + … + wₙ)
| Use case | Weights |
|---|---|
| General | Arena 35 / GPQA, MMMLU, HLE 20 / SimpleQA 20 / AIME, FrontierMath 15 / MMMU, MMMUPro 10 |
| Coding | SWE-bench, SciCode 35 / GPQA, AIME 25 / Arena 20 / FrontierMath, HLE 20 |
| RAG | SimpleQA, MMMLU 30 / Arena 30 / GPQA, HLE 20 / MMMU, MMMUPro 20 |
| Roleplay | Arena 50 / MMMLU, HLE 25 / SimpleQA, GPQA 25 |
| Math | AIME, FrontierMath 30 / SciCode, GPQA 25 / HLE 25 / Arena 20 |
| Reasoning | HLE 30 / GPQA, AIME 25 / FrontierMath, MMMUPro 25 / Arena 20 |
03
Fit the memory budget.
The compute budget is not an input. It is derived from the machine you describe, with 2 GB reserved for KV cache and system memory.
Windows / Linux
V_max = V_GPU − 2 GB
macOS / unified memory
V_max = (RAM_total × 0.75) − 2 GB
On a Mac, 75% of total RAM is treated as available to the GPU because unified memory is shared with the rest of the system. On Windows and Linux, the GPU's dedicated VRAM is the ceiling.
A variant is eligible only when size_variant ≤ V_max. Models that cannot load are excluded from the ranking, regardless of score.
04
Pick the variant.
Most models ship at several quantization levels. Your preference decides which eligible file is shown for each model.
Max quality
Highest precision that fits.
max(precision_rank) where size ≤ V_max
Balanced
Closest to 70% utilization, leaving room for context.
min(|size − 0.7 × V_max|)
Max context
Smallest eligible variant, maximizing KV-cache headroom.
min(precision_rank) where size ≤ V_max
05
Rank the shortlist.
Picks are ranked by their unified score for your use case. The displayed variant is the one that fits; size does not change rank.
Trending models appear separately and are not scored against your setup.
Fit test