16 GiB of VRAM: which local LLMs fit, and what they score
Document version: 1.37 · 20th November 2026 · gpt-oss's IFEval sample closes at n=50, its mean joins the column, and the counts that hang off it move
- Author: Dom Ashworth (github.com/barmynon) · measurements, analysis and writing, with Hermes Agent (Nous Research) driven by DeepSeek
deepseek-flash - Measured: 10th-17th November 2026 · 21 measured rows over 18 weight files (three files carry two rows each), 218.8 GiB of weights plus a 0.43 GiB drafter · one KV setting, one slot
- Axes: HumanEval 164 items · MMLU-Pro 120 · IFEval 50 prompts · needle 2k/8k · riddles 14 · protocol 3
- History of this page: §16 · Wider report on this platform: index.html
A 16 GiB graphics card is the tightest interesting class of consumer accelerator: 16 GiB of VRAM, about 15 GiB usable once the driver and runtime have taken their share, and a hard wall at the end of it. This page answers two questions about that class. Which open-weight models fit, and what do they score. Then: how much of that answer depends on the memory architecture underneath, because the same 13.4 GiB file behaves very differently on a unified-memory APU, a discrete card and an Apple SoC.
Every number here was measured on a machine that does not have the constraint being studied. The reference host is a Minisforum MS-S1 MAX with no VRAM ceiling — a 100 GiB GTT window means nothing is truncated or spilled while it is being benchmarked. That is deliberate: the scores describe the model, the arithmetic describes the card, and the two are never confounded. Where a figure is arithmetic rather than measurement, it says so.
Start here
Four numbers, and everything after this section is how they were produced and where they stop travelling.
| pick | model | mean | footprint | context | in one line |
|---|---|---|---|---|---|
| mean quality, inside the wall | clover-35b-a3b-iq2 | 0.876 | 14.45 GiB | 64k | highest mean that fits a 16 GiB card; autoregressive mode only (§13) |
| value | g4-12b-qat | 0.797 | 7.67 GiB | 32k | a statistical tie with the 26B MoE at half the footprint and twice the context |
| budget | g4-e4b-qat | 0.744 | 5.40 GiB | 32k | the only sane pick below 8 GiB; knowledge is its price (MMLU-Pro 0.467) |
| not 16 GB | huihui-gemma26b-qat-abl | 0.881 | 16.89 GiB | 16k | the best score on the page, and it needs a 24 GB card |
Card choice within the class is bandwidth, not capacity (§12). Quality travels between machines unchanged; prefill and absolute decode do not (§4, Table 9).
One row sits just outside those four and displaces none of them. gpt-oss-20b-mxfp4 now carries a mean of 0.821 on a full IFEval sample, which is sixth of the nineteen rows that have one and fourth among the rows that fit, behind clover-35b-a3b-iq2 (0.876), lfm25-24b-a2b-think (0.861) and axion-1.0-35b-iq3 (0.824). It is the smallest footprint of those four — 12.65 GiB against 14.45, 13.96 and 14.81 — which is the number a 16 GiB card notices. It is not the fastest of them, though: lfm25-24b-a2b-think decodes at 111 tok/s against its 72 and scores four points higher on the mean, so a fifth row here would be a footprint argument rather than a speed one.
What it ran on
Three of the numbers on this page — decode, prefill, and every footprint the fit arithmetic produces — are properties of the stack below as much as of the models, so the stack is stated rather than assumed.
| component | this host | why it is in the table |
|---|---|---|
| CPU | AMD Ryzen AI Max+ 395 (Zen 5), 16 cores / 32 threads, 625 MHz – 5.19 GHz, 64 MiB L3 | prompt handling and sampling run beside the GPU, so prefill is not a pure GPU rate |
| GPU | Radeon 8060S, RDNA 3.5 (gfx1151), 40 CUs | the only compute device; weights and ALUs share one memory, so there is no PCIe hop for a token to cross |
| memory | 128 GiB LPDDR5X-8000, 256-bit, unified: an 8 GiB UMA carve-out reported as VRAM, plus a 100 GiB GTT window (ttm.pages_limit 26,214,400 pages) | sets the ~256 GB/s decode ceiling (§11) and is the memory a footprint is held against; a 13.4 GiB file is resident in it, not paged from disk |
| storage | BIWIN X570 PRO 4TB NVMe, btrfs, /home: 905 GiB of GGUF files under ~/llama-local/models, of which this roster is 218.8 GiB | weights are mmap'd once at load and then sit in the GTT window, so the disk sets load time and nothing else — direct reads measured 0.5-1.9 GB/s on this drive, about 13 s for the largest roster file at that drive's best measured rate, and up to 51 s at the 0.5 GB/s end. A slower disk would move no score on this page |
| OS | CachyOS (Arch-based, rolling), systemd 262, glibc 2.44 | it fixes the kernel build, the Mesa version and the CPU governor below; the same silicon on a stock distribution would be a different stack |
| kernel | 7.3.3 (CachyOS) for seventeen rows, 7.3.4 for the four runs measured on 16 November, across three models — xing4-29b-a4b-iq3xs (two runs), lfm25-24b-a2b-htc and lfm25-24b-a2b-htc-think, which is why the abliterated pair's decode row is a cross-kernel comparison; amdgpu, amd-pstate-epp active, governor powersave with EPP balance_performance | the 100 GiB GTT window comes from a module option in /etc/modprobe.d/ttm.conf, not from a default, and its size decides whether a file stays resident; the pstate mode decides how far the 5.19 GHz boost is held under sustained load |
| graphics stack | Mesa 26.2.6, vulkan-radeon 26.2.6, Vulkan 1.4, RADV (STRIX_HALO) | llama.cpp's Vulkan backend is what executes, so a driver change moves every rate here |
| inference build | llama.cpp fork, b3-1a33095 (0.5.0-dev, commit 1a33095) for thirteen rows; b11581 at tree HEAD 19b458b5a, which adds clover_moe, for seven — the clover row, the two -htc rows and, from v1.25, four of the five round-eight files; and one row, xing4-29b-a4b-iq3xs, on the upstream master-bolt tree at b585-7c2653a. ggml-vulkan ABI 0.25.3, clean trees | three binaries for twenty-one rows: clover needed an architecture the base build does not carry, the two -htc rows and four of the round-eight rows were run after b11581 became the default, and the Xing4 shape (xing4_0) is missing from the fork altogether, so its row is the one not measured on the serving tree |
| server flags | -ngl 99, -c per model, --flash-attn on, --jinja, KV cache q8_0 on both K and V, -b 4096 -ub 2048, one slot (-np 1); speculative decoding off except on the MTP row; reasoning off with a zero budget on lfm25-24b-a2b and on its abliterated twin lfm25-24b-a2b-htc, and left at the template default on the nine rows whose templates open a think block (lfm25-24b-a2b-think, lfm25-24b-a2b-htc-think, groot-19b-a3b, axion-1.0-35b-iq3, clover-35b-a3b-iq2, ornith15-35b-abl-iq3m, holo4-35b-heretic-iq4xs, huihui-qwen36-35b-abl-q5k, and gpt-oss-20b-mxfp4) — all nine were then given a wider cap: 4096 on every axis (gpt-oss-20b-mxfp4 on 20 November), and 8192 on HumanEval for four of them (lfm25-24b-a2b-think, axion-1.0-35b-iq3, clover-35b-a3b-iq2, ornith15-35b-abl-iq3m) and 8192 on MMLU-Pro for clover (§2, Table 3), which is why those rows carry larger caps | one context and batching policy for every row, and the q8_0 KV is why §6 halves the fp16 figure |
| telemetry | the host's own monitor: CPU Tctl/Tdie to 0.1 °C, GPU temperature in 1 °C steps, CPU package power and GPU PPT, sampled every ~2 s, raw kept 72 h and 1-minute rollups for 90 days | every run can be re-read against its temperatures and power draw; the ranges quoted under this table come from it, not from a spot check |
Table 1 — the platform. Every number on this page comes off this stack; the right-hand column says which measurements would move if a part of it changed, and which are properties of the models alone.
Power and temperature are logged, not assumed: the host's own monitor samples CPU and GPU temperature and package power every couple of seconds, so every run can be read back against its thermals. Across the 24 runs long enough to have a measured window, GPU temperature averaged 69 °C against a hottest run average of 79 °C and a hottest single sample of 88 °C, and package power averaged 96 W with a peak of 129 W. No run applied a power cap — the SoC pulls what its firmware allows — so the throughput columns are this box's behaviour rather than a ceiling for the silicon.
The other thing this table does not claim is that the host is representative of the class it measures. It has no 16 GiB wall at all: the card being studied here is a budget applied in arithmetic, not a device, which is exactly why the measurements were taken on it.
Also published: the main report — the wider model matrix on this platform · leaderboards by axis · retired models · change log.
1. The roster
| model | class | weights GiB (bpw) | footprint at its context | context | pp / tg t/s | HumanEval | MMLU-Pro | IFEval | mean quality | quality per GiB | riddles |
|---|---|---|---|---|---|---|---|---|---|---|---|
| g4-e4b-qat | dense | 3.93 (4.52) | 5.40 ok | 32k | 2243 / 58 | 0.866 | 0.467 | 0.900 | 0.744 | 0.138 | 14/14 |
| g4-12b-qat | dense | 6.26 (4.52) | 7.67 ok | 32k | 911 / 28 | 0.896 | 0.575 | 0.920 | 0.797 | 0.104 | 14/14 |
| qwen3-8b-q4 | dense | 4.68 (4.91) | 7.93 ok | 32k | 1300 / 39 | 0.787 | 0.583 | 0.900 | 0.757 | 0.095 | 13/14 |
| qwen3-14b-q4 | dense | 8.38 (4.87) | 10.63 ok | 16k | 743 / 23 | 0.854 | 0.608 | 0.880 | 0.781 | 0.073 | 14/14 |
| gpt-oss-20b-mxfp4 | MoE | 11.28 (4.63) | 12.65 ok | 32k | 2205 / 72 | 0.963 | 0.800 | 0.700 | 0.821 | 0.065 | 14/14 |
| groot-19b-a3b | MoE hybrid | 11.63 (5.26) | 13.76 ok | 64k | 2176 / 65 | 0.939 † | 0.717 | 0.740 | 0.799 | 0.058 | 13/14 |
| lfm25-24b-a2b | MoE | 12.65 (4.53) | 13.96 ok | 32k | 3345 / 110 | 0.640 | 0.192 | 0.860 | 0.564 | 0.040 | 13/14 |
| lfm25-24b-a2b-think | MoE | 12.65 (4.53) | 13.96 ok | 32k | 3360 / 111 | 0.970 | 0.692 | 0.920 | 0.860 | 0.062 | 14/14 |
| lfm25-24b-a2b-htc | MoE | 12.65 (4.53) | 13.96 ok | 32k | 3225 / 106 | 0.689 | 0.267 | 0.820 | 0.592 | 0.042 | 12/14 |
| lfm25-24b-a2b-htc-think | MoE | 12.65 (4.53) | 13.96 ok | 32k | 3331 / 110 | 0.890 | 0.617 | 0.940 | 0.816 | 0.058 | 14/14 |
| clover-35b-a3b-iq2 | MoE diffusion (AR) | 12.32 (3.05) | 14.45 tight | 64k | 1849 / 68 | 0.945 † | 0.842 | 0.840 | 0.876 | 0.061 | 14/14 |
| mistral-24b-q4 | dense | 13.35 (4.86) | 14.66 tight | 4k | 460 / 15 | 0.823 | 0.608 | 0.820 | 0.751 | 0.051 | 12/14 |
| g4-26b-a4b-qat | MoE | 13.45 (4.58) | 14.70 tight | 16k | 1893 / 59 | 0.902 | 0.633 | 0.860 | 0.799 | 0.054 | 14/14 |
| qwen3-30b-a3b-q3 | MoE | 13.70 (3.85) | 14.79 tight | 4k | 1910 / 80 | 0.854 | 0.617 | 0.860 | 0.777 | 0.053 | 14/14 |
| axion-1.0-35b-iq3 | MoE hybrid | 12.69 (3.14) | 14.81 tight | 64k | 1543 / 94 | 0.933 | 0.758 | 0.780 | 0.824 | 0.056 | 14/14 |
| xing4-29b-a4b-iq3xs | MoE | 13.44 (3.70) | 14.91 tight | 32k | 671 / 44 | — | — | — | — | — | 13/14 |
| g4-26b-a4b-qat-mtp | MoE | 13.45 (4.58) | 15.06 NO | 8k | 1886 / 55 | — | — | — | — | — | 14/14 |
| ornith15-35b-abl-iq3m | MoE | 12.94 (3.13) | 15.06 NO | 64k | 1867 / 68 | 0.951 † | 0.792 | 0.800 | 0.848 | 0.056 | 14/14 |
| huihui-gemma26b-qat-abl | MoE | 15.64 (5.32) | 16.89 NO | 16k | 1690 / 58 | 0.970 | 0.833 | 0.840 | 0.881 | 0.052 | 14/14 |
| holo4-35b-heretic-iq4xs | MoE | 16.15 (4.00) | 18.28 NO | 64k | 1780 / 69 | 0.756 † | 0.500 | 0.720 | 0.659 | 0.036 | 14/14 |
| huihui-qwen36-35b-abl-q5k | MoE | 23.61 (5.71) | 25.74 NO | 64k | 1866 / 71 | 0.768 † | 0.583 | 0.820 | 0.724 | 0.028 | 14/14 |
Table 2 — the roster, sorted by footprint. ok = at least 1 GiB of headroom against a 15 GiB budget, tight = fits with less, NO = over budget. Effective bpw is computed from the file size and the parameter count the server reports, not from the file name. Cap policy is per row and per axis in Table 3, which also states each row's sample size. † floor, not a measurement: groot-19b-a3b lost 7 of its 10 HumanEval failures to the 4096 cap, clover-35b-a3b-iq2 lost 4 of 9 to the 8192 cap, ornith15-35b-abl-iq3m lost 7 of its 8 to the 8192 cap, holo4-35b-heretic-iq4xs lost 18 of its 40 to the 4096 cap and huihui-qwen36-35b-abl-q5k lost 35 of its 38 to it, so all five cells understate their models; groot's 8192 re-run is still outstanding. MoE diffusion (AR) is clover's class — a block-diffusion MoE measured under causal, one-token-at-a-time decode, the only mode llama.cpp can run (§13). The MTP row and xing4-29b-a4b-iq3xs have no quality axes: the MTP arm is the same weights as g4-26b-a4b-qat plus a 0.43 GiB draft head, measured for speed and fit only, and the Xing4 row's suite stopped after the perf, needle and riddle axes. Mean quality and quality per GiB are the two derived rows of §2: the first averages the three axes from the unrounded per-item scores, so it can differ by 0.001 from the mean of the printed cells (lfm25-24b-a2b-think is 0.8604 from 159/164, 83/120 and 46/50 and prints 0.860; mistral-24b-q4 is 0.7505 from 135/164, 73/120 and 41/50 and prints 0.751); the second divides that average by the footprint column rather than by the weights, which is the error that makes a big-context model look cheap. Neither is a measurement and neither carries a cap of its own.
The rule that makes any of this a comparison: matched family, matched quant family, one cap, one context policy, one build. The controlled pair is g4-12b-qat (dense) against g4-26b-a4b-qat (MoE) — same tokenizer, same training recipe, both quantisation-aware at ~4.5 effective bpw. A Q4_K_M row against a QAT Q4_0 row would be a quantisation comparison wearing an architecture label. The rows added since v1.21 come from two further architectures: the qwen3.5moe-derived hybrid-attention pair (10 full-attention blocks against 30 linear ones) and, in v1.22, a block-diffusion MoE run autoregressively, which Table 2's class column names MoE diffusion (AR) and §7 and §13 say what to do with. All three hold 64k where the 26B MoE stops at 16k, because the attention layout and not the parameter count sets the KV bill.
2. The axes, the caps and the budget
Every quality number on this page comes from one of seven probes, run here at a stated sample size and a stated token cap. They measure different things and they fail in different ways, so a table of scores is unreadable without knowing which one a column is.
| axis | what the model is asked to do | how it is scored | what was run here | what it does not tell you |
|---|---|---|---|---|
| HumanEval | 164 short Python problems: a docstring and a signature, write the function body | the generated code is executed against unit tests; pass@1 is the fraction passing on the first attempt at temperature 0 | 164 items, complete-program prompt, cap 512 — 8192 on lfm25-24b-a2b-think, axion-1.0-35b-iq3, clover-35b-a3b-iq2 and ornith15-35b-abl-iq3m, 4096 on groot-19b-a3b, the two -htc rows and the three round-eight rows with a code score; 7 of groot's 10 failures, 4 of clover's 9, 7 of ornith15's 8, 18 of holo4's 40 and 35 of qwen36's 38 were cut off at their caps (†) | other languages, multi-file work, debugging, or whether it can drive a tool |
| MMLU-Pro | multiple-choice questions over 14 academic domains, 10 options rather than 4, so guessing pays less than on MMLU | the answer letter is parsed out of a chat completion, 0-shot — no worked examples | 120 items, 0-shot chat probe, cap 512 — 4096 on lfm25-24b-a2b-think, groot-19b-a3b, axion-1.0-35b-iq3, the two -htc rows and the four round-eight rows, 8192 on clover-35b-a3b-iq2 | reasoning depth, recency or calibration; it is a knowledge quiz, not a task |
| IFEval | prompts carrying mechanically checkable constraints — "more than 400 words", "exactly 5 bullet points", "in JSON", "end with this phrase" | code checks each constraint: strict is exact text, loose allows whitespace and quote normalisation; prompt level means every constraint met, instruction level is the per-constraint average | 50 prompts at the task's own 1280-token cap — 4096 on all nine reasoning arms, whose reasoning block spends 1280 before it answers; gpt-oss-20b-mxfp4's own 50-prompt pass at that cap landed on 20 November | whether the content is right, or whether it obeys constraints nobody spelled out |
| prefill and decode (pp / tg) | the same server under load: a 2000-token prompt in, then generation out | tokens per second from the server's own counters, one slot | one measurement per model, one context, one build | batch throughput, power, or what a card would do — §4 and §11 cover transfer |
| riddles | 14 short arithmetic and logic probes | exact match, with a retry at four times the cap when the model ran out mid-answer, so the cap is not scored as a wrong answer | 14 probes at temperature 0.2 | anything statistical: it is a smoke test for gross failure, not an axis |
| needle 2k / 8k | one fact is planted in a filler passage and asked for at the end | exact match: the fact is either returned or it is not | 2k on all twenty-one rows; 8k on the nineteen rows measured at 8k context or more | retrieval past 8k, and a depth curve — the two 4k-context rows are not given the 8k probe at all |
| protocol 3 | three checks that an OpenAI-shaped client can read the reply: non-empty content, a JSON object with one key, a one-word answer | every one of the three must pass | 3/3 on every row | tool loops, multi-turn state, or anything past a single request |
| mean quality (derived) | nothing is run — it is the average of the three quality scores | (HumanEval + MMLU-Pro + IFEval) / 3, computed from the rows above | every model that has all three axes; the MTP and Xing4 rows have none | that a model is good at any one thing. It is a sorting device, not a score, and it hides which axis a model is actually good at |
| quality per GiB (derived) | nothing is run either — it is that mean divided by the model's footprint | mean quality ÷ footprint at its context, in GiB | footprint, not weights: the number the card actually spends | that a model is good. A cheap model can top this rank while scoring badly — which is exactly what the budget pick, g4-e4b-qat, does at 0.744 mean and 0.138 per GiB |
Table 3 — the axes, as they were actually run, plus the two derived numbers the tables lean on. The derived rows have no cap of their own and inherit the caps of what they average; §4 says what a sample of this size is worth.
Cap, budget, and what the budget is spent on
Every number on this page is qualified by one of these, so they are worth fixing before the scores.
- cap — the maximum number of tokens the model was allowed to generate for a single answer (
max_tokensin a chat request,max_gen_toksin lm-eval). It is part of the measurement, not a setting: the same model at cap 512 and at 8192 is two different results, and a deliberating model at a small cap produces no answer at all — the reasoning block eats the budget and the failure reads as a wrong answer rather than a truncated one. Wherever a cap is unusual it is printed in the row it belongs to, because a score without its cap cannot be compared with anything. - budget — the memory a card can actually spend: 15 GiB on a 16 GiB card, after the driver and runtime have taken their share. Weights, the KV cache for the context, and about 1 GiB of runtime overhead all have to fit inside it. A model over budget does not run slowly, it does not run — so fit is settled before speed is discussed, and every table marks it:
ok= at least 1 GiB of headroom left,tight= fits with less,NO= over budget. Three sentences on this page use the word fits and they are not the same test: fits with headroom (theokflag in Table 2), fits at the context it was measured at (§1's context column), and fits at all — the arithmetic behind Table 7 and Figure 2. That is why Table 7 can count a model as fitting at 8k while §1 runs it at 4k. - footprint — how much of that budget a model actually takes: weights, plus KV at the context it is run at, plus the sliding-window attention fixed cost where the architecture carries one (0.03-0.16 GiB at q8_0 on this roster — a hybrid's window buffer does not shrink with context), plus the 1 GiB overhead. Without that third term the arithmetic reads about 0.1 GiB light:
g4-26b-a4b-qatat 16k is 14.70 GiB, not 14.61, andhuihui-gemma26b-qat-ablis 16.89, not 16.80. It is why a 6.26 GiB file can occupy 7.67 GiB, and why the same file at 32k costs more than at 8k. - KV cache — per-conversation memory for attention. It grows linearly with context and is set by the model's attention layout, not its size: on this roster a 14B dense model costs 160 MiB per 1k tokens against a 26B MoE's 20 — attention layout, not size, sets the bill (§6).
- effective bpw — bits per weight, computed from the file size and the parameter count the server reports, never from the file name: 12.32 GiB over 34.7B parameters is 3.05 bpw. It is the number that decides whether a 35B model fits a 16 GiB card at all (§9).
- A4B / E4B — the active parameter count, not the total. A 26B-A4B runs 4B of its 26B for each token, which is why it decodes at 59 tok/s where a 12B dense file manages 28 while carrying half the weights (§7).
- context — three numbers wear the word. The measured context is what a row was run at (Table 2's context column); the affordable context is the largest at which its footprint still fits (Figure 2); the trained length is the ceiling the file was published with, and nothing here ran past it. When this page says a model "holds 64k" it means the first of those.
None of these is a preference. Cap is a fact about the run, budget is arithmetic about the card, footprint and KV are what the arithmetic is spent on, and context is whichever of those three numbers the sentence means.
Terms used on this page
| term | what it means here |
|---|---|
| bpw | bits per weight — file size over parameter count. The roster runs from 3.05 bpw (clover) to 5.71 (huihui-qwen36). |
| QAT | quantisation-aware training: the file was trained with the quantisation in the loop, which is why a 4.5 bpw QAT model holds up where a post-training 4-bit quant of the same family did not (§7). |
| QAD | quantisation-aware distillation — the same idea applied through a teacher: the LFM files are distilled into their quantised form (QAD-Q4_0 in Table 8), so their bpw is part of the training rather than a conversion applied afterwards. |
| KV cache | the per-conversation attention store. Grows with context and is set by the attention layout, not the parameter count (§6). |
| GTT | graphics translation table — the window through which this APU's GPU addresses system memory. This host exposes a 100 GiB GTT window on top of an 8 GiB VRAM carve-out (Table 1). |
| UMA | unified memory architecture: one pool shared by CPU and GPU, as on this APU and on Apple silicon, as against a card's separate VRAM (§11). |
| AR | autoregressive — one token at a time. The only mode llama.cpp can run, including for clover, which was trained for parallel block diffusion (§13). |
| MTP | multi-token prediction: a small draft head that proposes tokens ahead. The roster's single MTP row is a speed and fit arm, not a quality arm (§10). |
| needle | a retrieval probe: one fact planted in a filler passage and asked for at the end. Run at 2k and 8k tokens (§2). |
| pass@1 | the fraction of problems solved on the first attempt, at temperature 0. |
| abliteration | a weight edit that removes a refusal direction. Two suffixes carry it: -htc on the Liquid pair, -abl on the huihui and ornith rows. holo4-35b-heretic-iq4xs carries the word in its upstream id and is referred to by id, never as "the heretic row". §3 tests what abliteration costs. |
Each axis can also be satisfied without doing the thing it is named after — which is the part that makes the roster disagree with itself:
- HumanEval rewards code shape. 0.90 here and the model may still be unable to say why the function works.
- MMLU-Pro rewards discrimination between near-neighbours. Know the topic, miss the distinction, and the score is indistinguishable from knowing neither.
- IFEval rewards compliance. Format it correctly and the marks are yours even when the content is wrong.
That is why the winners differ by axis (§3), and why the mean of the three is only a way to order a shortlist — read the column that matches your workload.
3. What wins, on each axis and overall
Each axis has a different winner, and the model that wins overall is not the one that wins the cheap axis. Counts are in items, not percentages, because that is what the sample supports.
| axis | winner | result | next best | reading |
|---|---|---|---|---|
| HumanEval pass@1 | lfm25-24b-a2b-think | 0.970 (159/164) | gpt-oss-20b-mxfp4 0.963 (158/164) | 1 problem — 0.6 points, deep inside the ±4.4-point interval on the difference: a cluster of two at 0.97, with clover-35b-a3b-iq2 third at 0.945 † |
| MMLU-Pro | clover-35b-a3b-iq2 | 0.842 (101/120) | axion-1.0-35b-iq3 0.758 (91/120) | 10 items — 10 of 120, 8.3 points — the widest gap among the knowledge leaders, though not the widest in the column (that is 65 points, huihui-gemma26b-qat-abl 0.833 down to lfm25-24b-a2b 0.192), and still inside the ±10.1-point interval a two-row comparison needs at n=120: a lead, not a decisive win, and it belongs to a diffusion model run causally |
| IFEval strict | lfm25-24b-a2b-htc-think | 0.940 (47/50) | g4-12b-qat and lfm25-24b-a2b-think at 0.920 (46/50) | one prompt, which is inside the noise — but it is the abliterated arm taking it, and the usual cost of uncensoring is exactly this axis |
| decode | lfm25-24b-a2b-think | 111 tok/s | axion-1.0-35b-iq3 94 tok/s | the budget-0 arm of the same file is at 110; the 35B follows at 94 on the same 12.7 GiB of weights at 3.14 bpw |
| prefill | lfm25-24b-a2b-think | 3360 tok/s | g4-e4b-qat 2243 tok/s | fastest here, and the least portable number on the page; groot-19b-a3b, clover-35b-a3b-iq2 and axion-1.0-35b-iq3 follow at 2176, 1849 and 1543 |
| riddles | fifteen models tied | 14/14 | groot-19b-a3b, qwen3-8b-q4, lfm25-24b-a2b and xing4-29b-a4b-iq3xs at 13/14; mistral-24b-q4 and lfm25-24b-a2b-htc at 12/14 | a smoke test for gross failure, not an axis — 15 of the 21 rows clear it outright (14 of the 19 that carry quality scores; riddles were run on all 21), so it separates nothing above 12/14 |
| quality per GiB (derived, §2) | g4-e4b-qat | 0.138 mean/GiB | g4-12b-qat 0.104 | both beat every row of 12 GiB or more on this measure; the next is 0.062 |
| overall | clover-35b-a3b-iq2 | 0.876 mean | lfm25-24b-a2b-think 0.860 | the mean belongs to the diffusion model, in AR mode: third on code, first on knowledge, and only its IFEval of 0.840 falls outside the top four (10th–11th of 19, tied with huihui-gemma26b-qat-abl) |
Table 4 — the winners. "Overall" is the mean of HumanEval, MMLU-Pro and IFEval, which is a convenience, not a protocol: a coding workload should read the first row and ignore the last, and a knowledge workload the second. Every row in the top half of this table is a wide-cap row — the mean is only comparable within a cap band, and the 512-cap rows cannot win it by construction (§2, §4). Ranked only on rows whose footprint at the context measured in Table 2 is inside 15 GiB: two arms score higher on the mean and do not fit the card — huihui-gemma26b-qat-abl at 0.881 and ornith15-35b-abl-iq3m at 0.848 — so they are the top of the over-budget group, not of the class (§1, Table 7).
What that table means in practice:
- clover-35b-a3b-iq2 is the knowledge leader, the mean leader, and the least quotable row on the page. A block-diffusion MoE — 40 blocks, 256 experts, 34.7B parameters at 3.05 bpw, 12.32 GiB — run by llama.cpp autoregressively, so its 0.945 HumanEval, 0.842 MMLU-Pro and 0.840 IFEval are the causal-decode arm of weights trained for parallel block decode. That is not a hedge: the row describes what these weights do in this harness, not what the architecture is for (§13). On the numbers it leads knowledge by 10 items over
axion-1.0-35b-iq3and takes the mean at 0.876, in a 14.45 GiB footprint that still holds 64k of context —
tight against the 15 GiB wall, as are the five rows below it in Table 2 — the footprint, not the score, is what pushes it out of the ok band.
huihui-gemma26b-qat-ablis the highest mean on the page and does not fit the card. 0.970 HumanEval, 0.833 MMLU-Pro and 0.840 IFEval average 0.881, above clover's 0.876 — and it does it with the reasoning block closed, so unlike the four leaders above it, none of its three cells needed a wide cap. Its 15.64 GiB of weights put the footprint at 16.89 GiB at 16k and 16.82 at 8k: over the wall at every context it was run at, which is why it sits in Table 7's 24 GB rung and not in Table 4.ornith15-35b-abl-iq3mis the same case one step down — 0.848, over the wall by 0.06 only because it was measured at 64k, and 14.08 GiB if it were run at 8k, where no run exists. A 24 GB card changes this table's answer to both of them.- lfm25-24b-a2b with deliberation on is still the model to run for code if the card can spend 14 GiB of its 15. It tops HumanEval, is one prompt behind the abliterated arm on instruction-following, and takes decode and prefill inside a 32k context. What v1.22 changes is who it leads: 4 items over
clover-35b-a3b-iq2(0.945), still inside one interval — so it is first, not decisive, and no longer the highest mean on the page. Two costs are real: it deliberates before answering, so the throughput lead is decode rate and not time-to-answer; and the same weight file with reasoning disabled (lfm25-24b-a2b) is the worst knowledge row in the roster at 0.192. Never quote the file without naming the arm. - lfm25-24b-a2b-htc is the first controlled uncensoring test this class allows, and the folklore is wrong in both directions. Same base checkpoint, same QAD-Q4_0 recipe, a 12.65 GiB file of the same architecture and size: the only difference is the abliteration. Paired on the cap each row carries — the stock arm at 512 HumanEval and MMLU-Pro, where it pinned no HumanEval item, against the
-htcarm at 4096 — the budget-0 twin reads 0.689 / 0.267 / 0.820 against 0.640 / 0.192 / 0.860: better code and knowledge, slightly worse compliance. With deliberation on, both arms sit at 4096 on the two comparable axes: 0.617 MMLU-Pro and 0.940 IFEval against 0.692 and 0.920, so 9 items of knowledge lost and one prompt of compliance gained, while its 0.890 HumanEval at 4096 is not comparable with the stock arm's 0.970, which was measured at 8192. Decode and prefill are inside noise (3331 / 110 against 3360 / 111). What the ablation buys is measured by a probe that is not one of this page's three axes: across 200 benign prompts the stock file refuses 50 outright and hedges 13, the abliterated file refuses none — 25.0% → 0.0%, 63 per-item verdict changes. So it moves the refusal prior and the analysis channel, not the weights' knowledge. No cell of one arm may be quoted for the other: the file name differs by one word, the rows by rather more. - axion-1.0-35b-iq3 was the knowledge pick for one revision, and it is now second by 10 items. 34.7B parameters at 3.14 bpw — the shallowest quant here until clover — took MMLU-Pro outright at 0.758 in v1.21, sits fifth on HumanEval at 0.933, and holds 64k of context in a 14.81 GiB footprint, which is
tightbut real. The counter-intuitive part is in §9: a 3.14 bpw file beats every ~4.5 bpw row on knowledge, and the 3.05 bpw diffusion export beats it in turn, so on this card the parameter count buys more than quantisation depth does, until the file stops fitting. Its IFEval of 0.780 is the third-weakest among the rows that fit at 12 GiB or more —gpt-oss-20b-mxfp4(0.700) andgroot-19b-a3b(0.740) sit below it — andholo4-35b-heretic-iq4xsis lower still at 0.720, on a file that does not fit. - groot-19b-a3b is the context pick. 11.63 GiB of weights, a 13.76 GiB footprint and 64k of context — four
times the context the 26B MoE holds, at a footprint 0.94 GiB smaller (13.76 against 14.70). Its 0.939 HumanEval is a floor rather than a measurement (†, 7 of 10 failures cut off at the cap) and its IFEval of 0.740 is what a thinking arm costs on a constraint-following task. 13/14 riddles, 2176 tok/s prefill, 65 tok/s decode.
- g4-12b-qat is the model to run if half the memory matters. It is joint-first on IFEval, first among the dense rows on HumanEval (0.896, ahead of 0.866 and 0.854), and its mean quality is a statistical tie with the 26B MoE (0.797 against 0.799) at 52% of the footprint and twice the context.
- g4-26b-a4b-qat is no longer the knowledge choice, and v1.21 is what displaced it. Both new rows beat its 0.633 MMLU-Pro —
axion-1.0-35b-iq3by 15 items,groot-19b-a3bby 10 — while fitting four times the context and decoding 1.1-1.6x faster. What it keeps is narrower: a 0.799 mean that tiesgroot-19b-a3bfor eighth of nineteen, and the joint-best IFEval of the rows at 12 GiB or more that were not measured with a reasoning block open (0.860, level withqwen3-30b-a3b-q3andlfm25-24b-a2b). It remains the only QAT 4.58 bpw file at that size. - g4-e4b-qat is the only sane answer below 8 GiB. 5.40 GiB, 0.866 HumanEval, 0.900 IFEval, 32k context. Its MMLU-Pro of 0.467 is the price, and it is a category drop, not a rounding difference.
- Nobody wins on the 4k rows.
mistral-24b-q4andqwen3-30b-a3b-q3were run at 4k context, the largest context at which either keeps any headroom: at 8k their footprints are 14.97 and 14.89 GiB against the 15 GiB wall, and the arithmetic ceiling is about 8k for the Mistral and about 13k for the 30B (Figure 2). The 30B is fast and knowledgeable enough to be interesting, and still loses on usability to a 12B that runs at 32k.
4. How to read the numbers
- Quality does not depend on the GPU. HumanEval, MMLU-Pro, IFEval, the needle probes and the protocol checks are hardware-independent and transfer to a card unchanged (Table 9).
- Decode ranks transfer, absolute rates do not. Decode is bandwidth-bound on bytes read per token on all three architectures this page compares — the unified APU, a discrete card, an Apple SoC — so the ordering survives the hop even though the rates do not: the MS-S1 MAX sits at ~256 GB/s against 288-896 GB/s on a 16 GB card and 546-819 GB/s on Apple silicon.
- Prefill does not transfer at all. It is compute-bound, and an MoE's prefill touches every expert that receives a token, so a discrete card pulls further ahead on exactly the rows where the numbers below look slowest.
- Caps and sample sizes are part of the result. One standard error — a 68% interval, not the whiskers in the figures — is ±2.3 points on HumanEval (164 items at a score of 0.90), ±4.5 on MMLU-Pro (120 at 0.61) and ±3.8 on IFEval (50 prompts at 0.92); twice those bounds a single score at 95%. A comparison between two rows is a different test: it combines both errors, √(SE₁² + SE₂²), and this page reads every cross-row verdict against the 95% interval on that difference — and the band is pair-specific on all three axes: HumanEval ±4.4 at the leaders (0.970/0.945) to ±6.5 at the controlled pair (0.896/0.902); MMLU-Pro ±10.1 for the knowledge leaders (0.842/0.758) to ±12.4 for the controlled pair (0.633/0.575); IFEval ±10.0 for the top pair (0.940/0.920) to ±12.2 for the controlled pair (0.920/0.860). Those six numbers are generated by the gate, not typed. A gap inside those is called a cluster, not a win; §3 and §7 mark the places where an earlier edition called one the other way. These bands are unpaired, which makes them the conservative choice: every model answers the same items, so a paired test on the per-item rows would have less variance than the two independent samples this formula assumes, and the quoted intervals are therefore wider than a paired test would give. No paired test is computed on this page (§14). The band is also pair-specific — widest where the two scores sit near 0.5 and narrowest at the extremes — which is why Table 4's MMLU-Pro pair (0.842 against 0.758) gets ±10.1 while Table 6's pair (0.633 against 0.575) gets ±12 for a smaller gap: not a contradiction, the same formula at two score levels. The whiskers on Figures 1 and 4 are 95% confidence intervals — the range the score would fall in on a fresh sample of the same size, computed as ±1.96 × √(p(1-p)/n) over the item count, which is about double the standard errors quoted here. That is what the "95% CI" legend on Figure 1 and the panel title on Figure 4 mean: the text quotes one standard error, the charts draw 95% intervals, and both say which they are so a number cannot be read in the wrong units. MMLU-Pro here is a 0-shot chat probe, not the 5-shot loglikelihood protocol the public leaderboards use: the column is internally consistent and sits below published scores by design.
5. Quality against what the card has to hold
Figure 1 — Three panels, one per quality axis, each with its own 95% confidence interval (±1.96·√(p(1−p)/n) over the item count) and its own Pareto front — the dotted line, on which no model is both smaller and better. Each front is labelled at its two ends only, with a leader line where the ends sit in a crowded part of the panel; Table 2 carries every point with its footprint. The shaded band and the dashed line mark a 16 GiB card's 15 GiB budget. A point's horizontal position is its footprint — weights plus its KV cache at its context plus the 1 GiB overhead (§2) — so a model with expensive attention sits further right than its file size suggests. Nineteen quality rows exist and sixteen are inside this frame: the horizontal axis ends at 16.6 GiB, so huihui-gemma26b-qat-abl (16.89 GiB, 0.881 mean — the highest on the page), holo4-35b-heretic-iq4xs (18.28, 0.659) and huihui-qwen36-35b-abl-q5k (25.74, 0.724) are plotted in Figure 6 and carried in Table 2, not here. The MTP and Xing4 arms have no quality row at all. Only the vertical axis changes between the three panels.
Two things fall out of Figure 1.
On two of the three axes the best model is not the biggest. The HumanEval front runs from g4-e4b-qat at 5.40 GiB through g4-12b-qat and groot-19b-a3b to lfm25-24b-a2b-think at 13.96 GiB, and the largest footprint among the fitting rows — axion-1.0-35b-iq3 at 14.81 GiB — sits below it. IFEval is the one panel where size stops helping: lfm25-24b-a2b-htc-think holds 0.940 at 13.96 GiB, one prompt ahead of the 7.67 GiB g4-12b-qat, so no smaller file matches it and the front runs all the way to the wall. Knowledge is the exception: there the front ends at clover-35b-a3b-iq2 (14.45 GiB, 0.842), 10 items and 8.3 points above axion-1.0-35b-iq3 at 14.81 GiB — inside the ±10.1-point interval a two-row comparison needs at n=120 (§4), so it is the best-supported ordering on the page and still a lead rather than a win, and it belongs to a diffusion model decoded causally (§13). axion at 0.758 sits below that front, 0.36 GiB further right, so it is dominated rather than merely behind. Both tops are arms rather than files: the HumanEval leader is lfm25-24b-a2b-think, whose same weights with reasoning disabled score 0.640 on code and lose about 50 points of knowledge, and the IFEval leader is the abliterated arm of that file, with the stock arm at 0.920 on 46 of 50 prompts — one prompt behind it, and 0.192 knowledge with the think block closed.
Half the memory buys a statistical tie. g4-12b-qat sits at 7.67 GiB with 32k of context, the 26B MoE at 14.70 GiB with 16k, and their means are 0.797 and 0.799 — two points at almost the same height, twice the footprint apart. §7's controlled pair is the detail.
6. Context is a second budget
Weights are the visible cost. KV cache is the one that decides whether the model is usable, and it is not proportional to model size — it follows the attention layout. §2 gives the roster's two extremes: 160 MiB per 1k tokens for a 14B dense file against 20 for a 26B MoE. Six rows land at 36 MiB per 1k — the five qwen3.5moe-derived files (groot-19b-a3b, axion-1.0-35b-iq3, ornith15-35b-abl-iq3m, holo4-35b-heretic-iq4xs, huihui-qwen36-35b-abl-q5k) and the diffusion export (clover-35b-a3b-iq2) — under a quarter of the 14B dense rate, and that is the whole reason a 35B file holds 64k where a 24B dense file holds 4k. The abliterated LFM pair is not among them: it has the same architecture and size as the 20 MiB row, so its KV rate is identical and Table 5 carries one line for the four LFM arms.
| model | KV MiB per 1k tokens (fp16) | SWA fixed (q8_0, GiB) | 8k (q8_0) | 16k (q8_0) | 32k (q8_0) | 64k (q8_0) |
|---|---|---|---|---|---|---|
| g4-12b-qat | 16 | 0.156 | 0.06 | 0.12 | 0.25 | 0.50 |
| g4-26b-a4b-qat | 20 | 0.098 | 0.08 | 0.16 | 0.31 | 0.62 |
| lfm25-24b-a2b-think | 20 | — | 0.08 | 0.16 | 0.31 | 0.62 |
| huihui-gemma26b-qat-abl | 20 | 0.098 | 0.08 | 0.16 | 0.31 | 0.62 |
| gpt-oss-20b-mxfp4 | 24 | — | 0.09 | 0.19 | 0.38 | 0.75 |
| g4-e4b-qat | 28 | 0.036 | 0.11 | 0.22 | 0.44 | 0.88 |
| xing4-29b-a4b-iq3xs | 30 | — | 0.12 | 0.23 | 0.47 | 0.94 |
| groot-19b-a3b | 36 | — | 0.14 | 0.28 | 0.56 | 1.12 |
| axion-1.0-35b-iq3 | 36 | — | 0.14 | 0.28 | 0.56 | 1.12 |
| clover-35b-a3b-iq2 | 36 | — | 0.14 | 0.28 | 0.56 | 1.12 |
| holo4-35b-heretic-iq4xs | 36 | — | 0.14 | 0.28 | 0.56 | 1.12 |
| huihui-qwen36-35b-abl-q5k | 36 | — | 0.14 | 0.28 | 0.56 | 1.12 |
| ornith15-35b-abl-iq3m | 36 | — | 0.14 | 0.28 | 0.56 | 1.12 |
| qwen3-30b-a3b-q3 | 48 | — | 0.19 | 0.38 | 0.75 | 1.50 |
| qwen3-8b-q4 | 144 | — | 0.56 | 1.12 | 2.25 | 4.50 |
| qwen3-14b-q4 | 160 | — | 0.62 | 1.25 | 2.50 | 5.00 |
| mistral-24b-q4 | 160 | — | 0.62 | 1.25 | 2.50 | 5.00 |
Table 5 — KV cache cost. The per-1k column is the fp16 rate each model's attention layout implies; the SWA fixed column is the sliding-window attention buffer those architectures allocate regardless of context (q8_0, so half the fp16 figure in matrix.json), and it is the third term of the footprint arithmetic in §2; the four context columns are GiB at q8_0 — that fp16 figure halved by the loader's KV quantisation, which is the number the fit arithmetic in Table 2 spends. The two columns are not the same quantity, so a row's 32k value is half of 32 × its per-1k rate, not 32 × it. This is why a 24B dense model is run at 4k on a 16 GiB card while a 35B hybrid MoE runs at 64k. Four roster rows are deliberately absent: the four LFM arms share one row, so the three that are not lfm25-24b-a2b-think (lfm25-24b-a2b, lfm25-24b-a2b-htc and lfm25-24b-a2b-htc-think) are the missing three — the KV belongs to the weights and not to the reasoning flag — and the MTP arm is left out because its draft head carries a KV cache of its own that is not a roster rate (§10 counts it). The 35B hybrid row repeats the 19B's numbers because both are qwen3.5moe hybrids with 10 full-attention blocks against 30 linear ones; clover-35b-a3b-iq2 lands on the same rate for the same structural reason — 10 full-attention blocks, 30 linear — from a different architecture. That layout, not the family, is what makes 64k affordable at these footprints.
Figure 2 — The largest context at which each model still fits the card at all: weights plus KV at that context plus the 1 GiB overhead, against 15 GiB, capped by the trained length. The tick on each bar is the context the roster actually measured at — in most cases the card affords far more than was run, because the roster spends its headroom rather than its ceiling. qwen3-30b-a3b-q3 and mistral-24b-q4 are the two exceptions, and the reason §3 runs them at 4k. Three bars read 0k because their weights alone exceed the budget and no context fits: huihui-gemma26b-qat-abl, holo4-35b-heretic-iq4xs and huihui-qwen36-35b-abl-q5k. One bar is shorter than its tick — g4-26b-a4b-qat-mtp affords 2k and was run at 8k, 0.06 GiB over the wall, which is deliberate and is what §10 is about.
The practical rule: when a model does not fit, cut context first, then KV precision, then quantisation depth, and only then offload. Context is the cheapest thing to give up and the only one that costs no quality at all.
7. Dense or MoE
A dense model runs every parameter for every token. An MoE routes each token to a subset of experts, so a 26B model with 8 of 128 active does roughly the arithmetic of a 4B model while carrying 26B of weights. Footprint follows total parameters; decode speed follows active parameters; the two stop being proportional. Figure 3 draws that trade: dense rows on one band, MoE rows above them at every size, and one block-diffusion MoE that the figure colours as MoE because llama.cpp routes and decodes it the same way — its 67.5 t/s at 12.32 GiB is an MoE rate, not a diffusion one (§13).
Figure 3 — Weights against decode rate, with decode on a log scale, coloured and shaped by class. The dotted line is the Pareto front (no model above it is both smaller and faster); the dashed vertical line at 14 GiB marks where weights alone have spent the budget, once the 1 GiB overhead is inside it. The MoE band sits above the dense band at every size: lfm25-24b-a2b-think decodes at 111 tok/s on 12.65 GiB while mistral-24b-q4 manages 15 on 13.35.
Figure 4 — The controlled pair. Same family, same quantisation-aware ~4.5 bpw training, same caps, one box. Left: the three quality axes, bars carrying 95% confidence intervals at n = 164 / 120 / 50 — the difference on HumanEval is one problem, on MMLU-Pro seven. Right: decode for the same two files at the same cap, 28 against 59 tok/s.
The honest reading is that neither is strictly better:
| axis | g4-12b-qat (dense, 6.26 GiB) | g4-26b-a4b-qat (MoE, 13.45 GiB) | verdict |
|---|---|---|---|
| HumanEval | 0.896 | 0.902 | one problem — tie |
| MMLU-Pro | 0.575 | 0.633 | MoE, 7 items (5.8 points) — inside the ±12.4-point band, not separable |
| IFEval | 0.920 | 0.860 | dense, 3 prompts — inside the ±12.2-point band, so not separable |
| mean quality (derived) | 0.797 | 0.799 | 0.002 — a tie, and it hides opposite directions |
| quality per GiB (derived) | 0.104 | 0.054 | dense, 1.9x — memory is what the tie costs |
| decode | 28 tok/s | 59 tok/s | MoE, 2.1x |
| footprint | 7.67 GiB at 32k | 14.70 GiB at 16k | dense, 2x the context |
Table 6 — the controlled pair. Choose the MoE when knowledge and throughput matter; choose the dense model when instruction-following, context or headroom matter. The two derived rows say what the mean cannot: at 0.002 apart the pair is tied on average quality and still splits 7 items on knowledge against 3 prompts on instruction-following — for 1.9x the memory.
8. What a 16 GiB card can actually run
Figure 5 — How much of this roster fits at 8k context as the card grows: the count of roster rows whose footprint at 8k is inside (card size − 1 GiB), by pure arithmetic. The five rungs are Table 7's five rows, and the shaded band is the 16 GB class this page is about.
| card (GB) | usable (GiB) | models that fit at 8k | best of them by mean quality | what it means |
|---|---|---|---|---|
| 8 GB | 7 GiB | 2 of 21 | qwen3-8b-q4 | 4-8B dense only; every MoE in the roster is out |
| 12 GB | 11 GiB | 4 of 21 | g4-12b-qat | the dense 12B/14B tier arrives; nothing above 10 GiB fits |
| 16 GB | 15 GiB | 17 of 21 | clover-35b-a3b-iq2 | the MoE tier arrives; the mean lead moves to clover-35b-a3b-iq2, in AR mode only, and four rows miss the wall — the MTP arm by 0.06 GiB, huihui-gemma26b-qat-abl by 1.82, holo4-35b-heretic-iq4xs by 2.29 and huihui-qwen36-35b-abl-q5k by 9.75 |
| 24 GB | 23 GiB | 20 of 21 | huihui-gemma26b-qat-abl | everything but huihui-qwen36-35b-abl-q5k fits at 8k, with room for a second model |
| 32 GB+ | 31 GiB+ | 21 of 21 | huihui-gemma26b-qat-abl | this roster saturates; the limit is the roster, not the card |
Table 7 — the ladder. The 24 GB and 32 GB+ rows are roster-limited, not card-limited: the largest file here is 23.61 GiB of weights and the largest footprint is huihui-qwen36-35b-abl-q5k's 25.74 GiB at 8k, so they say nothing about how a 32 GB card behaves with a 30 GiB model. There is no quality-per-GiB column because that metric answers "what is cheapest per gigabyte" rather than "what is the best that fits" — at every rung it elects g4-e4b-qat, the smallest file, since it divides by footprint. "Best of them by mean quality" is the column that answers the question asked, and it inherits Table 4's cap caveat: the 16 GB rung elects clover-35b-a3b-iq2 and the two larger ones elect huihui-gemma26b-qat-abl because their caps are wide, not because the comparison is cap-neutral.
The step from 12 GB to 16 GB is the one that matters, and Figure 5 is that step drawn as counts. It is not a larger dense model that you gain — it is the MoE tier, and with it the eight rows that score above 0.90 HumanEval, four of them at 64k context where the 12 GB card tops out at 0.896 on a 6.26 GiB file.
Figure 6 — The decision map: the same score-against-footprint data as Figure 1, reduced to the mean of the three axes, with the 15 GiB budget drawn as a wall and two stated thresholds across — 0.78, the competitive line, and 0.85, the top of the class. The three rings are the shortlist: the mean pick inside the wall, clover-35b-a3b-iq2 (AR mode only, §13) — not the highest dot on the page, because huihui-gemma26b-qat-abl sits above it at 0.881 in a 16.89 GiB footprint that does not fit, and huihui-qwen36-35b-abl-q5k (25.74 GiB, 0.724) lies beyond the right-hand edge of the frame, the value pick g4-12b-qat at roughly half its footprint, and the budget pick g4-e4b-qat below the line because it costs 5.4 GiB. They are deliberately not the three highest dots. Both lines are stated thresholds, not statistics: eleven models clear the 0.78 line drawn here (0.881, 0.876, 0.860, 0.848, 0.824, 0.821, 0.816, 0.799, 0.799, 0.797, 0.781), three clear 0.85, and at 0.75 fourteen do — out of the nineteen rows that carry a mean. There is no median crosshair and no upper-right "leader" corner, because here smaller footprint and higher score are both good, which a quadrant cannot express.
The map is the one place where the three recommendations can be seen at once: the mean pick (clover-35b-a3b-iq2) as the highest point inside the wall, the value pick (g4-12b-qat) at half its footprint, and the budget pick (g4-e4b-qat) below the line, at 37% of the mean pick's 14.45 GiB footprint (5.40 ÷ 14.45 = 0.374). Of the other sixteen quality rows, nine clear the 0.78 line — huihui-gemma26b-qat-abl (0.881, the highest mean on the page and over the wall), lfm25-24b-a2b-think (0.860), ornith15-35b-abl-iq3m (0.848), axion-1.0-35b-iq3 (0.824), gpt-oss-20b-mxfp4 (0.821), lfm25-24b-a2b-htc-think (0.816), g4-26b-a4b-qat and groot-19b-a3b (0.799 each) and qwen3-14b-q4 (0.781) — and with the two ringed picks that clear it (clover-35b-a3b-iq2 0.876, g4-12b-qat 0.797) that is eleven of the nineteen rows that have a mean. The other seven non-pick rows sit below the line: qwen3-30b-a3b-q3 (0.777), qwen3-8b-q4 (0.757), mistral-24b-q4 (0.751), huihui-qwen36-35b-abl-q5k (0.724), holo4-35b-heretic-iq4xs (0.659), lfm25-24b-a2b-htc (0.592) and lfm25-24b-a2b (0.564); the budget pick is the eighth, and it is the one ringed below the line. The MTP arm has no quality row at all, which makes it the only point on this map that is over the wall and carries no score.
9. Quantisation on a fixed budget
Effective bits per weight is measured from the file, so the ladder below is arithmetic anchored on measured files rather than on vendor claims:
| 25B-class parameters (~25.2B) at | weights | fits inside the 14 GiB left after the overhead? |
|---|---|---|
| 4.6 bpw | 13.5 GiB | yes — and this is exactly what g4-26b-a4b-qat is (13.45 GiB at 4.58 bpw), leaving 0.55 GiB for KV |
| 5.0 bpw | 14.7 GiB | no — 0.7 GiB over, before any KV at all |
| 3.9 bpw | 11.5 GiB | yes, and leaves 2.6 GiB for KV — more than any model in this roster is trained to use |
Table 8 — the quantisation ceiling on a 16 GiB card. Arithmetic from file size and parameter count, at the 25.2B parameters g4-26b-a4b-qat's file implies; the budget is the 15 GiB pool less the 1 GiB overhead, so 14 GiB of weights and nothing else. Everything is GiB, which is why 5.0 bpw reads 14.7 GiB and not 15.6 — that figure is GB.
Two observations from the measured rows rather than the arithmetic. First, QAT pays: the quantisation-aware g4-12b-qat reaches 0.896 HumanEval at 4.52 bpw, which no 4-bit post-training quant in this roster matches at that footprint. Second, going deeper costs knowledge unevenly — the 3.85 bpw 30B keeps 0.617 MMLU-Pro, but at 30.6B parameters that file is 13.70 GiB, which leaves 0.30 GiB of KV room: about 13k tokens at its 24 MiB per 1k rate at q8_0 (Table 5 lists the fp16 rate, 48), and no headroom at all. That is why it ran at 4k, and why the trade it offers on this card is context, not accuracy.
The counterexample keeps getting stronger, and it is the more useful half of the answer. axion-1.0-35b-iq3 at 3.14 bpw kept 0.758 MMLU-Pro in v1.21, ahead of every ~4.5 bpw row; v1.22's clover-35b-a3b-iq2 runs at 3.05 bpw and beats that by 10 items at 0.842, the highest knowledge score on the page. So on this card the parameter count buys knowledge that quantisation depth does not take away, at least down to 3 bpw: what the deep quants cost is not accuracy but footprint pressure. Axion's 12.69 GiB of weights leave 1.31 GiB for KV, which is 74k tokens at its 18 MiB per 1k q8_0 rate — generous, and the reason the row is tight at 64k rather than ok. Whether either file at 2 bpw still holds that line is untested here.
10. When it does not fit
The MTP row is the useful failure: 15.06 GiB at 8k context, over budget by 0.06 GiB. That is what the wall looks like — not a crash, but a model that loads and then spends its life at the edge of the pool. In order of cost:
- Cut context. Halving 32k to 16k returns 0.1-1.25 GiB depending on the model's attention layout (Table 5), and costs nothing in quality for prompts that fit.
- Quantise the KV cache. q8_0 is already assumed throughout; q4_0 halves it again at a small quality cost and is the standard second lever.
- Drop the drafter. A speculative-decoding head costs weights, and in general a second KV cache; on this roster the head is 0.43 GiB, and that is the whole difference between the two rows at the same 8k context:
g4-26b-a4b-qatis 14.63 GiB there and the MTP row 15.06, so the drafter alone costs 0.43 GiB and takes it over the wall by 0.06. Table 2 lists the same pair at 16k and 8k respectively — 14.70 and 15.06 — which is why the two numbers differ between the table and this section. Dropping it recovers exactly that, and costs no quality: the weights are unchanged. - Drop quantisation depth. One step down is roughly 0.5-1 GiB on a 25B model. It is the only lever that changes what the model knows.
- Offload layers.
-nglbelow the layer count moves layers to CPU, which costs PCIe or system-memory bandwidth on every token — the 5-10x decode collapse that a discrete card shows on spill.
Every row above ran with the same flags, listed once in Table 1's server-flags row: -ngl 99 (all layers on the GPU), -c at the context measured rather than the model's maximum, --flash-attn on, --cache-type-k q8_0 --cache-type-v q8_0, -np 1 (one slot — concurrency multiplies KV), -b 4096 -ub 2048 for prefill batching. None of them is exotic; the discipline is that each one changes what fits, which is the list above.
11. Three memory architectures
Figure 7 — Peak memory bandwidth on a log scale, with the PCIe spill path for comparison. Bandwidth sets decode; whether there is a hard ceiling at all decides what runs. Bandwidth is quoted in decimal GB/s, as vendors publish it, while capacities on this page are GiB: 256 GB/s is 238 GiB/s.
- Unified APU (the MS-S1 MAX). 128 GiB of LPDDR5X-8000 on a 256-bit bus, ~256 GB/s, with a 100 GiB GTT window on top of an 8 GiB carve-out. No wall to cross, so a 40 GiB model runs — at the speed 256 GB/s allows.
- Discrete card. 24 GB at 1008 GB/s or 32 GB at 1792 GB/s: four to seven times the bandwidth on a fifth of the capacity. Over the VRAM limit, layers spill to host memory across PCIe (~64 GB/s on a 5.0 x16 link) and decode falls by roughly an order of magnitude. In this class the card is not capacity-constrained at all — every row but the 23.61 GiB Q5_K file fits at 24 GB (Table 7).
- Apple Silicon. Unified like the APU, with a wider bus: 546 GB/s on an M4 Max, 819 GB/s on an M3 Ultra, capacities from 36 GB to 512 GB. No hard wall, a driver-capped GPU share, and a Metal rather than CUDA/ROCm software path.
| quantity | transfers? | why |
|---|---|---|
| quality scores | yes, unchanged | depends on weights, quant, cap and prompt protocol only |
| decode ranking | yes | bandwidth-bound on bytes read per token on all three architectures, so the ordering holds |
| decode absolute rate | no | scale by bandwidth: 256 GB/s here against 1008 GB/s on a 24 GB card and 1792 GB/s on a 32 GB one |
| prefill ranking | partly | dense-to-dense mostly holds; MoE prefill is understated here |
| prefill absolute rate | no | a card has far more compute per byte of bandwidth |
| fit | no | arithmetic against the target's own memory ceiling |
| needle, riddles, protocol | yes | nothing in the path touches the GPU pool |
Table 9 — what may be carried from this page to another machine, and what may not.
12. Which 16 GiB card, and what it implies
Every card in this class has the same 16 GB of VRAM, so capacity decides nothing between them — bandwidth decides everything, and within the class bandwidth spans 3.1x. That single number is the purchase decision. The spread is not a brand story: AMD's seven 16 GB boards run from 288 to 640 GB/s, NVIDIA's four from 288 to 896, Intel's one entry sits at 560.
| card | memory | peak bandwidth | 26B-A4B MoE, predicted | 12B dense, predicted |
|---|---|---|---|---|
| Minisforum MS-S1 MAX (reference) | unified LPDDR5X-8000, 256-bit | 256 GB/s | 59 tok/s measured | 28 tok/s measured |
| NVIDIA RTX 4060 Ti 16 GB | 128-bit GDDR6, 18 Gbps | 288 GB/s | 44-69 | 24-33 |
| AMD RX 7600 XT 16 GB | 128-bit GDDR6, 18 Gbps | 288 GB/s | 44-69 | 24-33 |
| AMD RX 9060 XT 16 GB | 128-bit GDDR6, 20 Gbps | 320 GB/s | 49-77 | 26-37 |
| NVIDIA RTX 5060 Ti 16 GB | 128-bit GDDR7, 28 Gbps | 448 GB/s | 68-108 | 37-52 |
| AMD RX 6800 16 GB | 256-bit GDDR6, 16 Gbps | 512 GB/s | 78-123 | 42-59 |
| Intel Arc A770 16 GB | 256-bit GDDR6, 17.5 Gbps | 560 GB/s | 86-134 | 46-65 |
| AMD RX 6950 XT 16 GB | 256-bit GDDR6, 18 Gbps | 576 GB/s | 88-138 | 47-67 |
| AMD RX 7900 GRE 16 GB | 256-bit GDDR6, 18 Gbps | 576 GB/s | 88-138 | 47-67 |
| AMD RX 7800 XT 16 GB | 256-bit GDDR6, 19.5 Gbps | 624 GB/s | 95-150 | 51-72 |
| AMD RX 9070 16 GB | 256-bit GDDR6, 20 Gbps | 640 GB/s | 98-154 | 52-74 |
| NVIDIA RTX 4070 Ti Super 16 GB | 256-bit GDDR6X, 21 Gbps | 672 GB/s | 103-161 | 55-78 |
| NVIDIA RTX 5070 Ti 16 GB | 256-bit GDDR7, 28 Gbps | 896 GB/s | 137-215 | 73-104 |
Table 10 — 16 GB cards, vendor bandwidth, and the decode range it implies for the two models that matter here: the 13.45 GiB MoE (2.29 GB read per token) and the 6.26 GiB dense 12B (6.73 GB read per token). Bandwidth is the peak published in each card's own specification, in decimal GB/s as vendors quote it, against GiB capacities elsewhere on the page — a card sold as 16 GB has 14.9 GiB, and the 15 GiB budget used throughout is the usable share of that; every predicted figure is an extrapolation, not a measurement, and only the reference row is measured. Cards are grouped by bandwidth, not by brand.
How the prediction is made, so it can be checked and discounted: decode is bandwidth-bound, so predicted = bandwidth / bytes-read-per-active-token × efficiency, where efficiency is the fraction of the ceiling the hardware actually reaches. The MS-S1 MAX measures 53% of its own ceiling on the MoE (59 of 111 tok/s) and 73% on the dense model (28 of 38) — and 43% on axion-1.0-35b-iq3, which reads 1.18 GB per active token at 3.14 bpw (94 of 217). That third row is not in Table 10: its band would sit between the two and argue the same point, and every figure in it is an extrapolation from the same efficiency model. The bands in Table 10 use 35-55% for the MoE and 55-78% for the dense model — deliberately wide, because a discrete card's kernels and a 128-bit card's higher latency per byte will not match the MS-S1 MAX's Vulkan path exactly. Both measured values sit inside their bands, which is the only calibration available without borrowing a card.
Three things follow, and one warning:
- A 128-bit 16 GB card buys capacity at the cost of speed. The four 128-bit boards — RTX 4060 Ti and RX 7600 XT at 288 GB/s, RX 9060 XT at 320, RTX 5060 Ti at 448 — are the cheapest ways to hold 16 GB, and the predicted decode is a ratio of rates, not of bandwidths, and it straddles 1 because the slowest card's MoE band starts below this box's measured 59 tok/s. On the 26B-A4B the group runs 0.75–1.82× (44.017 to 107.598 tok/s predicted ÷ 59 measured); on the 12B dense model 0.84–1.86× (23.545 to 51.942 ÷ 28). If the plan is "run a 13 GiB MoE", they hold it and they run it slowly.
- The 256-bit cards are where the value is for this workload. 512-672 GB/s covers the RX 6800, the Arc A770, the RX 6950 XT / 7900 GRE / 7800 XT / 9070 and the 4070 Ti Super: 1.33–2.74× the MS-S1 MAX's decode on the MoE (78.253 to 161.397 ÷ 59) and 1.49–2.78× on the dense model (41.858 to 77.914 ÷ 28) for the same 16 GB, which turns the 26B MoE from usable into comfortable. AMD is the wider choice here — five boards between 512 and 640 GB/s, with NVIDIA's nearest the 4070 Ti Super at 672.
- The 5070 Ti is the top of the class and pays for it. 896 GB/s is 3.5× this box's bandwidth, which on the same bands is 2.3–3.6× its MoE decode and 2.6–3.7× its dense decode, and it is the only 16 GB card here whose dense-model band sits fully above 70 tok/s.
- Warning: bandwidth does not buy prefill in the same proportion. These predictions cover decode only. Prefill is compute-bound and an MoE's prefill touches every expert, so every card in this table will beat its decode-side prediction on long prompts by a margin the MS-S1 MAX cannot show — the same caveat that applies to Table 9. On the AMD side the software path matters too: these cards run through ROCm or Vulkan on Linux, and backend efficiency, not the memory spec, is what moves prefill.
Figure 8 — Left: vendor bandwidth for the 16 GB cards, against the MS-S1 MAX measured at 256 GB/s. Right: the decode band each bandwidth implies for the two reference models, with the MS-S1 MAX's measured points marked. The bars are a prediction from measured efficiency; the markers are the only measurements in this figure.
13. What this page does not measure
Listed because the gaps are the useful part of a benchmark:
- Agentic tool use. The three protocol checks —
preflightin the harness, Table 3's "protocol 3" — confirm an OpenAI-shaped client can read the reply (3/3 on every row: non-empty content, a JSON object, a one-word answer). A multi-round tool loop was not run, and a model can pass preflight and still fail to drive tools. - Long-context retrieval per model. The roster's needle probes are at 2k and 8k, and the two 4k-context rows were not given the 8k probe at all. A depth curve to 100k+ was run for other models on this platform, not for these.
- Energy and thermals per token. The envelope is measured and quoted with the platform table — 96 W mean package power against a 129 W peak, 69 °C mean GPU temperature against an 88 °C hottest sample — but it is a thermal record sampled every ~2 s, not an energy-per-1k-tokens figure, which would need the integral over a run.
- Concurrency. Every row was measured with one slot. Throughput under
-np 4is a different measurement with a different KV footprint. - Quant ladders per model. One quant per model was measured. The claim that QAT holds up better than post-training quant at equal bpw rests on the controlled pair, not on a sweep.
- Non-autoregressive decoding.
clover-35b-a3b-iq2is measured, but only in the mode llama.cpp can run: causal decode, one token at a time. The model was trained for parallel block diffusion, and nothing here measures that — so the arithmetic that makes the architecture interesting (a whole block decoded at once) is outside this page's evidence entirely. The row is an accurate measurement of the causal arm and no statement at all about the parallel one. It is worth naming as a gap even though the row exists, because the row's numbers are good enough that they will be quoted without the caveat. - A real 16 GiB card. The 15 GiB budget is arithmetic applied to files measured on this host, not a test on a 160 mm card: no run on this page was executed under a VRAM ceiling, so the
tightrows (five of them) are the ones with the least evidence behind their fit.Real spill behaviouris the same gap seen from the other side — this host cannot show it. The spill penalty quoted for discrete cards is from the literature and the PCIe arithmetic, not from a measurement on a card.
14. Reproduce any row
The roster is one directory of scripts driving one llama.cpp server at a time. matrix.py, report.py and humaneval.py are stdlib; two axes are not — MMLU-Pro needs datasets and IFEval runs through the lm-evaluation-harness API backend, both in ~/bench/.venv (lm-eval 0.4.14.dev0, a git-main install: pip install -e ".[api]"; torch is not needed for the HTTP backends):
cd ~/bench/16gb
python3 matrix.py list # roster, computed footprints, gap flags
python3 matrix.py budget --ctx 8192 # fit arithmetic at any context
python3 matrix.py run g4-12b-qat # launch, core axes, stop
python3 matrix.py run g4-12b-qat --full # + HumanEval 164 / MMLU-Pro 120 / IFEval
python3 report.py # the joined table
# standalone full axes: --max-tokens IS the cap, and Table 2 records it row by row
python3 humaneval.py --port 8180 --limit 164 --max-tokens 512 # every non-thinking row
python3 humaneval.py --port 8180 --limit 164 --max-tokens 8192 # the four published 8192 arms: lfm -think, axion, clover, ornith15; add groot-19b-a3b here for its outstanding re-run
python3 humaneval.py --port 8180 --limit 164 --max-tokens 16384 # ornith15-35b-abl-iq3m's re-run: it capped out at 8192
python3 humaneval.py --port 8180 --limit 164 --max-tokens 4096 # the 4096 arms: groot, holo4, huihui-qwen36, both lfm -htc rows; add gpt-oss-20b-mxfp4 for its 4096 re-run, which completed on 20 November at the full sample on all three axes
python3 mmlu_pro.py --port 8180 --limit 120 --max-tokens 512 # C axis; 4096 on the same wide-cap rows
.venv/bin/lm-eval run --model local-chat-completions \
--model_args model=local,base_url=http://127.0.0.1:8180/v1/chat/completions \
--tasks ifeval --limit 50 --batch_size 1 --apply_chat_template \
--gen_kwargs max_gen_toks=4096 # 4096 for all nine reasoning arms; gpt-oss-20b-mxfp4's 50-prompt pass landed on 20 Nov
The manifest at data/provenance.json covers 17 artefacts of the wider model set, 13 of them verified against the upstream LFS object ids, and only two of this page's 18 roster files are in it: LFM2.5-24B-A2B-QAD-Q4_0.gguf and gemma-4-26B_q4_0-it.gguf. The other sixteen are listed with their repository, file and licence in Table 8, but their SHA256s are not published here — publishing them is a data export rather than an edit, and until then Table 8's repository and file name are the provenance. The four artefacts that carry no upstream id — five files, the Flash-Next model being split into two shards — are gpt-oss-120b-MXFP4.gguf, Qwen3.8-35B-A3B-Distill-Q4_K_M.gguf, the two Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S shards and MiniCPM-V-4_7-35B-A3B-Q6_K.gguf; each was fetched before the fetcher recorded a source repository, so the local SHA256 is recorded and the upstream claim is not. None of the four is on this page's roster, and the per-run aggregates are in data/catalogue.json. The per-item rows behind them stay in ~/runs/*.jsonl on the measuring host and are not published, which is why every interval on this page is unpaired (§4). Figures are generated from the same run archive by charts16.py; the fit arithmetic in the tables is the expression matrix.py budget prints, so a figure and a table cannot disagree. The machine, build and server flags are the table at the top of the page; nothing in that stack is a variable in the commands above. Two HumanEval caps are worth re-running before anything else, and they are not the same cap. groot-19b-a3b needs 8192: its published row is the 4096 pass, which lost 7 of 10 failures to the cap, so 0.939 is a floor. ornith15-35b-abl-iq3m already ran at 8192 (156/164, 0.951) and still lost 7 of its 8 failures there, so its re-run needs 16384 — the 8192 pass is what it is, and it is not the same outstanding item. The third item was a sample rather than a cap, and it closed on 20 November: gpt-oss-20b-mxfp4's 50-prompt IFEval pass returned 0.700 at the same 4096 cap, so that cell now sits beside the others and its mean of 0.821 is in Table 2. clover-35b-a3b-iq2 no longer needs anything: its row is measured on the build that carries clover_moe, with flash attention on and q8_0 KV, and its four caps are in Table 2.
15. Licence
Everything on this page — the prose, the tables, the figures and the selection of measurements behind them — is published under CC BY 4.0. You may copy it, quote it, republish it and build on it, including commercially, provided you credit it, link the licence and say what you changed. The attribution line to use:
16 GiB of VRAM: which local LLMs fit, and what they score — strix-halo-128-benchmarks,
CC BY 4.0, https://strix-halo-128-benchmarks.github.io/16gb.html
That is worth stating precisely, because "a licence on information" is only partly possible:
- What is licensed is the expression and the compilation, not the facts. Nobody owns the measurement that this 26B MoE scores 0.902 on HumanEval at cap 512: you can re-measure it, quote the number, and disagree with it in public without asking anyone. The licence binds you when you copy the page, the figures or a substantial part of the dataset.
- The database right is licensed too. In the UK and the EU a collection of measurements can carry a separate sui generis right against extraction and re-utilisation even when individual facts are free. CC BY 4.0 covers that right explicitly, so bulk reuse of the rows is permitted with attribution rather than silently allowed.
- It is not a warranty and not a patent grant. The numbers are one machine, one build, stated caps; the licence gives you no assurance they describe your hardware, and CC licences grant no patent rights.
- Model weights are not distributed here. Every file named on this page carries its own licence from its publisher, and nothing on this page changes it — check the model card before you redistribute weights.
- The benchmarks belong to their authors. HumanEval, MMLU-Pro and IFEval are run against this box, not republished; cite the papers when you cite the scores.
- The code is MIT, the writing is CC BY 4.0.
scripts/andtools/carryLICENSE-MIT, added in 1.29: CC BY is written for prose, and this code is meant to be copied, changed and shipped inside someone else's harness, which MIT permits with no obligation beyond the copyright line. The prose, the figures and the compiled data on this page remain CC BY 4.0 as described above, and no model weights are distributed here under either licence.
The copyright holder is Dom Ashworth (github.com/barmynon); the repository's CITATION.cff carries the citable form of that attribution, as given-names: Dom / family-names: Ashworth with the barmynon alias.
The machine-readable form is alongside the page: a license field in the page's JSON-LD, a rel="license" link in the document head, and the same CC BY 4.0 statement in the repository's LICENSE file.
16. Document history
| version | date | what changed | |||
|---|---|---|---|---|---|
| 1.37 | 20th November 2026 | The last short sample is measured, so the ‡ row becomes a row — and the counts that read off the column move with it. gpt-oss-20b-mxfp4's IFEval came back at n=30 in its 20 November 4096 re-run, which is why Table 2 withheld its mean; the 50-prompt pass ran on the evening of the same day and returned 0.700 prompt-level strict, 0.7368 instruction-level, in 977 s. Table 2: the cell reads 0.700 with no marker, the mean and quality-per-GiB columns are filled (0.821 and 0.065), and the ‡ paragraph in the footnote is gone because no row is partial any more. Two perf cells also move, and not because of this run: gpt-oss-20b-mxfp4 prints 2205 / 72 and qwen3-14b-q4 743 / 23, both re-pointed at the newest perf row in the archive for the same build and context, where the page had been quoting an older one (2169 / 74 and 743 / 24). No ordering claim depends on either: the three rows below lfm25-24b-a2b-think's 111 tok/s are unchanged. What that moves, all of it down-chain from one measured cell: gpt-oss is sixth of nineteen by mean (0.821, between axion-1.0-35b-iq3 at 0.824 and lfm25-24b-a2b-htc-think at 0.816) and fourth among the rows that fit; §8's 0.78 count goes from eight to nine of the other sixteen, and the ringed picks make it eleven of nineteen where it read ten of eighteen; Figure 6's thresholds read eleven / three / fourteen (0.78 / 0.85 / 0.75) and its caption no longer explains an absence. Two prose rankings moved: §3's "ties groot for seventh of nineteen" is eighth, and gpt-oss-20b-mxfp4's IFEval in the same paragraph is 0.700 rather than the withheld 0.733. Start here gains a paragraph and no fifth row: gpt-oss is the smallest footprint to clear 0.82 (12.65 GiB against clover's 14.45 and lfm-think's 13.96), but lfm25-24b-a2b-think beats it on mean (0.861) and on decode (111 tok/s against 74), so the four picks stand. The gate: 71 checks. The PARTIAL exclusion that carried the ‡ row is removed, so every Table 2 mean is checked against the archive including this one; Figure 6's counts, the §8 tallies, the §3 placing and the new Start-here paragraph's mean, placing and decode comparison are derived from the archive rather than typed; and a new check compares every pp / tg cell in Table 2 with the newest perf row for that model — the check that found the two stale cells above. | |||
| 1.36 | 20th November 2026 | The prose rankings that the gpt-oss re-measure invalidated — none of them a measurement. Table 2's footnote still called gpt-oss "the one thinking arm left at the default caps" with three floor cells, which 1.35 had already turned into real measurements; it now names the one thing that is still short there, its IFEval sample of 30. §14's command comment said the task pins 1280 for gpt-oss while the run recorded 4096, and said "eight of the nine" reasoning arms where Tables 1 and 3 say all nine — both corrected, and the gate now proves the claim rather than quoting it: every thinking arm has an archived row at 4096 or wider on all three axes (it reads gen_max_toks for IFEval, which is where the first version of that check went wrong). §3's axion placing read "fourth on HumanEval" at 0.933; with gpt-oss at 0.963 it is fifth among the rows that fit — fourth is groot at 0.939, and counting ornith15-35b-abl-iq3m at 15.06 GiB, which the page flags NO, it would be sixth, so the check derives the placing from Table 2 with that flag applied. §8's "seven rows above 0.90 HumanEval" is eight (0.970, 0.970, 0.963, 0.951, 0.945, 0.939, 0.933, 0.902), and the clause that follows it — four of them at 64k — still holds. §3's "the weakest of the fitting rows at 12 GiB or more" was wrong before 1.35 as well and is corrected here: axion's 0.780 is the third-weakest at that footprint, with gpt-oss-20b-mxfp4 (0.733) and groot-19b-a3b (0.740, 13.76 GiB) below it. §5 kept the 8.4-point gap that §3 and Table 4 had already restated as 8.3 (10 items of 120), and now reads 8.3. Table 4's "the widest cross-row gap on the page" is not true of the page: it is the widest gap among the knowledge leaders, while the column's own range is 65 points, 0.833 down to 0.192, and the cell now says both. §14's provenance sentence counted "the four that carry no upstream id" and then listed five file names; it now says four artefacts, five files, with the Flash-Next model as two shards. The gate: 63 checks — six new ones that derive every ordinal and count claim in the prose from Table 2 (axion's placing, the 0.90 tally, the IFEval weakest claim, the scoped MMLU gap, and no stale 8.4 outside the history), so the next re-measure fails the build instead of going stale. | |||
| 1.35 | 20th November 2026 | The gpt-oss row is re-measured; nothing else measured moves. gpt-oss-20b-mxfp4 now carries 0.963 HumanEval (158/164) and 0.800 MMLU-Pro (96/120), both at a 4096 cap from the 20 November re-run, in place of the 0.585 and 0.558 default-cap floors; the † markers they carried are gone with them. Its IFEval is 0.733 at n=30 — the re-run's sample was allocated 30 of the 50 prompts — so that cell carries ‡ and the mean and quality-per-GiB columns are withheld, because an n=30 axis against n=50 axes is not comparable. Consequences, all applied: Table 4's HumanEval runner-up is now gpt-oss (1 problem behind the leader) with clover third; §3's clover bullet reads third on code; Table 1 and Table 3 say all nine reasoning arms were given the wider cap and that gpt-oss's IFEval sample is the small one; §8 and Figure 6 count ten of the eighteen rows that have a mean, and gpt-oss is named as the row that has none yet. §8's counts were also wrong on their own terms — it said ten of the sixteen non-pick rows clear 0.78 where eight do, and called the total twelve of nineteen; it now reads eight of sixteen and ten of eighteen. §3's gaps come from the item counts: 4 of 164 is 2.4 points and 10 of 120 is 8.3, where the rounded scores had given 2.5 and 8.4. Three history entries were lost by the 1.32–1.34 edit passes: each pass inserted its new entry by replacing the previous row's header cell, so the older header vanished and its body was merged into the new row — 1.31, 1.32 and 1.33 ended up inside the entries above them and were not readable as rows. They are restored here as rows of their own, from the text they were published with, and this pass's own insertion is checked by the gate. Only the quoted broken strings inside them were re-wrapped in code spans so they render. The gate: 57 checks — the §8 counts (eight clearing, fifteen others named), the two item-count gaps, and a means check that skips a row whose derived columns are withheld. | |||
| 1.34 | 20th November 2026 | Consistency, sixth pass — no measured value changed, and Table 2 is deliberately unchanged. Table 6's IFEval verdict called the dense model's 3-prompt lead a win; at n=50 a two-row comparison needs ±12.2 points, so it now reads "not separable" like every other cell in that table, and its MMLU-Pro verdict quotes the exact ±12.4 instead of a rounded ±12. §4's bands are pair-specific on all three axes — HumanEval ±4.4 to ±6.5, MMLU-Pro ±10.1 to ±12.4, IFEval ±10.0 to ±12.2 — each with the pair it belongs to, generated by the gate to one decimal. §8's map paragraph said "the other sixteen quality rows" and then named twelve; it now names all sixteen, with the two ringed picks making twelve of nineteen above the 0.78 line, and Figure 6's threshold counts (ten, three, thirteen) are checked against the roster. §9's "24 MiB per 1k" is labelled the q8_0 rate, with Table 5's fp16 figure of 48 named beside it. §2's terms gain QAD, quantisation-aware distillation, which the LFM file names have carried since v1.24 without explanation. gpt-oss-20b-mxfp4's 4096 re-run finished at 15:26, after 1.33 was published: HumanEval and MMLU-Pro landed at the full sample and IFEval returned at n=30, so that cell needs a 50-prompt pass before it is comparable. §14 records that, 1.33's "has not finished" is corrected here, and groot is described as one of two 4096-cap rows with a re-run on the books rather than the only one. Table 2 still carries the default-cap cells with their † markers: replacing them changes measured values, and this pass changes none. The gate: 55 checks — all six two-row bands, §8's sixteen names, Figure 6's three counts, Table 5's SWA column and the hand recipe that closes against Table 2. | Consistency, fifth pass — no measured value changed. Table 1 said the eight reasoning rows were given 4096 "on every axis", which Table 3 contradicts: four of them carry 8192 on HumanEval and clover 8192 on MMLU-Pro as well, and the cell now says which. §3's "byte-identical 12.65 GiB file" and §6's "the same LFM file" were both wrong about an abliterated rebuild — same architecture and size, different bytes — and both now say that; Table 5's caption names the three LFM rows it does not list rather than calling them "the three arms" when the roster carries four. §4's HumanEval band is pair-specific, like the MMLU-Pro one: ±4.4 at the leaders (0.970/0.945) and ±6.5 at the controlled pair (0.896/0.902), where it previously quoted a single ±4.5. The storage row's "about 10 s" was arithmetic no drive could do at the rate quoted — 23.61 GiB is 25.35 GB, so 13 s at 1.9 GB/s and up to 51 s at the 0.5 GB/s end. Table 5 gains the SWA column (0.036 to 0.156 GiB at q8_0), so every footprint in Table 2 can now be recomputed by hand as weights + the context column + the SWA column + 1 GiB, which the gate verifies row by row. Two 1.32 claims are corrected here: groot is not the only 4096 HumanEval pass — holo4-35b-heretic-iq4xs, huihui-qwen36-35b-abl-q5k and both -htc rows are too; it is the only 4096 row whose re-run is queued, the other two being over the wall at every context — and "no fit flag moves" read as though the SWA term had changed a verdict, when the published 15.06 GiB always included it and that is why the MTP row reads NO. Superseded wording quoted in this history is now in code spans. gpt-oss-20b-mxfp4's 4096 arm has not finished: HumanEval and MMLU-Pro have landed, IFEval has not, and Table 2 is deliberately untouched — replacing those cells is a measured change, not a consistency pass. The gate: 44 checks — Table 5's SWA column and the hand recipe that closes against Table 2, §4's two HumanEval bands, and the load-time arithmetic. | Consistency, fourth pass — no measured value changed, and no fit flag changed. §2's footprint definition and the roster's caption were incomplete: the arithmetic also carries the sliding-window attention fixed cost where the architecture has one (0.03-0.16 GiB at q8_0 here, because a hybrid's window buffer does not shrink with context), which is why g4-26b-a4b-qat reads 14.70 GiB at 16k and not 14.61, and huihui-gemma26b-qat-abl 16.89 and not 16.80. The numbers were right and the definition was not. That term is what the published value already carried — the MTP row is 14.96 GiB without it and the printed 15.06 with, which is why the row reads NO — and §10 now says so. §10 also states the context of both figures it compares and attributes the 0.43 GiB difference to the drafter alone, both rows being at 8k; Table 2's 14.70 is the same row at 16k. §3's riddle sentence read `15 of 19 scored rows where the roster has 21 rows and riddles were run on all of them (15 clear it outright, 14 of the 19 that carry quality scores). **§14's ornith15 status was wrong:** it already ran HumanEval at 8192 (156/164, 0.951) and still lost 7 of its 8 failures to that cap, so its outstanding re-run is at **16384**, not 8192 — groot is one of the two 4096-cap rows whose re-run this page tracks (the other is gpt-oss-20b-mxfp4), and it needs 8192; the other 4096 passes are floors too (holo4-35b-heretic-iq4xs, huihui-qwen36-35b-abl-q5k, the two -htc rows), but the first two are over the wall at every context and are not queued. The recipes carry both caps now. §12's ratios are computed with Decimal / ROUND_HALF_UP and their endpoints print three decimals, so dividing the printed numbers reproduces the printed range (41.858 ÷ 28 = 1.4949 → 1.49; the two-decimal 41.86 ÷ 28 = 1.495 would read 1.50, which is what made the previous edition look wrong on a calculator). The history entries now put the repaired strings they quote in code spans. The marked.Three a reader reported does not exist in Figure 1's title, description or alt text — the figure reads "…budget marked." in all three; the string existed once, in the 1.31 entry, in a nested-backtick span, and that markup is fixed. **The gate:** 40 checks — every Table 2 footprint re-derived at its own context, §10's two 8k footprints and their difference, the riddle denominator, and the ratio ranges taken through Decimal` instead of float formatting. | Consistency, third pass — no measured value changed. Table 2's rounding examples were wrong in the last digit (lfm25-24b-a2b-think is 0.8604 from 159/164, 83/120 and 46/50, not 0.8607; mistral-24b-q4 is 0.7505 from 135/164, 73/120 and 41/50, not 0.7503) and both are now computed from the run counts by the gate. Two joins and two lost spaces are repaired — `-htc,and` in Table 1's server-flags row, and ceiling.qwen3-30b-a3b-q3` in Figure 2's caption — and the sentence that quoted the alleged marked.Three / panelsThree typos no longer embeds them, since they exist in neither the caption nor its alt text. §3 says "the -htc arm" where it said "the heretic arm", which §2's naming rule requires. **§14's provenance claim was wrong:** the manifest covers 17 artefacts of the wider model set, 13 verified, and only **two of this page's 18 roster files** (LFM2.5-24B-A2B-QAD-Q4_0.gguf, gemma-4-26B_q4_0-it.gguf), so the sentence now says exactly that and points at Table 8 for the rest. §14's HumanEval recipes name the two outstanding re-runs alongside the published arms (groot-19b-a3b at 8192, gpt-oss-20b-mxfp4` at 4096). §12's divisions print three decimals so the arithmetic reproduces the printed range (41.858 ÷ 28 = 1.4949 → 1.49; 41.86 ÷ 28 = 1.495 would read 1.50, and 41.9 ÷ 28 = 1.50). The v1.29 entry's tally of repaired joins was inverted and now reads two joined words, three lost spaces. The gate: 37 checks — the two footnote means from the counts, the provenance coverage against the roster, and a lint that keeps code spans in place (so punctuation butting against them shows) and reads the rendered page's alt/title attributes and figcaptions with the JSON-LD and SVG blocks removed. |
| 1.33 | 20th November 2026 | Consistency, fifth pass — no measured value changed. Table 1 said the eight reasoning rows were given 4096 "on every axis", which Table 3 contradicts: four of them carry 8192 on HumanEval and clover 8192 on MMLU-Pro as well, and the cell now says which. §3's "byte-identical 12.65 GiB file" and §6's "the same LFM file" were both wrong about an abliterated rebuild — same architecture and size, different bytes — and both now say that; Table 5's caption names the three LFM rows it does not list rather than calling them "the three arms" when the roster carries four. §4's HumanEval band is pair-specific, like the MMLU-Pro one: ±4.4 at the leaders (0.970/0.945) and ±6.5 at the controlled pair (0.896/0.902), where it previously quoted a single ±4.5. The storage row's "about 10 s" was arithmetic no drive could do at the rate quoted — 23.61 GiB is 25.35 GB, so 13 s at 1.9 GB/s and up to 51 s at the 0.5 GB/s end. Table 5 gains the SWA column (0.036 to 0.156 GiB at q8_0), so every footprint in Table 2 can now be recomputed by hand as weights + the context column + the SWA column + 1 GiB, which the gate verifies row by row. Two 1.32 claims are corrected here: groot is not the only 4096 HumanEval pass — holo4-35b-heretic-iq4xs, huihui-qwen36-35b-abl-q5k and both -htc rows are too; it is the only 4096 row whose re-run is queued, the other two being over the wall at every context — and "no fit flag moves" read as though the SWA term had changed a verdict, when the published 15.06 GiB always included it and that is why the MTP row reads NO. Superseded wording quoted in this history is now in code spans. gpt-oss-20b-mxfp4's 4096 arm has not finished: HumanEval and MMLU-Pro have landed, IFEval has not, and Table 2 is deliberately untouched — replacing those cells is a measured change, not a consistency pass. The gate: 44 checks — Table 5's SWA column and the hand recipe that closes against Table 2, §4's two HumanEval bands, and the load-time arithmetic. | |||
| 1.32 | 20th November 2026 | Consistency, fourth pass — no measured value changed, and no fit flag changed. §2's footprint definition and the roster's caption were incomplete: the arithmetic also carries the sliding-window attention fixed cost where the architecture has one (0.03-0.16 GiB at q8_0 here, because a hybrid's window buffer does not shrink with context), which is why g4-26b-a4b-qat reads 14.70 GiB at 16k and not 14.61, and huihui-gemma26b-qat-abl 16.89 and not 16.80. The numbers were right and the definition was not. That term is what the published value already carried — the MTP row is 14.96 GiB without it and the printed 15.06 with, which is why the row reads NO — and §10 now says so. §10 also states the context of both figures it compares and attributes the 0.43 GiB difference to the drafter alone, both rows being at 8k; Table 2's 14.70 is the same row at 16k. §3's riddle sentence read `15 of 19 scored rows where the roster has 21 rows and riddles were run on all of them (15 clear it outright, 14 of the 19 that carry quality scores). **§14's ornith15 status was wrong:** it already ran HumanEval at 8192 (156/164, 0.951) and still lost 7 of its 8 failures to that cap, so its outstanding re-run is at **16384**, not 8192 — groot is the only row whose published HumanEval is the 4096 pass, and it needs 8192. The recipes carry both caps now. §12's ratios are computed with Decimal / ROUND_HALF_UP and their endpoints print three decimals, so dividing the printed numbers reproduces the printed range (41.858 ÷ 28 = 1.4949 → 1.49; the two-decimal 41.86 ÷ 28 = 1.495 would read 1.50, which is what made the previous edition look wrong on a calculator). The history entries now put the repaired strings they quote in code spans. The marked.Three a reader reported does not exist in Figure 1's title, description or alt text — the figure reads "…budget marked." in all three; the string existed once, in the 1.31 entry, in a nested-backtick span, and that markup is fixed. **The gate:** 40 checks — every Table 2 footprint re-derived at its own context, §10's two 8k footprints and their difference, the riddle denominator, and the ratio ranges taken through Decimal` instead of float formatting. | |||
| 1.31 | 20th November 2026 | Consistency, third pass — no measured value changed. Table 2's rounding examples were wrong in the last digit (lfm25-24b-a2b-think is 0.8604 from 159/164, 83/120 and 46/50, not 0.8607; mistral-24b-q4 is 0.7505 from 135/164, 73/120 and 41/50, not 0.7503) and both are computed from the run counts by the gate. Two joins and two lost spaces are repaired — `-htc,and` in Table 1's server-flags row, and ceiling.qwen3-30b-a3b-q3` in Figure 2's caption — and the sentence that quoted the alleged marked.Three / panelsThree typos no longer embeds them, since they exist in neither the caption nor its alt text. §3 says "the -htc arm" where it said "the heretic arm", which §2's naming rule requires. **§14's provenance claim was wrong:** the manifest covers 17 artefacts of the wider model set, 13 verified, and only **two of this page's 18 roster files** (LFM2.5-24B-A2B-QAD-Q4_0.gguf, gemma-4-26B_q4_0-it.gguf`), so the sentence now says exactly that and points at Table 8 for the rest. §14's HumanEval recipes name the outstanding re-runs alongside the published arms. §12's divisions print two decimals so the arithmetic reproduces the printed range. The v1.29 entry's tally of repaired joins was inverted and now reads two joined words, three lost spaces. The gate: 37 checks — the two footnote means from the counts, the provenance coverage against the roster, and a lint that keeps code spans in place and reads the rendered page's alt/title attributes and figcaptions. | |||
| 1.30 | 20th November 2026 | Attribution only; no measurement, table or figure changed. The copyright line in LICENSE and LICENSE-MIT, the author line at the top of this page and the copyright-holder sentence in §15 now read Dom Ashworth; CITATION.cff states given-names: Dom / family-names: Ashworth with the barmynon alias, which is the form a citation tool reads. | |||
| 1.29 | 20th November 2026 | Consistency, take two — no measured value changed. §8's "39% of the mean pick" divided by 13.96 GiB, which is the thinking arm's footprint and not the mean pick's; it is 5.40 ÷ 14.45 = 37% and the sentence now shows the division. Table 2's footnote claimed gpt-oss-20b-mxfp4's three cells were floors while the row carried no marker — the cells now carry †, and the footnote says why (the default caps with the reasoning block open) instead of folding it into the five cap-hit cases. Naming: the abliterated LFM pair is "the two -htc rows" throughout the body, holo4-35b-heretic-iq4xs is referred to by id, and the glossary states the two-suffix convention. §6 said the -htc pair was qwen3.5moe-derived and implied a Table 5 row at 36 MiB; it now names the five qwen3.5moe files and the diffusion export, and says the abliterated LFM pair shares the 20 MiB line because it is the same file. §4 rewords the unpaired paragraph — unpaired is the conservative choice, since a paired test on shared items would be tighter — and says why the same formula gives ±10.1 for one pair and ±12 for the other. §12's multipliers now show their divisions and separate the two models: 0.75–1.82× (MoE) and 0.84–1.86× (dense) for the 128-bit group, 1.33–2.74× and 1.49–2.78× for the 256-bit group; the earlier ranges mixed the MoE's low end with the dense model's high end. §14 names the four artefacts that carry no upstream id and the reason — fetched before the fetcher recorded a source repository — and notes none is on this page's roster. Licence: the code is MIT (LICENSE-MIT, covering scripts/ and tools/), the prose, figures and compiled data stay CC BY 4.0, and §15 and the footer now say that instead of recommending a split that had not been made. Table 2's footnote gains the two rounding examples that otherwise read as errors (lfm25-24b-a2b-think 0.8607 prints 0.860; mistral-24b-q4 0.7503 prints 0.751). Two joined words and three lost spaces are repaired (andthe, rowswhose, shortlist:the, statistics:ten, at once:the). The gate grows to 34 checks: it now derives §8's percentage, §12's four ratio ranges and the six rows §6 must name, and lints the prose for joined words and lost spaces after stripping code spans, filenames, domains and version numbers. Verified and unchanged: the two means a reader recomputed from the printed cells are correct from the unrounded scores (now said in place), and the two caption typos a reader reported (an alleged missing space after "marked" in Figure 1 and a run-together "panels Three") do not exist in the source, in the caption or in its alt text. gpt-oss-20b-mxfp4's 4096 re-run is still in flight — its HumanEval axis returned 158/164 at that cap — and its cells move when the arm lands. | |||
| 1.28 | 20th November 2026 | Consistency pass — no measured value changed. §5 quoted ±7.7 points for the MMLU-Pro gap while §4, Table 4 and Table 6 used ±10, ±10.1 and ±12; the band depends on the scores in the pair, which is why the numbers differ, and §5 now quotes the ±10.1 interval for its pair and calls clover's lead "the best-supported ordering on the page … still a lead rather than a win", which is what Table 4 has said since 1.27. §4 labels its standard errors as 68% intervals rather than the figures' whiskers, and says that every band on the page is unpaired — all models answer the same items, so a paired test on the per-item rows would be tighter, and is not computed here. §6 called 36 MiB per 1k "a third of" the 160 MiB dense rate (it is under a quarter) and named three rows where Table 5 carries six; §8's "39% of it" now names the 13.96 GiB footprint it divides by; Table 1's kernel note said four rows and named three models (four runs across three models); §12's multipliers are labelled decode ratios, and the 256-bit band reads 1.3–2.8x; §13 names the arithmetic-not-measured limit and the five tight rows as the least-evidenced ones. Table 2's caption says the mean is taken from the unrounded per-item scores, which is why two means differ by 0.001 from the mean of their printed cells. The meta description said thirteen 16 GB cards where §12 lists twelve plus the reference. §14 is brought level with the page it describes: the lm-eval environment and version are named (0.4.14.dev0, .[api], no torch), the standalone MMLU-Pro and 4096-HumanEval commands are added, the 4096 IFEval cap is stated as eight of nine reasoning arms rather than four, both outstanding 8192 re-runs are named (groot-19b-a3b, ornith15-35b-abl-iq3m), and the published artefacts are cited (data/provenance.json SHA256s, data/catalogue.json). v1.21's "13 items" for axion-1.0-35b-iq3's MMLU-Pro lead over lfm25-24b-a2b-think was 8 items; the correction is recorded here rather than inside the old entry, on the same rule v1.20 followed. Eight missing spaces left by the 1.27 edit pass are repaired. A build gate (scripts/check-derived.py) now recomputes the roster counts, the 8k ladder, the four two-row intervals, the reasoning-arm and 8192-arm counts and the riddle tallies from the archive, and fails the build when the prose disagrees. | |||
| 1.27 | 20th November 2026 | One bar for cross-row claims, an inclusion rule for the winner table, and six figure repairs — no measured value changed. §4 now states the 95% interval on a difference, √(SE₁²+SE₂²): about ±4.5 points on HumanEval, ±10 on MMLU-Pro and ±10 on IFEval at the scores here. Two verdicts failed it and were restated. Table 4's MMLU-Pro lead is 10 items, 8.4 points, inside a ±10.1-point interval — the widest cross-row gap on the page, not "the first decisive knowledge win" — and Table 6's 5.8-point MMLU-Pro gap is inside the band rather than "real"; §7's architecture argument now rests on decode (2.1x) and footprint, which it always did. Table 4 states what it ranks: only rows whose footprint at the measured context is inside 15 GiB, which is why huihui-gemma26b-qat-abl (0.881 — the highest mean on the page, over the wall at 16.89 GiB and measured with its reasoning block closed) and ornith15-35b-abl-iq3m (0.848, over by 0.06 only at 64k) are now named in §3 and in the caption instead of being absent. Kernel: Table 1 no longer claims one value — 7.3.3 for seventeen rows and 7.3.4 for the four measured on 16 November (xing4 twice, lfm25-24b-a2b-htc, lfm25-24b-a2b-htc-think), so the abliterated pair's decode row is a cross-kernel comparison and says so. Counts, recounted: nine reasoning arms rather than five or eight; four 8192-cap arms; fifteen rows at 14/14 riddles and four at 13/14; the 26B's mean is seventh of nineteen, not fourth; five tight rows below clover, not four; groot's footprint is 0.94 GiB smaller than the 26B's, not 0.06; §11's 24 GB rung excludes the Q5_K file; §8's map paragraph covers sixteen other quality rows. gpt-oss-20b is marked for what it is — a thinking arm left at the default caps, so its three quality cells are floors — rather than carried as a measurement. Figures: Figure 8's MoE band is 35–55% as Table 10 says (drawn at 45–65%, which also clipped the 5070 Ti); Figure 7's reference label no longer prints a literal \n; Figure 4's legend moved off its bars and its values sit above their whiskers; Figure 3's label gutter clears the two points that crossed it; Figure 2 prints "no context" instead of "0k"; the captions of Figures 1 and 6 name the rows their frames leave out and count the thresholds they draw (ten clear 0.78, not nine at 0.79). Structure: a start-here table gives the three picks plus the row that does not fit; §2 defines effective bpw, A4B/E4B and the three senses of context; Table 2's caption hands the cap policy to Table 3, where it is stated per axis. | |||
| 1.26 | 18th November 2026 | Correction to the two rows v1.25 got wrong below the roster: at 8k context, not each model's own, the 16 GB rung's casualties are four (g4-26b-a4b-qat-mtp 15.06, huihui-gemma26b-qat-abl 16.82, holo4-35b-heretic-iq4xs 17.29, huihui-qwen36-35b-abl-q5k 24.75), not one, and ornith15-35b-abl-iq3m is not among them because its 15.06 GiB is its 64k footprint; the 24 GB rung's "everything here fits" becomes "everything but the Q5_K file". Also the ladder caption's gemma figure is 16.82 GiB at 8k, not 16.89, which is its own-context footprint. No score changed. | |||
| 1.25 | 18th November 2026 | The round-eight roster lands: five rows, and three of them are floors at their own caps. xing4-29b-a4b-iq3xs (perf-only, 14.91 GiB), ornith15-35b-abl-iq3m (15.06 GiB, over the wall by 0.06), huihui-gemma26b-qat-abl (16.89 GiB), holo4-35b-heretic-iq4xs (18.28 GiB) and huihui-qwen36-35b-abl-q5k (25.74 GiB) join Table 2. The roster is now twenty-one rows over eighteen files, 218.8 GiB of weights, and the runs behind the five date from the 13th and 16th, so the measurement window is unchanged. Three of the five code cells are floors, not measurements: ornith15-35b-abl-iq3m lost 7 of its 8 HumanEval failures to the 8192 cap (0.951 printed), holo4-35b-heretic-iq4xs lost 18 of 40 and huihui-qwen36-35b-abl-q5k 35 of 38 to the 4096 cap, so the † marker that carried two rows now carries five and the caption says which. The ladder's top two rungs change hands — at 23 and 31 GiB usable the best mean quality is huihui-gemma26b-qat-abl at 0.881 against clover-35b-a3b-iq2's 0.876 — while the 16 GiB rung still elects clover, because the gemma file is 16.82 GiB at 8k and does not fit. Everything that counts rows was recounted rather than assumed: Table 2's cap policy, Table 3's four axis cells, needle coverage (21 rows at 2k, 19 at 8k), Table 5's KV rows, Table 7's ladder, Figure 1's panel count (nineteen quality rows), Figure 6's threshold counts (nine above 0.79, three above 0.85, thirteen above 0.75), the header's file and weight totals, and the build row, which now names three binaries because the Xing4 shape (xing4_0) is not in the fork. This also fixes a live defect: the figures have plotted all twenty-one models since v1.24, because the round-eight runs were already in matrix.json when the charts were regenerated, so 16gb-f2, f3 and f6 carried five points labelled with model names no table on the page defined. | |||
| 1.24 | 17th November 2026 | The abliterated twin measured: two rows over one new weight file, and the class's first controlled uncensoring test. lfm25-24b-a2b-htc (budget 0) and lfm25-24b-a2b-htc-think (think block open) join Table 2 at a 13.96 GiB footprint each, run on build b11581 at cap 4096. The roster is now sixteen rows over thirteen files, and everything that counts rows was recounted rather than assumed: Table 3's cap policy, the needle coverage, Table 4's IFEval and riddle rows, §3's cap band and the overall row's rank denominator. The IFEval lead changes hands — 0.940 (47/50) for the heretic think arm against 0.920 for the pair that held it — which is one prompt and therefore not a win, but it is evidence that abliteration did not cost compliance. The refusal delta (25.0% → 0.0% over 200 benign prompts) is the reason the pair exists; it comes from the suite's own benign probe, not from the three axes on this page, and is quoted in §3 with that said. | |||
| 1.23 | 12th November 2026 | Correction pass on v1.22: the clover row reached the tables but six sections still described the pre-clover page. Fixed: §5's knowledge paragraph ("the front ends at the biggest point, axion-1.0-35b-iq3") and its Figure 1 caption ("twelve models" — there are thirteen quality rows); §9's claim that axion-1.0-35b-iq3 is "deeper than anything else here" and "the highest in the roster", where clover-35b-a3b-iq2 is now 3.05 bpw and 0.842; §8's "three rows above 0.90 HumanEval, two of them at 64k" (five and three); Table 7's "nothing here is larger than 13.9 GiB" (13.70 GiB of weights, 15.06 GiB of footprint at 8k); §14's instruction to re-run clover on a build that carries clover_moe — that is the build this row came from, so only groot's 8192 HumanEval is still outstanding; §3's "ties groot for third" (fourth), "like the three rows above it" (the tight rows are below it), and a prefill row that omitted clover's 1849 t/s. Cap policy now names all four reasoning arms — Table 1, Table 2's caption, Table 3's three axis rows and §14's commands — because clover is a fourth (--reasoning auto) with caps of 8192 HumanEval, 8192 MMLU-Pro and 4096 IFEval, the widest band on the page. † floor marker extended to clover (4 of its 9 HumanEval failures at the 8192 cap) and the caption now distinguishes its case from groot's (7 of 10 at 4096). Table 5 gains clover at 36 MiB/1k, and its caption explains that the rate follows the 10-full-attention-block layout rather than the qwen3.5moe family. Table 2's caption was ~1,900 characters carrying cap policy, the † footnote and both derived-metric definitions; the policy now lives in Table 3, where a reader looks for it. Header: 14 measured rows over 12 weight files, 124.3 GiB of weights plus a 0.43 GiB drafter — the old "14 weight files, 150.4 GiB" counted rows as files and double-counted 26.1 GiB, since the think arm and the MTP arm each reuse a file. Figure 6 now rings the mean leader (clover-35b-a3b-iq2, AR mode only) instead of lfm25-24b-a2b-think, its caption counts six rows above 0.79 and two above 0.85 (it said five and one), the threshold labels have a background patch so the value-pick leader no longer runs through them, and the 13.7-14.8 GiB cluster labels sit in a widened gutter instead of clipping at the frame edge. Its caption also states that it plots Figure 1's data reduced to the mean, which is what it always was. Citations: Figures 1, 3, 5, 6 and 8 are now referenced from the prose of the section that carries them; before this pass five of the eight were referenced only from this history table. | |||
| 1.22 | 12th November 2026 | Clover measured: the fourteenth row, and the page's first decisive knowledge win. clover-35b-a3b-iq2 (block-diffusion MoE, 34.7B, IQ2_M at 3.05 bpw, 12.32 GiB) scores 0.945 HumanEval (155/164), 0.842 MMLU-Pro (101/120), 0.840/0.829 IFEval (50@4096) at 1849/68 t/s, in a 14.45 GiB footprint at 64k context — tight, but inside the wall. It is measured AR-only: llama.cpp runs the architecture autoregressively, so this is the causal arm of weights trained for parallel block decode, and the row and its verdicts say so. New leaders: knowledge moves from axion-1.0-35b-iq3 (0.758) to clover by 10 items — 8.4 points, outside the ±7.7-point interval, the first decisive win on this page — and the overall mean moves from lfm25-24b-a2b-think (0.860) to 0.876. HumanEval's second place changes hands to clover at 0.945, 4 items behind the leader and inside one interval, so the top is a cluster rather than a pair. IFEval settles the cap question on this model: 0.560/0.553 at the task's own 1280-token pin, 0.840/0.829 at 4096 — the same artefact groot-19b-a3b (0.220 → 0.740) and axion-1.0-35b-iq3 (0.300 → 0.780) produced. Build: the row comes from a second binary — fork tree HEAD 19b458b5a, which adds clover_moe — so the header row now names both and Table 1's build cell says which rows used which; every other flag is unchanged, including q8_0 KV and flash attention. Also: roster 14 rows, 150.4 GiB; Table 7's denominators 13 → 14, its 16 GB row 12 → 13 of 14 and its 24 GB/32 GB+ rows 13 → 14, all three with clover as the mean leader; the needle row's 8k count 11 → 12; the riddles row 9 → 10 models tied; §13's "no row" gap for this model is replaced by the AR-only caveat, because what remains unmeasured is parallel decode, not the model. | |||
| 1.21 | 12th November 2026 | Two rows added, both qwen3.5moe hybrid-attention MoEs measured on the same build and flag set as the other eleven: groot-19b-a3b (11.63 GiB, 64k context, 13.76 GiB footprint) and axion-1.0-35b-iq3 (34.7B at 3.14 bpw, 64k context, 14.81 GiB). Roster 13 rows, 138.1 GiB. New numbers: MMLU-Pro has a new leader — axion-1.0-35b-iq3 0.758 (91/120), ahead of groot-19b-a3b 0.717 and 13 items ahead of the previous leader lfm25-24b-a2b-think 0.692 — so §3's knowledge pick moves from g4-26b-a4b-qat to it, and the 26B's bullet now says what it still is rather than what it was. HumanEval's lead narrows from 11 items to 6 (axion-1.0-35b-iq3 0.933 against lfm25-24b-a2b-think 0.970), which is inside one standard interval, so "the only decisive win in the roster" is withdrawn. Overall mean: 0.860, 0.824, then 0.799 twice. Caps: the new arms think, so their caps differ and Table 2 now states them per row — HumanEval 4096 on groot-19b-a3b (cap-bound: 7 of its 10 failures at the cap, marked † as a floor) and 8192 on axion-1.0-35b-iq3; MMLU-Pro and IFEval 4096 on both, because the IFEval task's own 1280 is spent inside a reasoning block, which is why groot-19b-a3b's first IFEval pass read 0.220 before it was re-run at 4096 (0.740). Also changed: Table 2 is now sorted by footprint as its caption always claimed — the 7.93 GiB qwen3-8b-q4 row had been sitting above the 7.67 GiB g4-12b-qat; Table 5 gains the two 36 MiB/1k hybrid rows; Table 7's denominators are 13 and its 8k ladder 2 / 4 / 12 / 13; §9 gains the measured counterexample (a 3.14 bpw 35B keeps the best MMLU-Pro in the roster, so parameter count buys what quantisation depth does not take away); §5, §8 and Figure 6's threshold counts are recomputed for twelve quality rows. Figures: labels for the 13.7-14.8 GiB cluster now sit in a leader-lined gutter in Figure 6 and a stacked gutter in Figure 3, front ends are placed by collision test in Figure 1, and charts16.py no longer draws a bar or counts a rung for a roster entry with no measured run. Not published: clover-35b-a3b-iq2, an architecture the serving build does not implement — §13 and §14 say why, and what would fix it. | |||
| 1.20 | 11th November 2026 | One word in Table 4. Its quality-per-GiB row said the next of the rows at 12 GiB or more is 0.054; it is 0.062 — lfm25-24b-a2b-think at 0.860 mean over a 13.96 GiB footprint. 0.054 is g4-26b-a4b-qat, one place behind it. No other value changed. | |||
| 1.19 | 11th November 2026 | Read-through pass; no measured value changed. Renderer: section numbers were being dropped from the headings, so all 38 §N references in the prose pointed at headings the reader could not number — they now render, the contents list carries them too, and every §N where the section exists is a link to it. This is one fix in md2html.py and it applies to every page built from it. Axes: Table 3 gains the two probes the header and §4 had always promised but no table showed — needle 2k/8k and the three protocol checks — so the page no longer says four quality axes and describe three. Corrected: §3's "the 4k rows fit only at 4k" and §9's "the 30B affords 4k" both contradicted Table 7 (10 of 11 fit at 8k) and Figure 2 (the 30B's ceiling is ~13k); both now say the two rows were run at 4k because that is where they keep headroom, with the arithmetic ceiling named. §2 now defines the three different senses of "fits" that produced that contradiction. §3's "second among dense models on HumanEval" was first (0.896 against 0.866 and 0.854); "the only row that wins an axis outright" ignored its own prefill and decode rows, so it says quality axis; §8's "everything else is under the line, over the wall, or both" was false for qwen3-14b-q4 and g4-26b-a4b-qat, and "a third of it" is 39%; Table 4's "±8 confidence band" mixed items with points and is now 7 items, 5.8 points, inside a ±8.7-point interval; Table 8's header said 15 − 1 GiB and its cells said 13.6, its 5.0 bpw row printed GB as GiB (15.6 becomes 14.7), and its 3.9 bpw row claimed 2 GiB of KV room for a file that leaves 0.30 — the table is rebuilt in GiB at the 25.2B parameters the file implies. §10's drafter item now shows the arithmetic (14.63 → 15.06). Captions: Figure 1 said "the Pareto front" where each of its three panels has its own, and did not say the whiskers are 95% intervals; Figure 3 omitted the budget line and the front it draws, and did not say which axis is log; Figure 4 described only its left panel; Figure 6 said two models clear 0.79 where three do, and never accounted for its second line; Figure 8's "the only measurements on the page" meant in that figure; Figure 2's axis said "KV still fits 15 GiB" where the bar subtracts weights and overhead too. Figures: Figure 5 drew a 48 GB rung Table 7 has no row for, so it draws five; Figure 3's budget line moves from 15 GiB of weights to 14, since the overhead is inside the same 15; Figure 7's host is named as Figure 8 names it. Duplication: §6's opening sentence restated §2's KV bullet word for word and now points at it; §5 and Table 6 both recounted the 7-points/3-prompts split, so §5 reads the figure and Table 6 keeps the numbers; §2's closing paragraph, Table 3's caption and Table 7's caption each restated a claim already made once. Clarity: the three picks are now named where they first appear (g4-e4b-qat in Table 3, all three in Figure 6 and §8), because "the budget pick" was used five pages before the figure that defines it; §2's title covers its second half; Table 5 says why two roster rows are absent; §14's reproduce block showed only the thinking arm's 8192 cap under a claim about controlled caps, and now shows both. | |||
| 1.18 | 11th November 2026 | Dedup and labels pass; no measured value changed. Removed: the one-line roster summary after the platform table (the header already carries the file count, the axes and 113.8 GiB); §5's third paragraph, which restated the 4k-context warning §3 makes, and the score lists in its other two, so the section now reads the figure — the front's two ends, and a tie that sits twice as far right — and points at §3 and §7 for the numbers; §4's first bullet restated Table 9's first row and now cites it; §10's closing paragraph restated Table 1's flag row and now cites it. Labels: Table 5's per-1k column is marked (fp16) and its four context columns (q8_0), and the caption states the 32k value is half of 32 × the per-1k rate rather than 32 × it — the unlabelled header allowed the opposite reading; Table 7's header names its units, card (GB) against usable (GiB); Table 9's decode row names the two card sizes behind 1008-1792 GB/s; Table 3's transfer pointer read §12, it is §11. Corrected: Table 7's caption said "the 32 GB and 48 GB columns" where the ladder has a 24 GB and a 32 GB+ row; §3's "14 GiB to spare" is now "14 GiB of its 15", because the row's footprint is 13.96 GiB against a 15 GiB budget and it has about 1 GiB left, not 14; §13 quoted 96 W and 76 °C as a batch figure that contradicts the envelope measured and quoted with the platform table (96 W mean against a 129 W peak, 69 °C mean against an 88 °C hottest sample), and now defers to it. | |||
| 1.17 | 10th November 2026 | Corrections pass: the 30B's context in §9 read 13k, it is 4k; §12 counted three NVIDIA boards, Table 10 lists four; the 128-bit decode multiplier read 0.9-1.8x, it is 0.75-1.85x, and the 256-bit group 2.0-2.6x, it is 1.3-2.7x; §10's halving claim read 0.3-2.5 GiB against Table 5's q8_0 columns, which give 0.1-1.25; the §3 decode row now names the 110 tok/s arm as next best instead of skipping it; §2's KV bullet no longer says the 14B only fits 4k; the decision-map sentence placed the quality pick top-left, it is the top-right extreme; Figure 1's caption now names the 1 GiB overhead in the footprint. | |||
| 1.16 | 10th November 2026 | §4 now says which uncertainty it quotes: the three figures there are one standard error, the paragraph defines the 95% confidence intervals the charts draw and says they are about double these. §11's memory figures move to the host's own units — 128 GiB and a 100 GiB GTT window — replacing a 96 GiB figure that no reading on the box produces. | |||
| 1.15 | 10th November 2026 | Corrects the claim that power and temperature were not recorded: the host's monitor logs both at ~2 s throughout, and the platform table now says so with retention. The introduction quotes the campaign's measured envelope — GPU 69 °C mean, 88 °C hottest sample, package power 96 W mean, 129 W peak — so the throughput numbers come with a thermal context instead of an unstated one. | |||
| 1.14 | 10th November 2026 | The platform table gains the OS: distribution, init, C library, and on the kernel row the pstate mode and EPP hint, because on this host the governor is what decides how long the boost is held. | |||
| 1.13 | 10th November 2026 | The platform table gains storage: the NVMe drive and filesystem the weight files live on, with measured direct-read rates. It is in the table to say what it does not affect — weights are read once at load, so storage sets load time, not decode. | |||
| 1.12 | 10th November 2026 | The introduction gains a platform table: CPU, GPU, unified memory and the GTT window, kernel, graphics stack, inference build and server flags, each with the reason it is stated. Power and temperature are named as unrecorded rather than implied to be controlled. The spec list that used to sit in the prose is folded into the table instead of being repeated above it. It is Table 1, so every later table is numbered one higher than before. | |||
| 1.11 | 10th November 2026 | Mean quality and quality per GiB now appear in every table that lists per-model scores: two columns in the roster (§1) and two rows in the controlled pair (§7). Both are marked derived, and both state the divisor is the footprint at the model's context. The ladder (§8) says why it has no per-GiB column. | |||
| 1.10 | 10th November 2026 | Fixes "both platforms" in §4 and in the transferability table: that phrase dates from when the page compared only this box and a discrete card. It now names the three architectures the page actually compares and gives each one's bandwidth range. | |||
| 1.9 | 10th November 2026 | Removes duplication in §2: the axis bullets restated the table's 'what it does not tell you' column and the mean-quality row a third time. The bullets now carry only the incentive each axis creates, and the table keeps scope. | |||
| 1.8 | 10th November 2026 | The axis table gains the two derived metrics the rest of the page ranks on — mean quality and quality per GiB — marked as arithmetic rather than benchmarks, with what each can and cannot tell you. | |||
| 1.7 | 10th November 2026 | Defines cap, budget, footprint and KV cache in §2 alongside the axes — the four terms every number on the page is qualified by, none of which was explained where it was first used. | |||
| 1.6 | 10th November 2026 | Adds §2: what HumanEval, MMLU-Pro, IFEval, prefill/decode and the riddle probes actually measure, how each is scored, what was run here, and what each one cannot tell you — before any table of scores. Sections and tables renumbered. | |||
| 1.5 | 10th November 2026 | Names the author and copyright holder (Dom, github.com/barmynon) instead of leaving the person behind the measurements anonymous, and adds a CITATION.cff so the work can be cited. | |||
| 1.4 | 10th November 2026 | Title now says local LLMs. Adds the licence section: what CC BY 4.0 covers here (expression, figures, compilation and the sui generis database right), what it cannot cover (the facts, the weights, the benchmarks), and the attribution line. | |||
| 1.3 | 10th November 2026 | Adds a decision map (Figure 6): mean quality against footprint with the 15 GiB wall, a stated 0.78 competitive line and the three picks ringed, so "where is good" is visible without reading a table. Deliberately not a quadrant — no median crosshair, because the good direction here is up and to the left. Figures renumbered; the decode figure now marks the region that fits the card. | |||
| 1.2 | 10th November 2026 | the card comparison extended to thirteen cards across four vendors: five AMD boards added (RX 7600 XT, 9060 XT, 6800, 6950 XT, 7900 GRE), so the class can be compared within and across vendors, and the reference machine named as the Minisforum MS-S1 MAX instead of "this box". | |||
| 1.1 | 10th November 2026 | Adds the 16 GB card section: the 16 GB cards actually on the market, their vendor bandwidth, and the decode range each implies for the roster's two reference models. Predicted, not measured — the method and its calibration are stated with it. | |||
| 1.0 | 10th November 2026 | First release. Eleven files measured for the 16 GiB class on a host with no VRAM ceiling, three axes at stated caps, fit and KV arithmetic from the same roster file the runs used, six computed figures, the three-platform bandwidth comparison. |
The version number belongs to this page. The wider report on this platform has its own history in the change log; a correction to a number here bumps the version above and states what the previous value was, on the same rule the main report follows.
Source, data and build scripts: github.com/strix-halo-128-benchmarks/strix-halo-128-benchmarks.github.io. Text, figures and data: CC BY 4.0 · scripts/ and tools/: MIT.