Retired local models — restore recipes
Part of Local LLM benchmarks on AMD Strix Halo. Every deletion here is reversible: each entry carries file sizes, sha256s, the upstream repo and the flags that worked.
Kept so a deletion is reversible without re-researching URLs, hashes and flags.
Instella-MoE-16B-A3B-Think (retired 2026-10-12)
Removed at the user's request: the weights, its unit, run script, switch entry, log files, and the bespoke instella-moe llama.cpp fork tree it needed.
What was removed:
~/llama-local/models/Instella-MoE-16B-A3B-Think-Q4_K_M.gguf 10,470,249,600 B (9.75 GiB) ~/llama-local/llama-instella-test/ 644 MB (fork build tree) ~/llama-local/run-instella.sh the unit's ExecStart ~/llama-local/instella-download.log, instella.log, instella-server.log ~/.config/systemd/user/llama-instella.service (was disabled, never the boot unit) llama-switch entry instella (:8087) + its usage string llama-instella.service from every other unit's Conflicts= line
Reason to distrust that copy anyway: the local file was 10,470,249,600 B while the current HF artifact is 10,470,249,472 B — 128 bytes larger, so the on-disk file did not match the published artifact. Not a byte-for-byte clean download.
Restore (weights are public; the arch is NOT in upstream llama.cpp or the halo-box fork, so the bespoke fork is required — see the skill's references/instella-moe.md for the build):
cd ~/llama-local/models && wget -c \ https://huggingface.co/NANI-Nithin/Instella-MoE-16B-A3B-Think-GGUF/resolve/main/Instella-MoE-16B-A3B-Think-Q4_K_M.gguf # expected: 10,470,249,472 B, sha256 8b1f2759d3415850ffc3b87edf2c339936718805651f09945016e8676a135f81 # (verify with sha256sum against that LFS oid before serving)
Then re-create the unit on :8087 (free again now) with the usual template, re-add llama-instella.service to every peer unit's Conflicts= line, rebuild the fork, and add the instella case back to llama-switch.
Hermes-3-Llama-3.1-70B-Uncensored — retired 2026-10-13, never served
~/llama-local/models/Hermes-3-Llama-3.1-70B-Uncensored.i1-Q4_K_M.gguf 42,520,399,008 B (39.6 GiB) ~/llama-local/models/Hermes-3-70B-uncensored/ partial fp16 tree (1.8 GB) ~/llama-local/fetch-hermes3-gguf.sh, fetch-hermes3-70b.sh, run-hermes3-70b.sh ~/llama-local/hermes3-70b-download.log, hermes3-70b-gguf-download.log ~/.config/systemd/user/llama-hermes3-70b.service (:8087; wired, never started) llama-switch entry hermes3-70b + its usage string llama-hermes3-70b.service from every other unit's Conflicts= line
Deleted on request while the download was at ~90%: the unit was built and never started, so no run data, no benchmark, and no log worth keeping. Port :8087 returned to the free pool.
Restore (weights are public; the arch is plain llama, so the halo-box fork serves it — no bespoke build. This is a 70B DENSE model: 80 layers, 64 heads / 8 KV heads, so KV is 160 KiB/token at q8_0 and it wants the 100 GiB GTT pool to itself: 39.6 GiB weights + 20 GiB KV at 131072 ctx):
~/llama-local/hf-fetch.sh mradermacher/Hermes-3-Llama-3.1-70B-Uncensored-i1-GGUF Hermes-3-Llama-3.1-70B-Uncensored.i1-Q4_K_M.gguf # expected 42520399008 B, sha256 ca2e201e8a780b9e40d3b59c95756f8a5364443f783d802696fb828d40e82747 # (hf-fetch.sh verifies size + sha256 against the API's LFS metadata) # run script: llama-server -m <file> --host 127.0.0.1 --port 8087 --alias hermes3-70b \ # -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 \ # --spec-type none --reasoning off -np 1 --metrics
The upstream repo the model was requested from (Guilherme34/Hermes-3-Llama-3.1-70B-Uncensored) is fp16 safetensors only — 131.43 GiB across 30 shards, no GGUF — so serving it from source needs convert_hf_to_gguf.py (torch + transformers, neither installed) plus llama-quantize. The quant used above is mradermacher's imatrix build of that exact fine-tune (base_model declared on its card). Its model card also says the "uncensored" behaviour only appears with a specific system prompt; that prompt was never set on any channel here.
Meta Muse Glimmer-30B — retired 2026-10-13
Dense 30B causal LM + perception encoder, Apache-2.0, Meta Superintelligence Lab, Aug 2026 (distilled from "Muse Spark", agentic framing). Retired after a full evening's evaluation.
Restore recipe — repo meta-models/Muse-Glimmer-30B-GGUF, all four files, and sha256-check each after fetching (these hashes were computed from the verified local copies; the HF lfs.oid is the same value by definition, and each prefix matched what the fetch logged at download time):
Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf- 19,653,960,832 bytes, sha256
ac7023d6a4c704eb9af54ab53e476a66b7f5b6c0ef2fc4a8dde5253c291a6c38 - https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF/resolve/main/Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf
- 18.30 GiB, the default quant; the one every measurement below used
Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf- 16,756,683,904 bytes, sha256
4cc57c0f51040a226e5a72cc47b7613f7772950e460a665f7083de89f183f60e - https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF/resolve/main/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf
- 15.61 GiB, the smaller/faster-quality-trade variant; never benchmarked
dflash-Muse-Glimmer-30B-Q4_K_M.gguf- 1,631,208,128 bytes, sha256
b2e808bf656086fe86bd0d0bd990f01d33e377537a07c02d45371517c8b264ef - https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF/resolve/main/dflash-Muse-Glimmer-30B-Q4_K_M.gguf
- 1.52 GiB DFlash drafter; architecture
dflash, 58 tensors, 5 layers mmproj-Muse-Glimmer-30B-Q4_K_M.gguf- 1,400,328,928 bytes, sha256
f48b452316f9b213758e8659444029b961a24a07f99a1abb2a9f88b06f7c00c6 - https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF/resolve/main/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf
- 1.30 GiB vision projector; verified 3/3 on the shape probe
- unit
llama-muse-glimmer.service, port :8098, aliasmuse-glimmer, scriptrun-muse-glimmer.sh - flags:
-m <KQuant> --mmproj <mmproj> -ngl 99 -c 131072 --flash-attn on --cache-type-k/v q8_0 --reasoning auto -np 1 --metrics; the script carries aVISION(default on) and aDFLASH=0|1switch - arch
muse_glimmerworks on the halo-box fork (src/models/muse-glimmer.cpp) and mtmd has amuse_glimmerclip graph; served weights were 18.3 GiB with only ~14.7 GiB of GTT in use
Why retired — measured, not assumed:
| axis | result |
|---|---|
| speed, 3x512 greedy probe (median) | 10.77 t/s — perf axis 10.9, pp 374, ttft 4.93 s |
DFlash (--spec-type draft-dflash) | 0 draft tokens — spec_decode_num_draft_tokens_total 0 after a 256-token generation; tg identical to 0.00%. Invocation verified correct via bash -x, sidecar verified real. The card's 1.5x-3.1x is untestable here |
| riddles (core, default cap) | 8/14, weakest in the fleet; -rea off did not change it |
| long-form answers | "count to 40 in words" returns EMPTY content after burning 256 tokens on a prompt echo — with and without -rea off |
| needles | PASS / PASS |
-rea offA/B (server-side--reasoning off, same weights): riddles 8/14, identical, perf pp 366.8 / tg 10.9, and the long-form prompt still returns EMPTY content. So the weak core score is the model as served, NOT a reasoning-budget artifact — my interim hypothesis was wrong and this is the check that refuted it. (--no-reasoning-preserveremains untested;-rea offis the flag that made no difference.)
Reason it went: no niche. Dense 30B at Q4 on a bandwidth-bound box is inherently ~11 t/s here, its one distinctive feature produced nothing, and big + uncensored + vision + MTP is already covered by the Flash-Next GSQ-RCO abliterated (:8097) at roughly 3-5x the throughput and less GTT.
2026-10-17 batch — six superseded models (disk-audit queue, ~/llama-local/DELETE-QUEUE.md)
Retired on the user's explicit instruction after the four eligibility rules were applied: not a former daily driver (>12 h), not favoured, no kept model depends on the files, results recorded first. Every local artifact was hashed before deletion and 11 of the 12 matched the repo's published LFS oid exactly — the one exception is the Fable weights, whose source repo was never recorded (noted below).
Units, run scripts, llama-switch cases and the Hermes aliases pointing at them were removed in the same change (registry 19 → 13 units, Conflicts= regenerated and verified 13 × 12, no mismatches).
Qwen3.8-27B TurboFCFusion-735-882-Here-Uncen-NEO-CODER-MAX-MTP ("Fable") — retired 2026-10-17
Files (size, sha256): ~/llama-local/models/Qwen3.8-27B-TurboFCFusion-735-882-Here-Uncen-NEO-CODER-MAX-MTP-Q4_K_M.gguf 18,498,573,856 B (18.50 GB) sha256 bc7a6cf2bcc78d1190aaf04d1ab1c5cb845b6ff23aa0e7d24fe0d2ea6d3a7c7c ~/llama-local/models/mmproj-F16.gguf 927,606,976 B (0.93 GB) sha256 82e620db8cb83267e9775e5aad3e8d8aa5af7ace825e94c52d3d730eb35af88a
Measured: pp 367, tg 12.0 t/s, ttft 5.0 s, riddles 11/14 — UNRANKED: never completed an axis
Source / restore: source repo NOT recorded at fetch time. The only TurboFCFusion GGUF repo the Hub now shows is bbkdevops/Qwen3.8-27B-TurboFCFusion-CyberSec, and it publishes IQ4_NL (17.75 GB), not this Q4_K_M. Restoring this exact file requires finding it by basename; the IQ4_NL is the closest available substitute.
Evidence on file: 2026-10-12 suite + 2 run artifacts in ~/runs Also removed: unit llama-qwen38-27b-fable.service (: 8095), its run script, its llama-switch case, its log, and any Hermes alias pointing at :8095.
Qwen3.6-35B-A3B UD-Q4_K_XL — retired 2026-10-17
Files (size, sha256): ~/llama-local/models/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf 22,360,456,160 B (22.36 GB) sha256 707a55a8a4397ecde44de0c499d3e68c1ad1d240d1da65826b4949d1043f4450
Measured: HE 0.8354 (cap 1024), MMLU-Pro 0.4500 (512), IFEval 0.867, riddles 12/14, pp 1548, tg 57.1
Source / restore: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF/resolve/main/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
Evidence on file: row in the 2026-10-12 eight-model suite; 4 run files in ~/runs Also removed: unit llama-qwen36-35b.service (: 8092), its run script, its llama-switch case, its log, and any Hermes alias pointing at :8092.
Qwen3.8-27B Q6_K_L (Hermes alias qwen38) — retired 2026-10-17
Files (size, sha256): ~/llama-local/models/Qwen3.8-27B-Q6_K_L.gguf 24,958,908,128 B (24.96 GB) sha256 e8750bb81ba49f90eb68df99776b250dbc14666043098952de6422cbecd77a21
Measured: never scored on HumanEval or MMLU; riddles-only data from 2026-09-03/09-11 passes; ~9 t/s
Source / restore: https://huggingface.co/bartowski/Qwen3.8-27B-GGUF/resolve/main/Qwen3.8-27B-Q6_K_L.gguf
Evidence on file: 7 run files in ~/runs (riddle/depth passes only) Also removed: unit llama-qwen38.service (: 8088), its run script, its llama-switch case, its log, and any Hermes alias pointing at :8088.
Kolibri-1 78B-A3.46B Q5_K_M — retired 2026-10-17
Files (size, sha256): ~/llama-local/models/Q5_K_M/Kolibri-1-Q5_K_M-00001-of-00002.gguf 44,739,588,864 B (44.74 GB) sha256 857961501f9fc8847f37535c8cdfe0cd4c3ab064bb4cadbd4896c3d3824712f9 ~/llama-local/models/Q5_K_M/Kolibri-1-Q5_K_M-00002-of-00002.gguf 10,849,180,800 B (10.85 GB) sha256 929c6b679d7739e7617089a346d499c8b28f8699f7247432d0f5ebdf4f9252a2
Measured: HE 0.8293 (512, 0 rows at cap), MMLU-Pro 0.4000 (512, think-off arm), riddles 11/14 real, pp 1275-1295, tg 60.5 Only German-capable (En/De) model on this box — if German is ever needed, this is what to restore first.
Source / restore: https://huggingface.co/Hob-forge/Kolibri-1-GGUF/resolve/main/Q5_K_M/<shard> (2 shards, same dir)
Evidence on file: references/2026-10-14-kolibri-kimi-round.md + 6 run files Also removed: unit llama-kolibri.service (: 8098), its run script, its llama-switch case, its log, and any Hermes alias pointing at :8098.
Qwen3.5-122B-A10B UD-Q4_K_XL — retired 2026-10-17
Files (size, sha256): ~/llama-local/models/Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf 10,943,552 B (0.01 GB) sha256 467c9bd92ea518539cf75bf5a5fbfbd35e9a0b40d766ccaa67bf120e12041df3 ~/llama-local/models/Qwen3.5-122B-A10B-UD-Q4_K_XL-00002-of-00003.gguf 49,640,779,424 B (49.64 GB) sha256 ecdbd42d43b0df9fa0ef9a584e09e95a43966ef03a122aba0b87a99d44d9ad98 ~/llama-local/models/Qwen3.5-122B-A10B-UD-Q4_K_XL-00003-of-00003.gguf 27,378,273,056 B (27.38 GB) sha256 13300e0f059e6fa21aa0fabde2a554f9deea366c0e54f268045769b214b28c97
Measured: HE 0.9242 (n=66), MMLU-Pro 0.5417 (512), IFEval 0.917 — best of any local model measured here, riddles 12/14, pp 622, tg 22.9 (≈3 min to first token at 4k prompt: the reason it lost its slot)
Source / restore: https://huggingface.co/unsloth/Qwen3.5-122B-A10B-GGUF/resolve/main/UD-Q4_K_XL/<shard> (3 shards)
Evidence on file: row in the 2026-10-12 suite; 5 run files Also removed: unit llama-qwen35moe.service (: 8084), its run script, its llama-switch case, its log, and any Hermes alias pointing at :8084.
Qwen3.8-Flash-Next UD-Q2_K_XL — retired 2026-10-17
Files (size, sha256): ~/llama-local/models/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf 10,946,624 B (0.01 GB) sha256 a4f3b21e77353999829f2f767e9ac21ce9c71d29a74f2cc9eda48c9bf23c8b86 ~/llama-local/models/Qwen3.8-Flash-Next-UD-Q2_K_XL-00002-of-00003.gguf 49,979,779,296 B (49.98 GB) sha256 2e3bf1ee7d2a04e261e9f342a2d968f696cce5941d082b0e434deb9b1edc12c6 ~/llama-local/models/Qwen3.8-Flash-Next-UD-Q2_K_XL-00003-of-00003.gguf 28,878,402,944 B (28.88 GB) sha256 ec8c106759fdf4f463039c34c0707718d7d8908d53d892bd4f002e71620803f9
Measured: never scored on HumanEval or MMLU on the current suite; riddles 5/5 on the 2026-09-11 pass; ~30 t/s; 79 GB spent on a 2-bit quant. The measured Flash-Next numbers on this box belong to its protected abliterated sibling (":8097", IQ3_S/), which is kept — that is where to look for real scores.
Source / restore: https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF/resolve/main/UD-Q2_K_XL/<shard> (3 shards)
Evidence on file: references/qwen38flashnext-20260911.md + 8 run files Also removed: unit llama-qwen38fn.service (: 8089), its run script, its llama-switch case, its log, and any Hermes alias pointing at :8089.
Batch total: 278233439680.00 GB of weights (plus 9.67 GB of junk removed in the same pass). Disk after: see df -h /home; the btrfs snapshot holds roughly 30 GB back until it rotates.
Restore procedure (generic): re-download to the exact path above, sha256sum it against the recorded hash, then re-create the unit from the standard template, re-add it to every peer's Conflicts= line, add the llama-switch case, and — if a Hermes alias was removed — re-add providers.<name> plus model_aliases.<name> and restart the gateway.
flint (RecursiveSI/flint) — retired 2026-10-24, measured, not dominated
Why retired: it is a good model that is still strictly dominated by the daily driver. HumanEval 0.9167 (55/60) at cap 4096 vs gemma4-26b QAT 0.9390 and Ornith-122B 0.9512; riddles 10/14 vs 14/14 for gemma/Ornith/Kimi; decode 60.4 t/s vs 67 (gemma QAT) and 70 (Ornith 35B). Its only distinguishing axis, 262k native context, is covered by Kimi-Linear (1M, faster, 14/14).
Measured before deletion (t20, same harness, cap 4096):
- perf: pp 1433 t/s (1914 on the second run), tg 60.4 t/s, ttft 1248 ms @1789-token prompt
- preflight 3/3, needle 2k PASS, needle 8k PASS, riddles 10/14 (4 misses = EMPTY content, cap-limited)
- humaneval pass@1 = 0.9167 (55/60), wall 1061 s
- raw:
~/runs/20261025-063840-flint-q4-official.jsonl,~/runs/20261025-070907-flint-q4-he60.jsonl, logs~/llama-local/t20-flint-q4-official.log,~/llama-local/t20-flint-he60.log
Restore recipe
- weights:
flint-preview-UD-Q4_K_XL.gguf, 25,054,688,704 B, sha25663c2ecc461ca545134b670c660e76fa20f47240f8d39b6510bfd21c4dd6e072f=unsloth/flint-GGUF→flint-UD-Q4_K_XL.gguf(repo LFS oid 63c2ecc461ca). Note the local filename says "preview" but the sha IS the official UD-Q4_K_XL; do not re-fetch a "release". Other levels in that repo: IQ1_M 10.57, IQ2_XXS 11.04, Q2_K_XL 12.63, IQ3_XXS 13.54, Q3_K_XL 17.22, Q5_K_XL 27.86, Q6_K_XL 33.83, Q8_0 39.40, BF16 74.12 GiB. - unit
llama-flint-preview.service, port 8183 (now free), aliasflint, scriptrun-flint.sh:-ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --jinja -np 1 -b 4096 -ub 2048 - arch
flint(49 blocks, 2 dense + 47 MoE, 256 experts / 6 used / 2 shared, hidden 2048, ctx_train 262144, sliding window 513 on 35 layers, 32 q / 4 kv heads). - bespoke build required: arch
flintis NOT in the halo-box fork. Built tree wasflint-pr-llama.cpp(PR #30928, commit 1b44e30); it was deleted with this retirement because the only other script pointing at it (run-mammoth.sh) had no weights left either (also removed). Rebuild: clone llama.cpp, apply/merge PR #30928, Vulkan configure (see the fork's configure line), targetsllama-server. - Also removed with it:
run-mammoth.sh(orphan —models/mammoth-1.*were already gone).
sage-a1.5 and groot-19b-a3b — retired 2026-10-25 (measured, neither was placed)
Both fetched, wired (units 25/26, ports 8183/8189) and fully measured the same day, then retired: neither takes the daily-driver slot, and only sage had a defensible niche (fast vision + 262k ctx). User decision: delete both.
Measured before deletion (all on the halo-box fork, ctx 131072, q8_0 KV, Q4_K_M)
- Sage-A1.5, thinking OFF (
--chat-template-kwargs '{"enable_thinking":false}'): pp 1690 / tg 74.6 / ttft 1064 ms @1799 tok; preflight 3/3; needle 2k PASS, 8k PASS; riddles 14/14; HumanEval 0.8659 (142/164) in 270 s (1.6 s/item); MMLU-Pro 0.6250 (75/120) in 992 s. A--limit 60subset read 0.9167 (55/60) — ~5pp optimistic, do not compare to 164-item cells. - Sage-A1.5, thinking ON (template default, UNUSABLE): 20/20 HumanEval FAILs at ~53 s/item, every item burning the full 4096 cap and ending
NameError; riddles 7/14. One flag = 1.6 s vs 53 s per item. - Groot-19B-A3B (default arm): pp 1876 / tg 78.2 / ttft 958 ms; preflight 3/3; needle 2k+8k PASS; riddles 8/14; HumanEval 0.9333 (56/60) at 15.6 s/item — subset only, the full 164-item axis was never run (record as UNRANKED on the full axis); MMLU-Pro never run.
- vs the incumbent: gemma4-26b QAT is HE 0.9390 (164) / MMLU 0.8417 (120) / riddles 14/14 at 66.7 t/s.
Restore recipe
sage-a1.5-Q4_K_M.gguf— 21,713,463,520 B (20.22 GiB), sha256a64fe0bbf936db3eb54f66f142c2b2eee569ade83673649f549a21daef8ca395fromInternScience/Sage-A1.5-Q4_K_M-GGUF :: Sage-A1.5-Q4_K_M.ggufmmproj-Sage-A1.5.gguf— 899,282,880 B (0.84 GiB), sha25651cd4410057e8dc4152ca771a726fbfb34856f047e7049018c757d42ddd481f2fromInternScience/Sage-A1.5-Q4_K_M-GGUF :: Sage-A1.5-mmproj.ggufgroot-19b-a3b-Q4_K_M.gguf— 11,657,618,560 B (10.86 GiB), sha2560716cae1f19a39a786a463f7b8c068d50b3d7a74532fce73e5ea23ed98130796fromGrootTeam/Groot-19B-A3B-261023-GGUF :: Groot-19B-A3B-261023-Q4_K_M.ggufmmproj-Groot-19B-A3B-BF16.gguf— 902,822,016 B (0.84 GiB), sha25605a5c8a279b2589306824864d76d663063ba69ff1f399e6a283fae988fdff360fromGrootTeam/Groot-19B-A3B-261023-GGUF :: mmproj-Groot-19B-A3B-261023-BF16.gguf- units
llama-sage-a15.service(port 8183, aliassage-a1.5) andllama-groot-19b-a3b.service(port 8189, aliasgroot-19b-a3b); scriptsrun-sage-a15.sh,run-groot-19b-a3b.sh. - arch
qwen35moefor both (41 blocks; sage 256 experts top-8, groot 128 top-8; ctx_train 262144) — loads on the halo-box fork with no build. Vision needs the mmproj above (--mmproj), which loads as "loaded multimodal model" although this fork's/propsexposes novisionkey. - flags:
-ngl 99 -c 131072 --flash-attn on --cache-type-k/v q8_0 --spec-type none -np 1 --jinja --metrics; sage additionally--chat-template-kwargs '{"enable_thinking":false}'(see above — without it the model is unusable). - artifacts:
~/llama-local/t20-sage-nothink-core.log,t20-sage-a1.5-core.log,t20-groot-19b-a3b-core.log,he-sage-full.log,he-sage-nothink.log,he-groot-19b-a3b.log,mmlu-sage-full.log;~/runs/holds the t20 rows.
ornith-122b-reap50 (REAP50 + UD-IQ2_XXS of Ornith-1.5-122B) — retired 2026-10-25, failed preflight
Fetched and tested on the user's instruction as the one candidate on the board that could improve the roster (a quarter-size version of the box's #2 model). Rejected: it loops, and preflight catches it.
Header (local peek, not the card)
- arch
qwen35moe, expert_count 128 vs the base 122B's 256, 48 blocks vs 49, ctx_train 262144, 32 q / 2 kv heads. Genuinely pruned, not a rename.
Measured on the halo-box fork, ctx 32768, q8_0 KV, flags mirrored from run-ornith-122b.sh
- perf: pp 604 t/s · tg 37.2 t/s · ttft 2976 ms @1797-token prompt — 1.6x the base 122B's 23.3 t/s at ~15 GiB resident instead of ~74 GiB.
- preflight 1/3: content_nonempty PASS;
json_schemaFAIL (emitted the identical JSON blob four times back to back);instruction_one_wordFAIL (argued with itself about whether a trailing period counts as part of the word). - needle 2k PASS and 8k PASS, but both replies echo the needle repeatedly with stray
</think>fragments. - riddles: aborted with 0 items scored — the first item did not finish inside five minutes.
- HumanEval and MMLU-Pro were never obtained (the run was stopped after preflight failed). Their absence is deliberate, not a gap: do not read this model as having a code score.
- The repo's own
eval/summary.jsonclaims HE 0.9390 (154/164) and MBPP 0.8042 vs base 0.9451 / 0.8095, measured reasoning on, cap 32768, temp 0.6, parallel 8 — a protocol that flatters a looping model and is not comparable to the t20 cells here (cap 4096, temp 0).
Restore recipe
ornith-122b-reap50-IQ2_XXS.gguf— 21,415,883,744 B (19.95 GiB), sha2566c0e926ef3f8f963555927aea5937a72fb1d737d33de58e4d6ed46fcaf2da739fromanik-jha/Ornith-1.5-122B-24GB-GGUF :: Ornith-1.5-122B-24GB-REAP50-UD-IQ2_XXS.gguf- unit
llama-ornith-122b-reap50.service, port 8183 (free again), aliasornith-122b-reap50, scriptrun-ornith-122b-reap50.sh:-ngl 99 -c 32768 --flash-attn on --cache-type-k/v q8_0 -np 1 --jinja --metrics - TEXT ONLY — the repo ships no mmproj, unlike the base 122B.
- artifacts:
~/llama-local/t20-reap50-core.log,test-reap50.out,ornith-122b-reap50.log.
raon-1-mini-preview (KRAFTON, hybrid MoE) — tested and retired 2026-10-26
Fetched and fully measured on the user's instruction ("get and test 1,2 and 3"). Rejected on quality: it lands on mellum21-fork's HumanEval but loses on knowledge and riddles, and is slower than the deployed driver on every axis.
What it is
- 128 routed experts, 25 blocks, hidden 2560, 18 linear-attention + 7 full-attention layers, ctx 262144, arch
raon1_mini/Raon1MiniForCausalLM. - Requires KRAFTON's fork:
LLM_ARCH_RAON1exists in github.com/krafton-ai/llama.cpp and nowhere else — not in our fork, not in prism, not in ggml-org master, no open PR. Upstream support "pending" per their own model card. Read this as a standing maintenance cost: any other work on our fork does not carry this arch. - Their repo ships one extra file,
dflash-raon-1-mini-preview-Q8_0.gguf(0.68 GiB), unused here.
Measured (KRAFTON fork, ctx 32768, q8_0 KV, -ngl 99, flash-attn on)
- perf: pp 515 t/s · tg 46.9 t/s alone (19.5 t/s measured concurrently with another bench — always re-measure alone) · ttft 2108 ms @1086-token prompt.
- preflight 3/3 PASS; needle 2k PASS, 8k PASS.
- HumanEval 0.9146 (150/164), 17.6 s/item, wall 2878 s.
- MMLU-Pro 0.7750 (93/120), 24.7 s/item, 17 cap-burns (4% of items returned empty after burning the full 4096).
- riddles 8/14 — three failures were empty/cap-burn, two genuinely wrong (including a reverse-alphabet item where it wrote out the substitution correctly and still mis-decoded it).
- Verdict vs driver (gemma4-26b-a4b QAT): HE -2.4 pp, MMLU -6.7 pp, riddles 8 vs 14, and 46.9 vs 59.5-66.7 t/s. No seat.
- Thinking stays in
reasoning_contentand is short (~25 tokens), which is why it is efficient per item despite being weaker — a template that other thinking models on this box should copy.
Restore recipe
raon-1-mini-preview-IQ4_XS.gguf— 16,071,801,344 B (14.97 GiB), sha25685570a704f9adc0df8b5b64c4e74ce956a5519e4868636d5daa00a84eb30696ffromKRAFTON/Raon-1-mini-preview-gguf(also Q4_K_M 18.33, Q5_K_M 20.66, Q6_K 24.06, Q8_0 28.59)- build:
~/llama-local/krafton-llama.cpp(cloned, Vulkan-built, 15,976-byte llama-server) - serve:
~/llama-local/run-raon-1-mini.sh(port 8181, aliasraon-1-mini) - bench:
~/llama-local/bench-raon.sh; logst20-raon-core.log,he-raon.log,mmlu-raon.log
spark-x26-4b (XHToken, 4B dense, 1M ctx) — tested and retired 2026-10-26
Fetched because a 4B with a 1,048,576-token context was the only cheap way to fill a fast long-context lane. The context is real; the model is not good enough to serve.
What it is
- 4B dense, 36 blocks, hidden 2560, 16 heads / 4 kv (
spark2_5.block_count), declaredcontext_length1048576. Loader:LLM_ARCH_SPARK2_5already present in our fork and in upstream master — no rebuild needed, which made it the cheapest of the three to test.
Measured (fork, q8_0 KV, -np 1)
- perf: pp 2263 t/s @1857-token prompt · tg 60.7 t/s · ttft 820 ms.
- At long context prefill collapses: 455 t/s at 234,720 prompt tokens (a full 1M prompt would take ~37 minutes).
- preflight 3/3 PASS; needle 2k PASS, 8k PASS.
- HumanEval 0.6833 (41/60) on a 60-item subset, 36.0 s/item — 17 of 60 items burned the full 4096 cap and returned empty. The score understates capability, but it is what an agent gets at a realistic cap; the subset carries ~3.5 pp standard error.
- MMLU-Pro abandoned: 9 items in 20 minutes, 3 pass, near-every item cap-burning. Do not read this as a measurement gap — run it only if someone first fixes the cap-burn behaviour.
- IFEval not run: there is no standalone
~/bench/ifeval.py; IFEval on this box is an lm-eval task, and the archive's column comes from that path. - riddles 10/14 (three of the four failures were empty/cap-burn).
- context: loads at a true n_ctx 1048576 with ~20 GiB of KV (GTT 35.5 -> 55.5 GiB), and retrieved a planted passphrase at 234,720 tokens in 515 s. Live but not useful at that cost, with Kimi-Linear already holding the long-context lane.
Restore recipe
spark-x26-4b-Q4_K_M.gguf— 3,543,441,312 B (3.30 GiB), sha25691cb2811aebb47ac916aa619d27d4cdd7f4143106b7917accb8ead4cd7f5cbd9fromXHToken/Spark-X2.6-4B-GGUF(also Q8_0 4.07 GiB, BF16 7.66 GiB)- serve:
~/llama-local/run-spark-x26-4b.sh(port 8180, -c 65536) orrun-spark-1m.sh(-c 1048576) - bench:
bench-spark.sh/bench-spark2.sh; long-context:needle-long.py <port> <tokens> - logs:
t20-spark-core.log,he-spark60.log,mmlu-spark60.log,needle-200k.log
sensenova-u15-8b-mot (SenseNova U1.5 8B MoT) — NOT RUNNABLE, deleted 2026-10-26
Never ran, and no amount of local work changes that without upstream effort.
- Every public GGUF for this model (
realrebelai/SenseNova-U1.5-8B_GGUFs,smthem/SenseNova-U1-8B-MoT-Merger-gguf) declaresgeneral.architecture = wanwith only 4 metadata keys and tensors namedfm_modules.vision_model_mot_gen.*,fm_modules.noise_scale_embedder.*,patch_embedding.*. That is Wan video-diffusion packaging, not an autoregressive LLM. The loader rejects it before anything else withtensor name 1 is too long: 65 >= 64, which is a red herring: even reading the names shows it is the wrong architecture. - ggml-org master registers
LLM_ARCH_SENSENOVA_U1(merged 2026-10-21, PR #28919) and has allama_model_sensenova_u1class, butconvert_hf_to_gguf.pyhas no sensenova or wan converter — so the official 32.7 GiB safetensors cannot be turned into a loadable GGUF here. Testing it means writing a converter first. - Deleted:
sensenova-u15-8b-mot-Q5_K_M.gguf— 15,107,169,664 B (14.07 GiB), sha2561c496256eb114a5ff8fef278a63b39a75bd0c36e76f4280e89a06bb6ecb76ade. - Kept:
~/llama-local/sensenova-llama.cpp(upstream master, Vulkan-built) for when a converter appears. - General lesson: check
general.architecturein a GGUF header before downloading anything big. A repo named "*-GGUF" under a model's name can still be diffusion weights for ComfyUI. Onehead -c 4M+ header read costs seconds; the Q5 file here cost 14 GiB and an hour.
nanbeige-33b (Nanbeige4.3-33B Q4_K_M) — measured, dominated, deleted 2026-10-26
Fetched at the user's request, measured on all axes, then retired on "retire and delete". Not a broken model — clean preflight 3/3, both needles PASS, zero MMLU cap burns — it simply loses to models already on disk.
Measured (bartowski imatrix Q4_K_M, 19.36 GiB, fork Vulkan, :8183, -c 32768, q8_0 KV, cap 4096):
HumanEval 0.8963 (147/164) 169 min, 62.0 s/item, mean 640 tok, 9 cap burns MMLU-Pro 0.7167 ( 86/120) 53 min, 26.5 s/item, mean 270 tok, 0 cap burns riddles 13/14 · preflight 3/3 · needle 2k+8k PASS perf pp 352.1 t/s · tg 10.4 t/s · ttft 5.19 s (dense 33B, bandwidth-bound)
Why it went: gemma4-31b is the same size class and scores +6.7 pp HumanEval, +15.0 pp MMLU-Pro, 14/14 riddles, and decodes 1.6x faster; the gemma4-26b-qat driver beats it on all three while running ~6x faster. Decode at 10.4 t/s is the bandwidth law for 19.4 GB read per token, so no tuning recovers it. Ranks 14/26 HumanEval, 11/25 MMLU-Pro, last of 26 on decode.
What was removed:
~/llama-local/models/Nanbeige_Nanbeige4.3-33B-Q4_K_M.gguf 20,788,825,216 B (19.36 GiB) ~/llama-local/models/.cache/huggingface/ hf download staging ~/llama-local/run-nanbeige-33b.sh the unit's ExecStart ~/.config/systemd/user/llama-nanbeige-33b.service llama-switch entry nanbeige-33b (:8183) + its usage string llama-nanbeige-33b.service from every other unit's Conflicts= line transient bench units bench-nanbeige, bench-nanbeige-he
Restore:
cd ~/llama-local/models && wget -c \ https://huggingface.co/bartowski/Nanbeige_Nanbeige4.3-33B-GGUF/resolve/main/Nanbeige_Nanbeige4.3-33B-Q4_K_M.gguf # expected 20,788,825,216 B, # sha256 2cef79855ca47ee6571e166db32a07089e34f3cf78e0397b262d124ddb814c37
# then recreate the unit; the flags that matter are the plain ones — arch is llama, so the # stock fork loads it with no special build: # --port 8183 -ngl 99 -c 32768 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 # --jinja --reasoning off -np 1 -b 4096 -ub 2048 --metrics # (no MTP/drafter ships for this base; there is no spec-decode arm)
- Note for a future uncensored run:
dealignai/Nanbeige4.3-33B-UNCENSORED-GGUF(Q4_K_M 18.7 GiB) exists and was NOT tested. It inherits this model's speed class (~10 t/s dense), so judge it only as a §16 abliteration datapoint, not as a daily driver. - Measurement hygiene found while ranking: three catalogue rows
20261010-17*carry HumanEval 1.000 and MMLU 1.000 for qwen3next-q6 at n=2 — assertion smoke tests, not measurements. Filter onhumaneval_n >= 50before ranking, or they sit at the top of every table.
————— Pruning pass 2026-10-30 (retire and delete, record kept) —————
Eight artefacts retired, 245.1 GiB. Two further candidates were HELD mid-pass (NVIDIA-Nemotron-3.5-Super-VL-120B, LFM2.5-24B-A2B): the numbers I proposed them on were cap-demoted, and at cap 4096 both read far better than quoted. See the last section.
Swift-Qwen3.8-Flash-Next-GSQ-RCO IQ2_XS (abliterated v2, huihui-ai) — retired 2026-10-30 [T1, 63.5 GiB]
Why it went: Capability never measured — both runs on it are perf-only, so there is no HumanEval or MMLU-Pro row for this exact file. It is a second, lower-quant encoding of the model :8097 already serves as IQ3_S (which is protected), so nothing on the box loses a capability.
Measured:
perf tg 31.2-31.5 t/s (20261027-111946-flashnext-smoke, 20261027-112508-flashnext-abl, :8190) riddles 13/14 · HumanEval none · MMLU-Pro none header qwen4exp arch, 48 layers, 512 experts, ~4B active, 262144 ctx, ftype IQ2_XS 2.3125 bpw
What was removed:
~/llama-local/run-qwen38-flash-next-abl.sh ~/llama-local/models/qwen38-flash-next/ (whole tree incl .cache/huggingface staging)
Files and hashes (sha256 taken immediately before deletion):
~/llama-local/models/qwen38-flash-next/.cache/huggingface/trees/411f1f7349c9711e6f75aa8d20e1612eb3a4d55d.json 13,973 B (0.00 GiB) ~/llama-local/models/qwen38-flash-next/swift-IQ2_XS/Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf 39,788,473,344 B (37.06 GiB) sha256 514a369bff816ed78f23e0054688b570eae538299ccae45173908b5a69e468c9 ~/llama-local/models/qwen38-flash-next/swift-IQ2_XS/Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf 28,363,693,824 B (26.42 GiB) sha256 9ffb75cdd312d669bc783b4d861d587f9553ab678fbc6963765ffde6b2728234
Restore:
# publisher repo (huihui-ai) is named in the script header, but the published file basename # lacks the "Swift-" prefix this copy carries, so restore by HASH not by name: # Swift-...-00001 68d... see the sha256 lines above; llama.cpp fork build b3-1a33095 needed # (qwen4exp arch), -c 16384 on purpose: 63.5 GiB + driver overflows a 100 GiB pool.
Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated IQ3_S1 — retired 2026-10-30 [T1, 11.3 GiB]
Why it went: Dense 27B (fails the standing shape rule) with no capability row at all: the only run is a perf probe. Measured 16.3 t/s, which is the bandwidth law for a dense 27B on this box and not a tuning problem. The script header itself says IQ3_S is aggressive for a coder, so it was never a fair test of the model — only of this file.
Measured:
perf tg 16.3 t/s (20261027-012115-coder390, :8183) HumanEval none · MMLU-Pro none · riddles 14/14 header dense 65 layers, 24 heads, 4 kv heads, head_dim 256
What was removed:
~/llama-local/run-coder390-abl.sh ~/llama-local/models/coder390/ (whole tree incl .cache/huggingface staging)
Files and hashes (sha256 taken immediately before deletion):
~/llama-local/models/coder390/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-IQ3_S1.gguf 12,143,609,632 B (11.31 GiB) sha256 8c5d471ff4b5e26d1fc6c53fffef8fa9f09b441b0fd4425baec42ff9567b0df5
Restore:
cd ~/llama-local/models && wget -c \ https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-GGUF/resolve/main/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-IQ3_S1.gguf # expected 12,143,609,632 B, sha256 8c5d471ff4b5e26d1fc6c53fffef8fa9f09b441b0fd4425baec42ff9567b0df5 (= repo LFS oid, verified) # if retried, use a higher quant: the repo publishes IQ4_XS and up, and only then is a HE number about the model.
NVIDIA-Nemotron-3.5-Bolt-E4B UD-Q8_K_XL — retired 2026-10-30 [T1, 8.9 GiB]
Why it went: Never measured: no run ever attached to its port (8187 was reused by other scratch models), so there is no perf and no capability row. Its unit was disabled and it is not referenced by any script that survives.
Measured:
no rows at all in ~/runs (120 run files checked by port and by id) header dense-ish 42 layers (Mamba hybrid), 8 heads, head_dim 512, ftype UD-Q8_K_XL
What was removed:
~/llama-local/run-nemotron-bolt-e4b.sh ~/llama-local/models/dflash-NVIDIA-Nemotron-3.5-Bolt-E4B-Q8_0.gguf (its drafter) ~/llama-local/models/mmproj-NVIDIA-Nemotron-3.5-Bolt-E4B-F16.gguf (its projector) ~/.config/systemd/user/llama-nemotron-bolt-e4b.service
Files and hashes (sha256 taken immediately before deletion):
~/llama-local/models/NVIDIA-Nemotron-3.5-Bolt-E4B-UD-Q8_K_XL.gguf 8,192,955,584 B (7.63 GiB) sha256 6176bd788769bd58042c165d3dc8e8c86a8ca5c90d5de188a364b31dc9560810 ~/llama-local/models/dflash-NVIDIA-Nemotron-3.5-Bolt-E4B-Q8_0.gguf 398,892,256 B (0.37 GiB) sha256 9a63662ae8395b8bfc1a1b00d7a9ef354120f172704bd6ba88d3870cf22d4d03 ~/llama-local/models/mmproj-NVIDIA-Nemotron-3.5-Bolt-E4B-F16.gguf 990,373,568 B (0.92 GiB) sha256 6bc77b7ac2b62cf61bc86757605e45e39f6345373f0a6b76cc5b1ed97f2e233e
Restore:
# repo nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF, UD-Q8_K_XL + drafter + mmproj; # verify against the sha256 lines above. Arch needs the fork build; -ngl 99, dflash spec-type.
gemma-4-26B-A4B-it Q8_0 — RETIRED, but its row was genuine — retired 2026-10-30 [T2, 26.1 GiB]
Why it went: Duplicate quant of a family that already has two better-value files on disk. Re-measured 2026-10-30 specifically to settle a suspicion that its Oct 17 baseline row had borrowed the driver numbers: it had not — it reproduces item-for-item — so this is a SIZE/SPEED decision, not a distrust decision. Q8 157/164 + 104/120 @2048 at 49.4 t/s and 26.1 GiB versus the uncen Q4_K_M 157/164 + 102/120 at 56.9 t/s and 15.6 GiB, i.e. +10.5 GiB and -8 t/s for 2 items (1.7 pp against a 3.3 pp SE = noise).
Measured:
20261030-045147-gemma-q8-remeasure HE 0.9573 (157/164) MMLU 0.8667 (104/120) both @2048 tg 49.4 riddles 14/14 20261017-140527-gemma-q8-baseline HE 0.9573 (157/164) MMLU 0.8500 (102/120) both @2048 tg 41.6 header gemma4 arch, MoE 30 layers, 16 heads, head_dim 512, 25,233,142,046 params
What was removed:
~/llama-local/run-gemma4-26b-a4b-q8.sh ~/llama-local/models/gemma-4-26B-A4B-it-Q8_0.gguf (+ its -sha256.txt) ~/llama-local/models/mmproj-gemma4-26B-A4B-F16.gguf (+ its -sha256.txt) # the MTP drafter mtp-gemma-4-26B-A4B-it-Q8_0.gguf STAYS — the QAT driver loads it.
Files and hashes (sha256 taken immediately before deletion):
~/llama-local/models/gemma-4-26B-A4B-it-Q8_0.gguf 26,859,861,728 B (25.02 GiB) sha256 5f7cbd0f4564e84342fc34321a09acb54b1a3da9215124e5bf444baa6dda152c ~/llama-local/models/mmproj-gemma4-26B-A4B-F16.gguf 1,193,058,784 B (1.11 GiB) sha256 418a6d8723067cd712235facbbc5cba6c8fbbd413fc1292d2aace5a027d5a42f
Restore:
cd ~/llama-local/models && wget -c \ https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/resolve/main/gemma-4-26B-A4B-it-Q8_0.gguf # expected 26,859,861,728 B, sha256 5f7cbd0f4564e84342fc34321a09acb54b1a3da9215124e5bf444baa6dda152c (= repo LFS oid, verified) # projector from the same repo (mmproj-F16.gguf); serve with -c 131072, q8_0 KV, --reasoning off.
Ling-3.0-flash Q4_K_M (inclusionAI) — retired 2026-10-30 [T3, 71.7 GiB]
Why it went: 71.7 GiB — the second-largest artefact on the box — for a single measured axis. MMLU-Pro 0.7667 is respectable; HumanEval was never run, so the code axis is unknown, and it takes the whole 100 GiB pool to itself (77 GB weights + KV), meaning serving it silences the driver and the WhatsApp channel.
Measured:
20261022-231115-ling3-flash-core MMLU 0.7667 (92/120) @2048 HumanEval none tg 43.5 t/s riddles 11/14 header bailingmoe3 arch, 43 layers, 512 experts / 8 per token, 262144 ctx, MIT
What was removed:
~/llama-local/run-ling3-flash.sh ~/llama-local/models/Ling-3.0-flash-Q4_K_M/ (2 shards)
Files and hashes (sha256 taken immediately before deletion):
~/llama-local/models/Ling-3.0-flash-Q4_K_M/Ling-3.0-flash-Q4_K_M-00001-of-00002.gguf 49,995,534,208 B (46.56 GiB) sha256 24ff06f1ab74546a91f7c07bd725b3015896bbe9dd82ab0e5838be9ed06a6dcf ~/llama-local/models/Ling-3.0-flash-Q4_K_M/Ling-3.0-flash-Q4_K_M-00002-of-00002.gguf 27,014,610,464 B (25.16 GiB) sha256 1a6a8c91562b817fd8ec5b250453be696b0fc209ed8f2acdb6c13f4e17f7e188
Restore:
# publisher: inclusionAI/Ling-3.0-flash. The exact GGUF repo was never recorded in the script # header and that org does not expose a -GGUF repo with these shard names, so restore by hash: # shard 1 24ff06f1ab74546a91f7c07bd725b3015896bbe9dd82ab0e5838be9ed06a6dcf # shard 2 1a6a8c91562b817fd8ec5b250453be696b0fc209ed8f2acdb6c13f4e17f7e188 # arch bailingmoe3 is in the fork tree; -c 131072 needs the pool to itself.
NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB IQ2_XXS — retired 2026-10-30 [T3, 23.1 GiB]
Why it went: The one genuinely split result on the box: HumanEval 0.945 (155/164) but MMLU-Pro 0.367 (44/120) @2048 — it codes and does not know. At 2.0625 bpw the quant is also the most damaged artifact here, and the knowledge axis is the one that cannot be recovered by a bigger cap (the 4096 arms do not exist for this file).
Measured:
20261016-095244-placement HE 0.9451 (155/164) MMLU 0.3667 (44/120) both @2048 tg 72.8 riddles 13/14 20261016-092339 tg 74.3 (perf only) 20261030-034330-bolt74b-core tg 69.4, riddles 14/14 header nemotron_h MoE 88 layers, 32 heads, head_dim 128, ftype IQ2_XXS 2.0625 bpw
What was removed:
~/llama-local/run-nemotron-bolt.sh, run-nemotron-bolt-74b-fork.sh, run-nemotron-bolt-74b-think.sh, fetch-nemotron-bolt.sh ~/llama-local/models/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf (+ -sha256.txt) ~/llama-local/models/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr.gguf (its dflash drafter, + -sha256.txt) ~/llama-local/models/mmproj-NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf (+ -sha256.txt) ~/.config/systemd/user/llama-nemotron-bolt-74b.service
Files and hashes (sha256 taken immediately before deletion):
~/llama-local/models/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf 22,018,326,528 B (20.51 GiB) sha256 91d63363e782c8a3d5f4be42816eba8e0d42dd2530bd07a7d0d2dc440599af75 ~/llama-local/models/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr.gguf 998,242,560 B (0.93 GiB) sha256 080e1ea41c9562a7804dbf30c6fbd4d17e216c6b8ff666f88e6edb1816010194 ~/llama-local/models/mmproj-NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf 1,733,692,736 B (1.61 GiB) sha256 0ee891eadc7bf0cae6289b249c74eb1ef420534351f6ef8b64d49f0ca6a926b0
Restore:
cd ~/llama-local/models && wget -c \ https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF/resolve/main/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf # expected 22,018,326,528 B, sha256 91d63363e782c8a3d5f4be42816eba8e0d42dd2530bd07a7d0d2dc440599af75 (= repo LFS oid, verified) # drafter: nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr-GGUF, mmproj from the main repo. # if re-fetched, take a HIGHER quant: the MMLU 0.367 is the 2-bit floor, not the model.
Flint-40B-A2.5B-abliterated i1-Q4_K_M (mradermacher) — retired 2026-10-30 [T1, 22.5 GiB]
Why it went: No capability row attributable to THIS file. Its provenance is the abliterated derivative of dima6312/Flint-40B-A2.5B-abliterated, re-quantised by mradermacher — the 123/164 + 62/120 numbers that appear under the run alias "flint" belong to the earlier, differently-named FLINT artefact (RecursiveSI publisher UD-Q4_K_XL), which was itself retired on 2026-10-24. All this file has is a 60-item HumanEval probe at cap 4096 (55/60 = 0.9167) and no MMLU at all, so nothing about its knowledge axis is known and 22.5 GiB buys an unmeasured duplicate of a retired family.
Measured:
20261025-070907-flint-q4-he60 HE 55/60 (0.9167) @4096 — a 60-item probe, not a full axis; MMLU none 20261027-165746-flint40b-abl, 20261027-170747-flint40b-abl-8k perf only, tg 70.8-71.1 t/s, riddles 10/14 header flint arch, MoE 49 layers, 256 experts / 2 shared, 4 kv heads, 262144 ctx, sliding window NOT QUOTABLE: the 123/164 @2048 and 62/120 @2048 under run alias "flint" (20261021-210703) are the other file.
What was removed:
~/llama-local/run-flint40b-abl.sh ~/llama-local/models/Flint-40B-A2.5B-abliterated.i1-Q4_K_M.gguf
Files and hashes (sha256 taken immediately before deletion):
~/llama-local/models/Flint-40B-A2.5B-abliterated.i1-Q4_K_M.gguf 24,166,842,912 B (22.51 GiB) sha256 a7f6f33bbef675e3f243123d22af35667d6063515fd2c71bb122083d73395178
Restore:
# dima6312/Flint-40B-A2.5B-abliterated-GGUF (publisher quant, 24,166,842,272 B, # sha256 5ac7c931803a9fff117c5cde1cda6b7bd840f5678b28a31cfe77870e9c201cf7) or # mradermacher/Flint-40B-A2.5B-abliterated-GGUF (imatrix re-quant, 24,166,842,656 B, # sha256 25ab4aee8a3fe5e55de4cec8f6a33c58263bcb876e9121ab0c57aa725403238f). # The retired local file was 24,166,842,912 B with its own hash (see above) — near but not equal # to either, so restore by hash or accept a different file. # arch flint is NOT in the halo-box fork: it needs ~/llama-local/flint-llama.cpp (upstream master, PR #30928).
gemma-4-31B-it QAT Q4_0 (DENSE) — retired 2026-10-30 [T3, 18.0 GiB]
Why it went: Best HumanEval on the box (158/164 = 0.9634) and equal-best MMLU-Pro, and still the right thing to drop: it is DENSE, which is out of scope by the standing rule, and 17.0 t/s makes it the slowest useful model here — a suite run on it costs ~2.5x the wall clock of any MoE at the same score. Losing the top HE row is also the honest outcome: the 26B-A4B Q8 posts 157/164 at 49 t/s and the uncen Q4_K_M 157/164 at 57 t/s, i.e. one item for 3x the decode speed.
Measured:
20261021-115453-gemma4-31b-full HE 0.9634 (158/164) MMLU 0.8667 (104/120) both @2048 tg 17.0 riddles 14/14 20261021-115218-gemma4-31b tg 17.2, riddles 14/14 header gemma4 arch, DENSE 60 layers, 32 heads, head_dim 512 (global), vocab 262144
What was removed:
~/llama-local/run-gemma4-31b.sh ~/llama-local/models/gemma-4-31B_q4_0-it.gguf ~/llama-local/models/mtp-gemma-4-31B-it-Q8_0.gguf (its MTP drafter) ~/llama-local/models/mmproj-gemma4-31B-QAT.gguf (its projector) ~/.config/systemd/user/llama-gemma4-31b.service
Files and hashes (sha256 taken immediately before deletion):
~/llama-local/models/gemma-4-31B_q4_0-it.gguf 17,651,001,568 B (16.44 GiB) sha256 179cfb99212709597eae5929112cfca677e1bbf566178b479ae1da0c4772874b ~/llama-local/models/mtp-gemma-4-31B-it-Q8_0.gguf 514,705,920 B (0.48 GiB) sha256 e6e88dea4fcf79a0a71cd7732e61632355126dbbda9b72a08d6301fa06fbaf22 ~/llama-local/models/mmproj-gemma4-31B-QAT.gguf 1,200,726,368 B (1.12 GiB) sha256 6bd60bdb958548b4093196d38744b0f2290c12503a3fddd7486bffa9c5eb07a4
Restore:
cd ~/llama-local/models && wget -c \ https://huggingface.co/google/gemma-4-31B-it-qat-q4_0-gguf/resolve/main/gemma-4-31B_q4_0-it.gguf # expected 17,651,001,568 B, sha256 179cfb99212709597eae5929112cfca677e1bbf566178b479ae1da0c4772874b # (unsloth mirror: unsloth/gemma-4-31B-it-qat-GGUF). KV is cheap: 50 of 60 layers are sliding-window 1024.
HELD — not deleted in this pass
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B UD-Q3_K_XL (62.9 GiB) — I proposed it on HE 137/164 and MMLU 80/120, which are the @2048 numbers. The cap-4096 arms exist and are much better: 20261018-063326 HE 156/164 + MMLU 91/120 @4096, 20261019-213244 HE 153/164 + MMLU 92/120 @4096, tg 27.1. So it is mid-field, not weak — the case for deleting it is footprint only, and that needs a deliberate call.
LFM2.5-24B-A2B QAD-Q4_0 (13.3 GiB) — same error: I quoted the @2048 pair (HE 73/164, MMLU 65/120) when 20261017-002528-lfm25-cap4096 reads HE 140/164 (0.8537) + MMLU 83/120 (0.6917) @4096 at 106.6 t/s. At that cap it is a strong small model, not a weak one.
Both are still on disk. Their hashes are in retire-hashes.json in the cache if a later pass needs them.
Lesson recorded for the next pruning pass: a score quoted without its cap understated these two by 12-19 pp; the run directory holds the 4096 arms, and cap-arm.py re-runs only the cap-pinned rows.
Mistral-Small-4-119B-2603 (AtomicChat AD-IQ3_XXS) — fetched, measured, retired 2026-10-30
The one fit candidate of the 2026-10-30 sweep: 119B total, 6.5B active, 36 layers, 128 routed experts + 1 shared (top-4), MLA attention (kv_lora_rank 256 / qk_rope 64) so KV is 1.4 GiB at 128k, native 1M ctx, base mistralai/Mistral-Small-4-119B-2603, apache-2.0.
It loaded and served; it was the decode path that killed it. arch mistral4 is registered in halo-box-strix-llama.cpp AND implemented — llama_model_mistral4 : llama_model_deepseek2 reuses the DeepSeek2 MLA graph and tensor loader, and every key that loader demands was present (q_lora_rank 1024, kv_lora_rank 256, expert_shared_count 1, expert_ff 2048, dense-lead 0; 36 layers misses deepseek2's is_lite test, so it takes the full path). Served on :8086 with the driver still up at 79.9 GiB of the 100 GiB pool — it genuinely fits beside the channel.
Measured:
perf (fork, b3-1a33095) pp 424.8 t/s tg 4.6 t/s ttft 5.5 s perf (upstream-master-bolt) pp 296.7 t/s tg 4.7-5.0 t/s — same on both trees -ot "exps=Vulkan0" tg 4.8 t/s no change: the experts were not on CPU for placement reasons cores while decoding ~10-12 of 16 busy, GPU 80% busy → host-side MoE matmuls, the Mellum signature preflight 3/3 PASS · needle 2k PASS · needle 8k PASS MMLU-Pro, 30-item probe 0.4000 (12/30) @512, every answer 2 tokens (bare letter, no deliberation)
Why it went: 4.8 t/s is a 6x miss against the 10B-active anchor class (~22 t/s), so a full suite would have cost 6-8 hours and a chat channel would be unusable. The arch needs a Vulkan MUL_MAT_ID path for this MLA+MoE graph (or a different implementation of mistral4); the quant is not the limiter — IQ3_S on qwen4exp runs at 52.9 t/s on the same build. Revisit only if upstream adds that path.
What was removed:
~/llama-local/models/Mistral-Small-4-119B-2603/ (whole tree, 3 files, 47.03 GiB) ~/llama-local/run-mistral4-119b.sh, run-mistral4-119b-bolt.sh, run-mistral4-119b-ot.sh ~/llama-local/fetch-mistral4.py, mistral4.log, mistral4-bolt.log, mistral4-ot.log
Kept: the run rows — ~/runs/20261030-213551-mistral4-119b-core.jsonl (identity + perf + preflight + needle rows; the run was stopped before the riddles axis, so no riddles or capability row exists), ~/runs/mistral4-core.out (the same run's stdout) and ~/bench/runs/mmlupro-20261030-215224.jsonl (the 30 knowledge items).
Files and hashes (verified against the repo's LFS oids before deletion — all three MATCHED):
Mistral-Small-4-119B-2603-AD-IQ3_XXS-00001-of-00002.gguf 44,675,271,232 B (41.61 GiB) sha256 eecc0ba10b0625cba7b84c174423c83127f8eeef55393ece734cdf02595aba93 Mistral-Small-4-119B-2603-AD-IQ3_XXS-00002-of-00002.gguf 4,966,467,968 B ( 4.63 GiB) sha256 c32b5c36dce7efd5c5cc6fdab4c6faef1dafa4ec94aeb0bd1415d9cb6bc0e62a mmproj-Mistral-Small-4-119B-2603-F16.gguf 857,078,016 B ( 0.80 GiB) sha256 f6fe6adda938b42073e256a582012d179f8c6b61361d5ca1f395b0fb6087bb53
Restore:
mkdir -p ~/llama-local/models/Mistral-Small-4-119B-2603 && cd $_ for f in "AD-IQ3_XXS/Mistral-Small-4-119B-2603-AD-IQ3_XXS-00001-of-00002.gguf" \ "AD-IQ3_XXS/Mistral-Small-4-119B-2603-AD-IQ3_XXS-00002-of-00002.gguf" \ "mmproj-Mistral-Small-4-119B-2603-F16.gguf"; do curl -sL -C - -O "https://huggingface.co/AtomicChat/Mistral-Small-4-119B-2603-GGUF/resolve/main/$f" done # flags that worked: -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 # --reasoning off -np 1 -b 4096 -ub 2048 --metrics (mtmd has PIXTRAL but no MISTRAL projector # type, so vision was never probed; add --mmproj only as a separate test) # If the IQ4_XS is ever tried: 64.5 GiB, needs the driver stopped, KL 0.0609 vs 0.1279 for this file.
SandLogic Hy-Memory-30B-A3B Q4_K_M — retired 2026-10-29 (record backfilled)
Deleted in the 2026-10-29 pass (61.6 GiB batch: Kimi-Linear-48B-A3B, Hy-Memory-30B-A3B, Mellum2.1-12B Q6_K and its MXFP4 twin). This section was written afterwards: that pass predates the rule that every deletion is recorded before the files go, so no sha256 was captured and the byte figure is the 17.0 GiB from the batch summary, not a measured stat.
Measured — best run 20261017-110734-hy-memory-placement:
HumanEval 0.7805 (128/164 @ 2048) MMLU-Pro 0.7250 ( 87/120 @ 2048) riddles 12/14 perf pp 1514.6 t/s tg 74.7 t/s
Why it went: 0.78 code is ordinary, and its one strength — 74.7 t/s — does not distinguish it. gptoss-abliterated scores 0.9268 on code at 50 t/s for 59 GiB, and the 15.6 GiB gemma driver beats it on both axes at 57 t/s. Nothing it did justified 17 GiB of resident space on a box that has to choose.
Restore: upstream is SandLogicTechnologies/Hy-Memory-30B-A3B, Q4_K_M. There is no hash to verify against, so verify by measuring: HumanEval should land near 0.78 at cap 2048 and decode near 74–75 t/s on Vulkan. A materially different number means a different quant from the one that was tested.
SandLogic Hy-Memory-30B-A3B Q8_0 — retired (record backfilled)
The Q8 arm of the same model, measured as the quant-ladder datapoint against the Q4_K_M above (20261017-124731-hy-memory-q8):
HumanEval 0.7683 (126/164 @ 2048) ← −1.2 pp against Q4 MMLU-Pro 0.7500 ( 90/120 @ 2048) ← +2.5 pp against Q4 riddles 11/14 perf pp 1459.2 t/s tg 52.3 t/s ← 30 % slower than Q4 for the extra bits
Deletion date, size and hashes were never recorded — the file left disk before the ledger rule existed, so treat all three as unknown rather than reconstruct them. What survives is the run and the delta, which is the useful part: doubling weight precision bought 2.5 pp of knowledge and cost 30 % of decode speed, at identical caps. Expect roughly 32 GiB if it is ever restored from the same publisher.
graft-50b-a3b (50.7B Q4_K_M) — retired (deletion record incomplete)
Measured 2026-10-19, run 20261019-122149-graft-50b-a3b:
HumanEval 0.7805 (128/164 @ 2048) MMLU-Pro 0.6583 ( 79/120 @ 2048) riddles 14/14 ← one of only seven 14/14 sweeps on this box perf pp 1367.7 t/s tg 54.3 t/s
This is the least complete entry in the ledger, and it says so deliberately: no deletion date, no byte count, no sha256 and no upstream repository were recorded before the file left disk. The run and the scores above are the only surviving evidence. It cannot be restored byte-for-byte — re-obtaining it means identifying the publisher from the model name and re-measuring. If anyone does, the 14/14 riddle sweep is the number worth checking, because that is rarer on this box than a 0.78 HumanEval.
Written with Hermes Agent (Nous Research) driven by DeepSeek's deepseek-flash, from the run archive and from the sha256 values taken immediately before each deletion. Where a hash or date was never captured, the entry says so rather than filling the gap.
Apexintelligence-AI Apex-35B-V1.0 Q4_K_M (bartowski) — retired 2026-11-01
Rejected on evidence, not on size. Fetched 2026-11-01 11:47 → 12:06 (21.23 GiB at 40 MB/s), tested the same afternoon, deleted the same afternoon.
Why
The finetune does not terminate its reasoning on this build. Riddle items return completion_n equal to the cap with an empty content — the model thinks until the budget is gone and never answers:
| cap | riddles | items that burned the whole cap with no answer |
|---|---|---|
| 4096 | 9/14 | 5 of 14 (n=4096, content empty) |
| 8192 | 9/14 | 3 of 14 (n=8192, content empty) |
HumanEval tells the same story at 0.4146 (68/164) with a 2203 s wall — 13.4 s/item at 61 t/s, i.e. ~800 tokens of thinking per item and zero credit when it runs out. MMLU-Pro with thinking forced off reached 0.5800 (58/100). At the same 35B-A3B shape this roster carries qwen38-35b at 0.8720/0.6417 and Ornith-1.5-35B at 0.8354/0.4417, so the model is outside the band even before the non-termination.
Measured rows
| run | axes | result |
|---|---|---|
20261101-124818-apex35b-cap4096 | HE 164, riddles, perf, preflight, needles | HE 0.4146 (68/164) 2203 s; riddles 9/14; pp 1991.7 / tg 61.5 / ttft 902 ms; preflight 3/3; needles 2k+8k PASS |
20261101-133139-apex35b-cap8192-probe | core at cap 8192 | riddles 9/14; pp 2005.5 / tg 60.1 / ttft 896 ms; preflight 3/3; needles PASS |
Removed
| path | bytes | sha256 |
|---|---|---|
models/Apexintelligence-AI_Apex-35B-V1.0-Q4_K_M.gguf | 22,795,471,072 | c1bf4a6008bda54c6da3be1e5b6137a9b31cbe7d5a5432950215b0d13da2296b |
run-apex-35b.sh, run-apex-35b-bench.sh, apex-bench.log | small | — |
Restore
hf / bartowski/Apexintelligence-AI_Apex-35B-V1.0-GGUF → Apexintelligence-AI_Apex-35B-V1.0-Q4_K_M.gguf, expect 22,795,471,072 bytes and the sha256 above. Re-download only to study the template — the harness (t20) and a --jinja server were not the problem, and a reasoning-effort cap did not fix it either.
Dark-Craving-26B-A4B v0.3 (ReadyArt) — retired 2026-11-01, dominated by the driver
Fetched and measured the same afternoon; it is a good model that loses to a smaller, faster one on every axis. gemma4 arch, 26B total / 4B active — the driver's exact shape, so the comparison is finetune-only.
| axis | Dark-Craving v0.3 Q4_K_M | gemma4-26b-uncen Q4_K_M (driver) |
|---|---|---|
| HumanEval @4096 | 0.9390 (154/164) | 0.9573 (157/164) |
| MMLU-Pro @4096 | 0.8167 (98/120) | 0.8500 (102/120) |
| riddles | 14/14 | 14/14 |
| decode | 47.3 t/s | 56.9 t/s |
| size | 16.45 GiB | 15.64 GiB |
Run 20261101-165147-darkcraving-26b-a4b-cap4096, 2399 s, preflight 3/3, needles 2k+8k PASS, pp 1755.8. The capability gaps (3 and 4 items) sit inside the sampling error at n=164/120, so this is a draw on quality; the 17% decode deficit is not noise. It answers riddles crisply and its knowledge is strong for the shape — it simply has nothing the driver does not do better, in less space.
| path | bytes | sha256 |
|---|---|---|
models/Dark-Craving-26B-A4B-v0.3-HB16-Q4_K_M.gguf | 17,666,860,256 | 4ef7c7f0c343267861beb29a0c10d59f390bef4ef1993bdfbc0ceab566f526af |
Restore: ReadyArt/Dark-Craving-26B-A4B-v0.3-GGUF, file above, expect the byte count and hash. Script run-darkcraving-26b.sh (port 8086, gemma4 flags) is in the transcript and trivially rewritable.
Holo4-35B-A3B (Hcompany) MXFP4_MOE — retired 2026-11-01, no knowledge retrieval
18.43 GiB for 35B total / 3B active, qwen3_5_moe base, MXFP4 — capacity per byte is the best of any 35B here. Code holds up; knowledge does not.
| axis | Holo4 MXFP4 | qwen38-35b Q4_K_M | Ornith-1.5-35B Q4_K_M |
|---|---|---|---|
| HumanEval @4096 | 0.8598 (141/164) | 0.8720 (157→143/164) | 0.8354 (137/164) |
| MMLU-Pro @4096 | 0.3250 (39/120) | 0.6417 (77/120) | 0.4417 (53/120) |
| riddles | 13/14 | 14/14 | 13/14 |
| decode | 60.5 t/s | 74.8 t/s | 74.1 t/s |
| size | 18.43 GiB | 20.22 GiB | 20.22 GiB |
Run 20261101-173210-holo4-35b-a3b-cap4096, 382 s, preflight 3/3, needles PASS, pp 2062.9.
The MMLU number is the finding, not a bug: all 120 items came back as a bare letter in 0.2 s, one unextractable. At 10 options the chance floor is 25%, so 32.5% is 1.6 SE above guessing — this finetune does not retrieve knowledge when asked for a single token, and it is not a thinking model that would recover with a larger cap. HumanEval and riddles force generation, and there it is competent, which is exactly why the collapse only shows on the knowledge axis.
| path | bytes | sha256 |
|---|---|---|
models/Holo4-35B-A3B.MXFP4_MOE.gguf | 19,790,379,456 | 8e9a845fb6bcec527626c9a1e5c3929314c311af0f49dee450e83e821dc2a252 |
Restore: prithivMLmods/Holo4-35B-A3B-MXFP4-MoE-GGUF, file above, byte count and hash as recorded.
Qwen3.8-Flash-Next-32B-A6B-Prune2-QMix (WeeAIResearch) — re-fetched, re-tested and retired 2026-11-03 (loads; unusable)
Superseded — the file loads and was re-tested on 2026-11-03; read the correction at the end of this section before acting on anything below.
Fetched the same evening it was spotted, then deleted without a measurement, because it cannot load. The "Prune2" export kept the pre-prune metadata: qwen4exp.expert_count says 177, the expert tensors hold 64, and the router (ffn_gate_inp) still has 177 rows. No single expert_count satisfies both checks, so neither tree will load it and a header patch only moves the error one tensor along:
check_tensor_dims: tensor 'blk.0.ffn_down_exps.weight' expected 640, 2560, 177, got 640, 2560, 64, 1 (after patching expert_count to 64) check_tensor_dims: tensor 'blk.0.ffn_gate_inp.weight' expected 2560, 64, got 2560, 177, 1, 1
The file was restored to its upstream bytes after the patch experiment (sha256 verified back to the LFS oid) before deletion, so the record below is the pristine artefact.
| path | bytes | sha256 |
|---|---|---|
models/Qwen3.8-Flash-Next-32B-A6B-Prune2-Mixed.gguf | 51,869,664,064 | faea180d49c5494b18443a9db0f4f4be68b72b22b4f9e5bbda7f470f831ea0c0 |
Restore: WeeAIResearch/Qwen3.8-Flash-Next-32B-A6B-Coder-Mixed, file ..Prune2-Mixed.gguf, byte count and hash as above. Only worth re-fetching if a re-export appears with matching router and expert counts — the shape itself (32B total / 6B active in 48 GiB, qwen4exp) is the one gap this roster has never covered.
Correction + re-test, 2026-11-03 — the verdict above was WRONG
The file was re-fetched (same sha256) and inspected properly: it carries THREE expert banks per layer (ffn_*_exps tier0 Q4_K/Q5_0, _tier1 Q2_K, _tier2 Q1_0) and those bank dims sum to the router width — blk.0 64+60+53 = 177, blk.1 64+60+30 = 154, blk.24 64+41+10 = 115. expert_count 177 was never inconsistent; reading tier 0 alone made it look so, and a header patch was never the fix.
It LOADS on the publisher's own runtime: upstream tag b10662 (18443257) + runtime/variable-runtime.patch from the same repo (97 added lines across llama-arch.cpp/.h, llama-model.h, models/qwen4exp.cpp), Vulkan build, 12 s load, GTT 29.9 -> 46.6 GiB, 32k ctx.
What it then does is worse than not loading:
- tg 20.0 t/s / pp 396.8 / ttft 4534 ms at 32k (the same-shape Qwen3.8-35B-A3B Distill runs 74.8 / 1550).
- Vulkan: repetition loops on anything non-trivial — "17 23 = 17 23 = ...", a wall of "/" on factual questions, sometimes empty content. Reproduced with f16 KV and flash-attn off, so it is the tier graph on the GPU backend, not the KV quant.
- CPU (
-ngl 0): fluent and terminating, instruction-following intact (2+2=4, 5*6=30, "TEST"), but knowledge destroyed — capital of Australia "No", capital of France "No", largest planet "Earth", 17*23 "37". The pruning (3,659 layer/expert slots) plus a 1-bit tier bank cost the facts.
Disposition: deleted 2026-11-03. Do not re-fetch until a re-export drops the Q1_0 tier (or the publisher fixes the Vulkan path) — the shape (32B total / ~6B active in 48 GiB) is still the one gap this roster has never covered, so a clean export of this shape would be worth another look.
Run rows kept: ~/runs/20261103-190055-qwen38fn32-prune2.jsonl (identity, perf, preflight, 2x needle, plus the backend-split probe note), ~/llama-local/t20-qwen38fn32-core.log.
NVIDIA-Nemotron-3.5-Lightning-30B-A3B (ggml-org Q4_0) — measured 2026-11-01, deleted: parity quality, dominated on speed and termination
The coverable coverage gap from verified-not-runnable.md ("released 2026-08-11, never on this box"), finally measured. Arch nemotron_h_moe, 128 routed experts / top-6 + 1 shared, 52 layers hybrid mamba/moe/attention, 31.58B params, header context_length 1048576 but config max_position_embeddings 262144 (serve 262k as the trained window). 18.90 GB weights + 1.16 GB MTP drafter.
The finding is that its two arms are mutually exclusive. Thinking off (template default overridden with --chat-template-kwargs '{"enable_thinking":false}') is the only arm that finishes code; thinking on is the only arm that retrieves knowledge — and it does not terminate often enough to be served for code at all.
| axis | thinking off | thinking on |
|---|---|---|
| HumanEval 164 @2048 | 0.8598 (141/164), wall 629 s, 3.8 s/item, ZERO items at the cap | killed at 9 items: 5 pinned at 2048 with NameError: name '…' is not defined, 33 s/item (~90 min projected) |
| MMLU-Pro @2048 | 0.4667 (56/120), 0.6 s/item, every answer 2 tokens | 0.6667 (20/30 probe), committed 20/24 = 0.8333, 6/30 pinned at 2048, mean 19.0 s/item |
| riddles | 7/14 @512 (7 pinned, empty content) | 11/14 @2048, and the same 3 items still pin at 4096 with 13,277 / 14,834 / 15,793 chars of reasoning and empty content |
Perf (plain decode, no drafter, 65k ctx, driver resident): pp 2200.4 t/s, tg 63.8 t/s, ttft 820 ms. With the MTP drafter (--spec-type draft-mtp --spec-draft-n-max 1 --spec-draft-n-min 0 --spec-draft-p-min 0.75 -md mtp-…Q4_0.gguf): tg 70.4-72.3 t/s, acceptance 261/292 = 89.4% — the head pays, but +11% does not change the verdict.
Protocol: preflight 3/3, needles 2k + 8k PASS (finish=stop), single-shot tool gate PASS (finish_reason: tool_calls, correct {"city":"Bern"}, 45 tokens), agentic probe 2/3 (advances the loop, then reads src/pay.py instead of the file holding the 110 assertion).
Verdict: dominated by the resident qwen38-35b — HE 0.8598 vs 0.8720 (2 items, parity against a 1.7 pp SE), MMLU 0.6667 vs 0.6417 (parity), but 63.8 / 71.5 t/s vs 74.8 plain, riddles 11/14 vs 14/14 with three items that never terminate at any cap, and 2/3 on the agentic loop. It is 1.3 GB smaller, which is not a reason to carry a second 30B-A3B that needs a per-axis config switch to be usable.
Run rows kept: ~/runs/20261101-212604-nemotron-lightning-30b.jsonl (cap 512, riddles 7/14), ~/runs/20261101-212857-nemotron-lightning-30b-cap2048.jsonl (riddles 11/14), ~/bench/runs/humaneval-20261101-215421.jsonl (141/164), ~/bench/runs/mmlupro-20261101-215628.jsonl (56/120), ~/bench/runs/mmlupro-20261101-220000-nemotron-lightning-think30.json (20/30, thinking on).
| path | bytes | sha256 |
|---|---|---|
models/Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf | 18,898,091,584 | 61f87e75974e4b535dcdf9aad056541a9514f1dfa4538b463b081d19b7a00e3c |
models/mtp-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf | 1,155,907,520 | 19f964207d5236dc88662686f00604a5494974c23fb04dd16a5ad7b2eebbd5b4 |
Restore: ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF, files as above, byte counts and hashes as recorded (both were sha-verified against the repo's LFS oids on fetch). Worth re-fetching only for a tool-use / agentic comparison, where its 2/3 is the one number not already covered by a resident model.
Qwopus3.6-35B-A3B-Uncensored-Genesis (APEX-Mini) — measured 2026-11-02, deleted: knowledge intact, code formatting broken
Found by sweeping the moe tag instead of filter=gguf — the first candidate in weeks that was in class, runnable and not a duplicate: an uncensored 3B-active coder, a role the roster had empty. Header verified before download (qwen35moe, 256 experts top-8, 41 blocks, 262144 ctx, nextn_predict_layers=1, 753 tensors, Q4_K_M), base "Qwen3.6 35B A3B Uncensored HauhauCS Aggressive".
| axis | value | comparator |
|---|---|---|
| HumanEval @2048, thinking off | 0.7012 (115/164), 279 s, 1.7 s/item, 0 items at cap | ornith-mtp-21g 0.9207, minicpm-v47 0.9573, qwen38-35b 0.8720 |
| MMLU-Pro @2048, thinking off | 0.7667 (92/120), 987 s, 8.2 s/item, 14 at cap | ornith-mtp-21g 0.7750, minicpm-v47 0.7750, qwen38-35b 0.6417 |
| perf (MTP drafting) | pp 1851.0 / tg 86.5 / ttft 971 ms | fastest 35B-A3B on the box (qwen38-35b 74.8) |
| riddles @2048 | 10/14, 3 pinned empty at cap, 1 real wrong answer | ornith-mtp-21g 14/14 |
| core | preflight 3/3, needle 2k PASS, needle 8k PASS | — |
The finding is a code-formatting defect that no cap can explain. 49 failures, ZERO at the cap, median completion 53 tokens against 98 for passes: 25 IndentationError ("unindent does not match any outer indentation level", at 49-57 tokens), 9 SyntaxError ('return' outside function, unterminated strings), 11 AssertionError, 2 NameError, 1 ValueError, 1 IndexError. Control on the identical harness, cap and thinking-off arm: the Nemotron Lightning run hours earlier produced ZERO IndentationErrors across 164 items with every failure a plain AssertionError, so this is the weights, not the rig. Read the failure CLASS, not just the rate: a malformed-output defect shows as syntax errors at short completions, while a cap floor shows as n == max_tokens.
Disposition: dominated for every role. Knowledge is level with ornith-mtp-21g (0.9207 / 0.7750 at 65.2 t/s) and minicpm-v47-35b (0.9573 / 0.7750 at 64.6), which sit in the same 20.8-28.5 GB footprint and beat it by 22-26 HumanEval items; its only win is decode (86.5 t/s), which does not outrank a 13-26 pp quality gap. Fourth uncensored model on this box and the first with a real quality cost — gemma-4-26B, gpt-oss and Mellum were all parity.
Run rows kept: ~/runs/20261102-194814-qwopus36-35b-uncen.jsonl (core, riddles 10/14), ~/bench/runs/humaneval-* (115/164), ~/bench/runs/mmlupro-* (92/120).
| path | bytes | sha256 |
|---|---|---|
models/qwopus36-35b-uncen-apex-mini.gguf | 20,737,845,536 | 38a08a4d9c99b6931a10c0a36bd898dca6caaa73480f041e8b5ee49f03991b49 |
Restore: LuffyTheFox/Qwopus3.6-35B-A3B-Coder-Uncensored-Genesis-MTP-GGUF, file Qwopus3.6-35B-A3B-Uncensored-Genesis-Hermes-APEX-Mini.gguf, byte count and hash as recorded.
clover-35b-a3b-preview — retired 2026-11-03 (dominated, AR-only)
DiffuSpace Clover-35B-A3B-Preview, a Qwen3.6-35B-A3B diffusion finetune. New arch clover_moe (upstream PR #31306, AR mode only — self-speculative diffusion decode is a separate unmerged PR), so it needed its own upstream tree; no fork/upstream local tree carried the arch.
Measured (2026-11-03, cap 4096, zero items at cap on both heavy axes): HumanEval 0.8598 (141/164), MMLU-Pro 0.4000 (48/120), riddles 14/14 scored but 3 non-terminating at n=4096 (11/14 committed), preflight 3/3, needles 2k+8k PASS, tool gate finish_reason=tool_calls first try, agentic probe 2/3, tg 53.7 t/s / pp 1461 / ttft 1231 ms. Code parity with its base family (qwen3.6-35b 0.8354/0.45), knowledge at base-family level, and the gemma4-26B-A4B driver beats both axes at 48.9 t/s.
Restore: bartowski/Diffuspace_Clover-35B-A3B-Preview-GGUF, file Diffuspace_Clover-35B-A3B-Preview-Q4_K_M.gguf, 22,318,513,056 bytes, sha256 62f43125c5f08c559cb476aa96baf436683836340472116a64a9ea311587c4f1. Build its own tree first (no local tree registers CLOVER_MOE):
git clone --depth 1 https://github.com/ggml-org/llama.cpp ~/llama-local/clover-llama.cpp cd ~/llama-local/clover-llama.cpp && cmake -B build -DCMAKE_BUILD_TYPE=Release \ -DGGML_VULKAN=ON -DGGML_NATIVE=ON -DGGML_OPENMP=ON -DGGML_CCACHE=ON \ -DVulkan_INCLUDE_DIR=$HOME/.local/include -DVulkan_LIBRARY=/usr/lib/libvulkan.so -DLLAMA_CURL=OFF cmake --build build --target llama-server llama-cli -j 28 # ~2 min, not 10
Server flags: -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --spec-type none --reasoning off --jinja -np 1 -b 4096 -ub 2048 --metrics on port 8091. Run rows kept: ~/runs/20261103-181904-clover35b-q4km.jsonl, ~/bench/runs/humaneval-20261103-182953.jsonl, ~/bench/runs/mmlupro-20261103-183404.jsonl.
2026-11-05 pass — two orphans (8.14 GiB freed)
Tier A of the 2026-11-05 roster audit. Both files had no serving unit, no run script, no llama-switch case and no Hermes provider; each was re-hashed and matched against its repo's published LFS oid before removal, so both are restorable with confidence.
Mellum2.1-12B-A2.5B-Thinking Q4_K_M — 8,071,295,264 B (7.52 GiB)
- File
models/Mellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf, sha256ecc4d5b8107fc219e23c494b2d107552137637b6c6002ee713e0ba8af7fa2755— identical to the repo's LFS oidJetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF, pathMellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf. - Arch
mellum, 12.1B total / ~A2.5B active. The same repo also ships an MTP variant (mtp/Mellum2.1-12B-A2.5B-Thinking-MTP-Q4_K_M.gguf, 8.33 GB) — never fetched. - Why it went: orphan. No
run-*.sh, nollama-*.service, no switch case of its own, noproviders.*/model.aliases.*entry. Themellumswitch case pointed atllama-mellum.service, a unit deleted with the Q6_K file on 2026-10-29; that case and its three usage strings are pruned in this pass. - Measured and on record: HE 0.9146 (164@8192), MMLU-Pro 0.8083 (120@8192), riddles 10/14, pp 2723 / tg 111.9 t/s — run
~/runs/20261018-161122-mellum21-12b-fork-cap8192.jsonl, build fork b3-1a33095. It was the best sub-15B model on this box by HE and the fastest after lfm25; re-fetch if that slot is wanted again. - Restore:
hf download JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF Mellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf, verify the sha256 above, then serve with--jinja --spec-type none(thinking model) at the fork's b3-1a33095 or later. The recorded run used-c 8192,-ngl 99, q8_0 KV.
LFM2.5 DSpark draft head (F16) — 663,691,296 B (0.62 GiB)
- File
models/lfm25-dspark-F16.gguf, sha2560cd847ca049aeabf53cb41d62091ae20eb50245cf89594c75af204784a1b1a0f— identical to the LFS oid ofLiquidAI/LFM2.5-24B-A2B-DSpark-GGUF, pathLFM2.5-24B-A2B-DSpark-F16.gguf(local name shortened). - Why it went:
--spec-type draft-dsparkhead for LFM2.5-24B-A2B whose only consumer wasdspark-ab.sh, itself retired here. Never scored — the A/B script exists but no result was ever recorded, and no serving unit loaded it. - Restore: re-download
LFM2.5-24B-A2B-DSpark-F16.gguffrom the repo above and re-run the A/B before trusting it. The box's record on draft heads is that provenance decides: baked MTP +17 to +29%, grafted heads −8 to −11%, this one untested.
Also retired in this pass
dspark-ab.sh— the A/B driver for the head above; no other script referenced it.llama-switchcasemellumand its usage strings — it pointed atllama-mellum.service, which was removed on 2026-10-29. Case count and usage list rebuilt in the same pass.
2026-11-05 pass (second) — Tier B: Groot-65B-A6B, Nemotron-3.5-Super-VL-120B, Kimi-Linear-48B-A3B-abl
169.60 GiB across 11 files, all re-hashed in this pass and 11/11 matched their repos' published LFS oids (169.60 GiB read back off disk before deletion). Two of the three were measured on the same build as everything else that day; the third's row is from 2026-10.
Groot-65B-A6B-261023 Q4_K_M — 79.07 GiB (6 files)
- Files (all verified):
Q4_K_M-00001..00004-of-0000435,211,114,400 / 16,413,787,680 / 16,536,168,576 / 10,585,663,488 B,mtp-…-BF165,251,556,576 B,mmproj-…-BF16907,542,656 B. sha2563135f646…f62094,f7a33224…a75759,d83b7f7a…1f14,ba78ec97…630f4,2a116a0c…4ec91c,e90bd114…df6e9— each identical to the LFS oid ofGrootTeam/Groot-65B-A6B-261023-GGUF. - Arch
qwen4exp, 116,514,464,640 params (a "65B" name on 116.5B of weights), 48 blocks, 256 experts top-10, 12 QSA / 36 Gated DeltaNet layers, PLE bank, 262k ctx. - Measured 2026-11-05: HE 0.9000 (54/60 @2048), MMLU-Pro 0.7500 (30/40), riddles 9/14 (all five misses cap-pinned; at 4096 the family-relations puzzle answers correctly after 1,953 tokens), pp 596.6 / tg 27.6 / ttft 3083 ms, and the archive's cleanest drafter result: its bundled MTP head took decode 27.66 → 35.66 t/s (+28.7%, 76.3% acceptance). Runs
~/runs/20261105-014824-groot65b-q4km-think-cap2048.jsonland~/runs/20261105-013414-groot65b-q4km-nomtp.jsonl. - Footprint (the reason it was a spare, not a unit): all-resident it pins GTT 71.0 GiB and leaves ~1 GiB MemAvailable — a first attempt with
-np 4 -c 32768+ mmproj was globally OOM-killed in 7 minutes. Only the fork's--load-mode mmap --lazy-mode autoserves it comfortably (50.4 GiB, 63 GiB free, no decode cost). - Why it went: with Swift-1.5 (same shape class, same 79 GiB, measured the same day) kept, this was the second of two unhooked ~79 GiB reasoners next to the protected
qwen38fn-abl. Portfolio duplicate. - Restore:
~/llama-local/fetch-groot65b.sh(sequential curl + resume + sha256 check) rebuilds all six files; serve with~/llama-local/run-groot65b.sh mtpafter settingGROOT_TREEto the halo-box fork andGROOT_LAZY=auto. The oldrun-groot65b.shis archived inretired-scripts/.
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B UD-Q3_K_XL — 62.86 GiB (3 shards + projector)
- Files (all verified):
UD-Q3_K_XL/…-00001-of-000037,874,568 B,-0000249,225,290,592 B,-0000316,528,375,872 B, plusmmproj-Nemotron-3.5-Super-VL-120B-A12B.gguf1,733,695,840 B (repo namemmproj-BF16.gguf). sha256b2a570b4…e02fa,94827cf2…c68674,c2245a2f…cef8ce,9c09381a…924d9— identical tounsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF. - Measured: best HE 0.9512 (164@4096) — run
~/runs/20261018-063326-nemotron-super-120b-cap4096.jsonl(MMLU 0.7583 on that run); best MMLU 0.7667 (120@4096) — run~/runs/20261019-213244-nemotron-super-vl.jsonl(HE 0.9329 there). Table 4 quoted the best of each axis, as it does everywhere. pp 411 / tg 27.1 / ttft 4.5 s, riddles 11/14, preflight 3/3, needles 2k+8k PASS. - Why it went: third 120B-class file beside the two favoured gpt-oss builds (59.03 GiB each) which run ~50 t/s. +2.4 pp of HumanEval at 1.8x the time per token is not a reason to hold 62.86 GiB. It was also the only vision-capable 120B — noted as the one real loss.
- Registry removed: unit
llama-nemotron-super-120b.service(disabled), switch casenemotron-super-120b, run scriptrun-nemotron-super-120b.sh. - Restore:
unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF, theUD-Q3_K_XL/triple plusmmproj-BF16.gguf; verify the four sha256s above; serve on :8189 with the projector,-c 131072, q8_0 KV,--jinja(hybrid Mamba-2 MoE, needs a build registeringnemotron_h_moe).
Kimi-Linear-48B-A3B-Instruct-abliterated i1-Q4_K_M — 27.66 GiB
- File
Kimi-Linear-48B-A3B-Instruct-abliterated-bf16.i1-Q4_K_M.gguf, 29,702,759,968 B, sha256ad4eca4c37c688e9d03aaf4558ddb869bd7740bc02423451be6d33e9502cde80— identical to the LFS oid ofmradermacher/Kimi-Linear-48B-A3B-Instruct-abliterated-bf16. - Measured: HE 0.7866 (164@8192), MMLU-Pro 0.5833 (120@8192), riddles 13/14, pp 1774 / tg 74.8 — run
~/runs/20261101-140102-kimi-linear-abl-cap8192.jsonl. Earlier 2026-11-01 runs are the abortive probes (-abl,-abl2,-probe) that led to the 8192 cap. - Why it went: no unit; loses both capability axes to kept files, and its distinct claim (long context) is covered by Qwen3-Next's 262k and the 176.9B pair at ~55 t/s.
- Registry removed: switch case
kimi(which pointed at the long-deletedllama-kimi.service),run-kimi-linear-48b.sh,run-kimi-linear-48b-bench.sh. - Restore: re-download the single file, verify the sha256, serve with
--jinja -c 65536(its 1M ctx is measured only to 65k here) on any free port.
Registry changes made in this pass
llama-nemotron-super-120b.servicedisabled and its unit file removed;daemon-reloadrun.llama-nemotron-super-120b.servicestripped from every survivor'sConflicts=line (13 files) so no unit references a target this pass removed. Pre-existing stale peers in those same lines (llama-mellum,llama-kimi,llama-b4-27b,llama-gemma4-31b,llama-hy-memory-*,llama-ingot-genesis-27b,llama-ling3-flash,llama-nemotron-bolt-*,llama-qwen38-27b,llama-granite-switch-*) are left for a dedicated registry audit — they are inert, not dangerous.llama-switch: casesnemotron-super-120bandkimiremoved, all three usage strings rebuilt, syntax checked, 22 cases remain.- Scripts archived to
retired-scripts/:run-groot65b.sh,run-nemotron-super-120b.sh,run-kimi-linear-48b.sh,run-kimi-linear-48b-bench.sh.
Verification
- Hashes: 11/11 matched the repos' LFS oids before deletion (169.60 GiB read back).
- Models directory: 798.71 → 629.11 GiB (−169.60). Files: 47 → 36.
- Survivors after: :8094 (boot Ornith) and :8100 (gemma driver) both
/health200, bridgeconnected. - No script, switch case, unit or Hermes config entry names a deleted basename.
2026-11-06 pass — Laguna BF16 reference pair out, Q4_K_M stays
Deleted after the Q4_K_M arm proved the model is deployable at 4x the speed. Both files re-hashed in this pass and both matched their repos' published LFS oids, so both are restorable to the byte.
Laguna-XS-2.1 BF16 — 66,930,226,304 B (62.33 GiB)
- File
models/Laguna-XS-2.1/Laguna-XS-2.1-BF16.gguf, sha256d1d99abe30c37749ec1b1ae2681cec74caa550ac6243d2ea70b61b1ff9d187ca= the LFS oid ofpoolside/Laguna-XS-2.1-GGUF. - Measured 2026-11-06: pp 230.6 / tg 11.6 t/s / ttft 7941 ms, preflight 3/3, needles 2k+8k PASS, 3x greedy code generations 11.68 t/s median. No riddles or heavy axes (40 min for the riddle block alone).
- Why it went: BF16 has no fast matmul path on this box. Both local Vulkan trees compile zero coopmat2 shaders and the backend gates BF16 on
coopmat_bf16_support/coopmat2_bf16_support; the same weights at Q4_K_M run 87.7 t/s. The BF16 file was a reference copy of something we now have at usable speed for a third of the disk. - Restore:
poolside/Laguna-XS-2.1-GGUF, fileLaguna-XS-2.1-BF16.gguf; verify the sha256 above. Serve as inrun-laguna-xs.shwithLAG_FILE=Laguna-XS-2.1-BF16.gguf— expect ~11.6 t/s until a build with the BF16 coopmat path exists.
Laguna-XS-2.1 DFlash draft head — 494,735,360 B (0.46 GiB)
- File
models/Laguna-XS-2.1-dflash/Laguna-XS-2.1-DFlash-Q8_0.gguf, sha25640eb6bf48f42ebc1990a0ba8c74d2070a256e7f1842a58d8aa9b195e907b04e8= the LFS oid ofmeshllm/Laguna-XS-2.1-DFlash-GGUF. - Why it went: it never loaded.
done_getting_tensors: wrong number of tensors; expected 64, got 58->failed to load draft model-> the server exits, on build b3-1a33095. Nothing to measure, and no spec-decode arm exists for this family here. - Restore: re-download the file and re-test after a build bump; the tensor count must match the laguna loader in that tree, so a newer head or a newer build is required — not just a re-download.
Kept: Laguna-XS-2.1 Q4_K_M — 20,274,300,032 B (18.88 GiB)
pp 1320.6 / tg 87.7 t/s / ttft 1386 ms; preflight 3/3; needles 2k+8k PASS; riddles 10/14; HE 0.7000 (42/60), MMLU-Pro 0.6000 (24/40), IFEval 0.7667/0.8444 — with 35 of 308 generations pinned at the 2048 cap, so the HE/MMLU figures are floors rather than ceilings. Sha256 1ac7079101fca5a6df8c5a7523a3c30ea7d1c0e4b1258090e7d6d4039287f6cb.
2026-11-20 pass — Hy4-flash IQ2 out (79.79 GiB freed)
The 218.9B arm measured on 2026-11-19 answered its question and the answer was no, so the weights went the next day rather than sitting on disk as an exhibit. All three files were hashed BEFORE deletion and all three matched the LFS oids published by AtomicChat/Hy4-flash-GGUF, so this copy is restorable to the byte.
Hy4-flash AD IQ2_S/IQ2_XXS — 84,700,799,872 B of weights (78.88 GiB), plus a 0.91 GiB projector
models/hy4-flash-iq2/Hy4-flash-AD-IQ2_S-IQ2_XXS-00001-of-00002.gguf— 49,060,453,184 B, sha256cf32bc2cec9a7cc815faf9356500155b058926c2e3d5a8aabc87f8f11f277348models/hy4-flash-iq2/Hy4-flash-AD-IQ2_S-IQ2_XXS-00002-of-00002.gguf— 35,640,346,688 B, sha256c89e1c7767630dffcca8b2fe76611e04f86c2aaf18f1715192241ce9cea48a74models/hy4-flash-iq2/mmproj-Hy4-flash-F16.gguf— 971,849,408 B, sha256eb44b47d4d254968ae80c88655925ef544aef7524e136ee882909a44275620f3(text-only arm: the PR-tree projector rejects this file withload_hparams: unknown projector type: hy_v4_flash, so vision was never usable)- Measured 2026-11-19, run
20261119-214032-hy4flash-iq2-full, buildhy4flash-pr b1-f8d4cc4(upstream PR
#32517), caps 512 on all three capability axes: HE 0.9207 (151/164) · MMLU-Pro 0.6167 (74/120) · IFEval 0.7667/0.8444 (30) · riddles 14/14 · preflight 3/3 · needles 2k+8k, pp 276.2 / tg 18.9 t/s, 5,984 s wall.
- Why it went: no slot. Knowledge tied a 30B Q3 build and sat 0.23 under the driver; code was third; decode was 3.5x Ornith-35B's cost on the same suite. 78.88 GiB of weights needs 79 of the 100 GiB pool at 32k, so it can only ever be served by displacing the driver and the boot unit.
- The bound is the quantisation, not the architecture: 2.6 bpw. The only less-damaged builds are Q3_K (144.7 GiB) and Q4_K_M (156.8 GiB), neither of which this box can hold — so "re-download at a better quant" is not an available experiment.
What stays: run-hy4-flash.sh (port 8101, HY4_VISION gated off by default) and the hy4flash-pr-llama.cpp build tree, because PR #32517 is still open and the tree is what would serve a re-download. There is no systemd unit and no llama-switch entry for this model, and the row stays in Table 4 of the report marked ✗ 11-20.
Restore:
mkdir -p $HOME/llama-local/models/hy4-flash-iq2 && cd $HOME/llama-local/models/hy4-flash-iq2 for f in Hy4-flash-AD-IQ2_S-IQ2_XXS-00001-of-00002.gguf \ Hy4-flash-AD-IQ2_S-IQ2_XXS-00002-of-00002.gguf \ mmproj-Hy4-flash-F16.gguf; do $HOME/llama-local/fetch-hf-curl.sh AtomicChat/Hy4-flash-GGUF "$f" . || exit 1 done # verify each against the sha256 above (the repo's own LFS oids), then serve with $HOME/llama-local/run-hy4-flash.sh # the loader needs the hy4flash-pr tree: hy_v4_flash is not in the serving fork or the Bolt tree
Only re-download this for a reason: PR #32517 merging does not improve the numbers, it only makes the build ordinary. The number that would change the verdict is a sub-3 bpw artefact with a different scheme, and none existed when this was retired.
Source: github.com/strix-halo-128-benchmarks/strix-halo-128-benchmarks.github.io. Entries written after 2026-10-30 record the sha256 before deleting; earlier ones say when a hash was never captured.