Round 2.
Same 10 tasks.
Recipes by Mia and Ash.

Every app is checked by a script that opens it and clicks through it. The checks come in three groups: must-pass (does it start and do the basics), quality (does it work properly) and top (the extra polish). The tier says how far the app got:

Tier 3 = all three groups passed. Tier 2 = must-pass and quality passed, not all top checks passed. Tier 1 = only must-pass passed: the app runs, but some quality checks failed. No counted commit = the app never passed the must-pass checks.

The text under each badge, for example "quality 8 of 9", shows how many quality checks passed. The tier belongs to the last commit before the time limit that passes must-pass. It is not a speed rating and not a ranking between setups. One run per task: a Tier 1 next to a Tier 3 can be one unlucky run.

temperature 0.6 enforced by my proxy for both. Mia's one-click kit is at 622c796: EXL3 2.5 bpw on exllamav3 1.4.4 (1.5.3 is out), 176,128-token window, KV 4 bit, MTP draft, images on. 2.5 bpw is the kit's default for 16 GB (176K with images); the kit also lists 2.0 bpw, 3.0 bpw and 3.5 bpw profiles at other context sizes. It ran with --no-harness, its own harness off. My llama.cpp b11151 (b11349 is out): UD-Q3_K_XL, a 13.1 GB file, about 3.8 bpw, 98,304-token window, KV q4_0, MTP draft, no vision. Engines ran on the 16 GB box's GPU 1, the harness on System A. Both cells shared my test box with other agents; three runs on her kit got brief pkill hits in the agent phase (no counted result changed), and llama.cpp, which ran later, got none.

Mia's EXL3 kit | llama.cpp UD-Q3_K_XL Tasks with a counted commit: 9 | 9 of 10 Tasks at tier 3 (top): 6 | 8 Quality checks passed: 78 of 82 | 81 of 82 Output tokens in total: 1,587,451 | 1,242,419

Definitions. Counted commit = the last commit at the hard stop that passes all must-pass checks (the runner also commits leftover work at the stop; only a commit that passes counts). Tier 3 = must-pass, quality and top checks all pass. Tasks s02, s04 and s08 ran up to 150 minutes, the other seven up to 45.

temperature 0.6 enforced by my proxy for both. Mia's recipe is at a5f9bff: RadixArk NVFP4 on vLLM 0.27.1 with her backport of vLLM PR 40914 (vLLM 0.30.0 is out; PR 40914 is not merged yet), 262K window, MTP 3 drafting, one sequence at a time, prefix caching off by design so every request is cold. TensorFold 0.6.0 at c464617 (0.6.3 is out): Vontra MLX 4bit plus a DFlash2 drafter, 156K window picked by TensorFold, requests mostly warm; without a cache hit its first token still took a median of 30.0 s (median of the ten task medians). That cache difference is why the first-token line sits where it does. TensorFold wrote 4.9 times the output tokens under the same time limits. More speed is not more quality.

Mia's NVFP4 recipe | TensorFold 0.6.0 Tasks with a counted commit: 9 | 10 of 10 Tasks at tier 3 (top): 9 | 10 of 10 Time to first token, median of 10 task medians: 20.2 s | 0.99 s Output tokens in total: 1074247 | 5236055

Definitions. Counted commit = the last commit at the hard stop that passes all must-pass checks (the runner also commits leftover work at the stop; only a commit that passes counts). Tier 3 = must-pass, quality and top checks all pass. Tasks s02, s04 and s08 ran up to 150 minutes, the other seven up to 45.

one run per task, seed 1, temperature 0.6 set by my proxy for both. TensorFold recipe at a3aa898 (newer commits exist) with PARALLEL=4 (her default 5 did not fit my free memory), MLX 4bit, TensorFold 0.3.6.3 from her image (0.6.3 is out). vLLM kit at 7d0712d on its default .env, NVFP4. Both with a 262K window.

Definitions. Counted commit = the last commit at the hard stop that passes all must-pass checks (the runner also commits leftover work at the stop; only a commit that passes counts). Tier 3 = must-pass, quality and top checks all pass. Tasks s02, s04 and s08 ran up to 150 minutes, the other seven up to 45.

temperature 0.6 enforced by my proxy for all three. Mia's vLLM cell: her kit at 674155d (newer commits exist), vLLM, EXL3 4bpw, tp 2, 850K window, fp8 KV pool, DFlash2 drafts, vision on, 4 parallel, on the ASUS Ascent GX10 pair; the host side was mine: earlyoom off, oom_score_adj 800, the RAM guard armed under 2 GiB, my Jarvis timers on the first ASUS Ascent GX10 box paused from 02.10. 16:13 to cell end (3 of 10 runs started before), image rebuilt locally with one documented line change. Mia's TF kit at 978b225 (v1.3; v1.5 is out, with a new default checkpoint): TensorFold 0.6.0 plus her 53 patches, EXL3 4bpw, tp 2, 1M window, fp8 KV, DFlash2 + copy drafts, vision on, 4 parallel, on the Gigabyte AI TOP ATOM + Lenovo ThinkStation PGX pair; my harness for this cell ran as its own user (r2iso34); the host side was mine here too: earlyoom off mid-cell, the RAM guard armed under 2 GiB. The Mia TF cell ran on the Gigabyte AI TOP ATOM + Lenovo ThinkStation PGX pair, the weaker pair: a prefill check at the time gave 72-74% of Mia's README prefill (the ASUS Ascent GX10 pair: 98-99%), under Mia's 2200 MHz cap. Cause: the Lenovo box's ConnectX-7 was stuck at ~13 Gb/s after a hot-plug on my side. I fixed it after all ten runs (re-test under the same cap: 100.4-101.7%); no run was repeated. The TF cell itself ran uncapped. Ash's recipe at c464617 (newer commits exist) on TensorFold 0.6.0 (0.6.3 is out; Mia's TF cell runs 0.6.0 too): MLX 4bit, tp 2, 64K window (my pick; the recipe default of 2051 was too small for the agent tasks, its examples use larger ones), on the ASUS Ascent GX10 pair, run on 01.10. with the host as it was: earlyoom on, my Jarvis timers running. GPU clock on the ASUS Ascent GX10 pair cells was not recorded.

Definitions. Counted commit = the last commit at the hard stop that passes all must-pass checks (the runner also commits leftover work at the stop; only a commit that passes counts). Tier 3 = must-pass, quality and top checks all pass. Tasks s02, s04 and s08 ran up to 150 minutes, the other seven up to 45.