2026-09-07_aiden-2x-hardmode.md 2.0 KB
aiden production-1.2 — 2x hardmode (first ever measurement, 2026-09-06 night)
Image: aidendle94/sparkrun-vllm-glm53-gb10:production-1.2 (vLLM
0.1.dev20051+g487ecf187, torch 2.12.0 / flashinfer 0.6.15 lineage)
Weights: LibertAIDAI/GLM-5.3-Flash-NVFP4 (97 GiB/node), KV 461,730 tokens
(fp8_e4m3, 4 GiB pin), --enforce-eager, DFlash2 k=7 (incoai drafter),
SM90-NoPE FA2 route, TP=2 on 2nodes-dac (dgx1+dgx2).
Bench: tool-eval-bench 2.6.1.dev46, seed 42, hardmode, --perf, c1/c2.
Score: 159/176 (90.4) — 74 pass / 11 partial / 3 fail
New 2x quality record of the campaign (+1 vs r18-2x-nvfp4kv-spark 158), but the perf profile is weaker and the KV pool 5x smaller:
| lane (2x, seed 42) | score | pp c1 d0 | tg c2 d0 | TTFT d0 c1 | KV tokens |
|---|---|---|---|---|---|
| r18-2x-nvfp4kv-spark | 158 | 1,861 | 36.0 | 1,236ms | 2,136,194 |
| aiden 1.2 (this run) | 159 | 1,271 | 30.1 | 1,802ms | 461,730 |
| r27 (same 2x config) | 156 | 1,356 | 31.4 | 1,535ms | 2,206,618 |
Perf (llama-benchy pp2048 tg128): c1 1,271/23.5 → 1,669/22.8 (d0→d8192); c2 1,428/30.1 → 1,643/22.2. DFlash2 spec healthy: mean acceptance length 4.27–4.94 (k=7; recipe warns <0.15 = broken aux capture).
Fails (3): TC-43 Omitted Required Parameter, TC-58 Fake System Message in File, TC-61 Async Polling. Partials (11): TC-11/35/39/46/53/57/63/69/75/82/85.
Interpretation
- Aiden's lane is a quality winner at 2x by one point — within run-to-run noise of the r18-spark lane, NOT a clear win.
- It trades latency (eager, no graphs), prefill and 5x less KV headroom for that point. For interactive agent serving the r18-spark profile (2.1M KV, 36 t/s, 1.2s TTFT) remains the better 2x shape; aiden is the reference lane for "community stack" parity (and the DFlash2 drafter WORKS here — the r18-fork C4-page assert is fork-specific, not drafter-specific).
- Note: benched with tool-eval-bench dev46 (campaign anchors ran dev45) — ±1 point version drift vs the r17abi 153 gate.