|
|
@@ -0,0 +1,85 @@
|
|
|
+# Night shift 2026-09-05/06 — r18 campaign (GLM-5.3-Flash)
|
|
|
+
|
|
|
+Full lane matrix: re-baselined anchor, r18 build (base `a8c796f3` + #582 + #646,
|
|
|
+tree `67e7e6e`), 2x and 4x cells, RedHat/eugr model A/B. All benches:
|
|
|
+tool-eval-bench `2.6.1.dev45+gcf54b4bfe`, seed 42, hardmode (88 scenarios),
|
|
|
+`--perf` (llama-benchy pp2048/tg128, runs=3, c1+c2). Score = /176 weighted
|
|
|
+points; /100 = quality score; dep = deployability (α=0.7).
|
|
|
+
|
|
|
+## Combined table (score + perf)
|
|
|
+
|
|
|
+| Lane | image | score | /100 | dep | KV tokens | pp c1 | tg c1 | TTFT c1 | pp c2 | tg c2 | TTFT c2 | tg c2 @8k-ctx | verdict |
|
|
|
+|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---|
|
|
|
+| r17abi 2x (anchor) | r17 (`glm53-jovian-r12`) | 153 | 87 | 69 | ~1.25M | 1,577 | 29.0 | 1,362ms | 1,595 | 41.7 | 2,093ms | 29.7 | re-baseline |
|
|
|
+| r18 2x (ds4) | r18 | 157 | 89 | 71 | ~1.25M | 1,473 | 30.7 | 1,481ms | 1,374 | 40.2 | 2,268ms | 30.6 | PASS, 2x production candidate |
|
|
|
+| RedHat MTP k=5 2x | lab | 156 | 89 | 70 | ~1.25M | 1,340 | 26.8 | 1,638ms | 460 | 22.4 | 4,667ms | 14.7 | quality PASS, perf FAIL at c2 |
|
|
|
+| eugr MTP k=5 2x | eugr b12x | 143 | 81 | 65 | ~1.25M | 1,292 | 31.1 | 2,397ms | 1,388 | 41.6 | 2,751ms | 17.7 | FAIL (−10 vs anchor) |
|
|
|
+| r18 4x RoCE | r18 | 156 | 89 | 76 | 1.47M | 2,904 | 47.7 | 762ms | 2,413 | 70.9 | 1,399ms | 35.9 | PASS, throughput king |
|
|
|
+| r18 4x DCP4 | r18 | 161 | 91 | 75 | 4.84M | 1,639 | 43.1 | 1,521ms | 1,756 | 51.1 | 1,637ms | 30.0 | BEST score, prefill tax |
|
|
|
+| r18 4x nvfp4 KV | r18 | 161 | 91 | 77 | 2.35M | 2,614 | 45.3 | 873ms | 2,250 | 62.6 | 1,368ms | 35.6 | PASS, best deployability |
|
|
|
+| r18 4x DFlash2 k=7 | r18 | — | — | — | — | — | — | — | — | — | — | — | BLOCKED (fork C4-page assert) |
|
|
|
+| dsv4-flash-vision eugr 2x | eugr b12x | — | — | — | — | — | — | — | — | — | — | — | BLOCKED (no V4-Vision in image) |
|
|
|
+
|
|
|
+pp/tg = tokens/s (prefill 2048 / generate 128); TTFT = ms; d0 = no extra
|
|
|
+context, @8k-ctx = 8192 tokens of history.
|
|
|
+
|
|
|
+## Reading the table
|
|
|
+
|
|
|
+- **4x RoCE** = the all-round production candidate: best prefill (2.9k, 2×
|
|
|
+ 2x), best c2 decode (70.9), half the 2x TTFT, quality 156 = 2x r18.
|
|
|
+- **4x DCP4** = capacity + quality play: 4.84M KV (3.3× plain 4x at the same
|
|
|
+ 10GiB pin — the sparkring steal) and the best score (161), but the DCP ring
|
|
|
+ exchange costs prefill (1.6k vs 2.9k).
|
|
|
+- **4x nvfp4 KV** ≈ RoCE perf, +1.59× KV, best deployability (77).
|
|
|
+ Next cell: DCP4 + nvfp4 KV combined (~7.7M KV projected).
|
|
|
+- **RedHat MTP k=5**: quality fine, but the lab image's MTP path collapses
|
|
|
+ under concurrency (pp c2 460 t/s, TTFT 4.7s) — dropped from the candidate
|
|
|
+ set once perf was on the table.
|
|
|
+- **eugr**: no edge anywhere (quality −10, decode degrades at depth); 4x cell
|
|
|
+ skipped. Lanes demoted.
|
|
|
+- **r18 2x** beats the anchor on both axes (157/89 vs 153/87) — 2x default
|
|
|
+ candidate pending operator sign-off.
|
|
|
+
|
|
|
+## Blocked lanes (documented in recipe headers)
|
|
|
+
|
|
|
+- **DFlash2 k=7 drafter**: drafter inherits the engine's FP8 KV dtype; B12X
|
|
|
+ and FLASH_ATTN both reject it on SM121 ("FP8 KV cache requires FA3 on SM90
|
|
|
+ or FA4 on SM100"); with drafter `kv_cache_dtype=auto` the fork's own
|
|
|
+ assertion fires ("GLM MLA parent-page stride must be an exact number of C4
|
|
|
+ pages"). Needs a fork-side fix.
|
|
|
+- **dsv4-flash-vision eugr**: the eugr 0904 image's
|
|
|
+ `DeepseekV4ForCausalLM` has no `aligner` module (predates V4-Vision
|
|
|
+ support); the Vision-Exp vision weights fail to load. Fork lane remains
|
|
|
+ the only V4-Vision path.
|
|
|
+
|
|
|
+## Ops log (condensed)
|
|
|
+
|
|
|
+- Re-baseline (queue 1a) first, on the current r17 image, before any r18
|
|
|
+ gate — all comparisons above use the 153/176 anchor.
|
|
|
+- r18 build on dgx4: 5 attempts. Root causes: transient apt 503s;
|
|
|
+ kv-cache-cleanup patch anchor replaced upstream (MemorySnapshot flow —
|
|
|
+ patch now skips cleanly); **staged-deps submodule stubs** — both
|
|
|
+ `flash-attention/csrc/cutlass` and `MSA/python/fmha_sm100/cutlass` were
|
|
|
+ empty dirs in `deps-src.tar.gz` (r18 tree dropped implicit cutlass include
|
|
|
+ wiring; MSA install needs the dir) — populated with each project's own
|
|
|
+ pin (nvidia/cutlass `62750a2b` / `eb61c911`), 23k macOS AppleDouble
|
|
|
+ files purged.
|
|
|
+- DCP4 requires `--decode-context-parallel-size 4` +
|
|
|
+ `--cp-kv-cache-interleave-size 4` (GLM C4 DCP); B12X is not DCP-compatible
|
|
|
+ (engine falls to FLASH_ATTN for that cell).
|
|
|
+- #646 geometry verified live: "attention block size 2304" = the 2048+256
|
|
|
+ split in the serve log.
|
|
|
+- Image separation (operator direction): `docker-images/glm53-jovian-r18/`
|
|
|
+ + `glm53-jovian-r18:local` tag; r12 dir restored to r17 state
|
|
|
+ (`git checkout 582343c`); r12 tag removed from nodes (points at nothing
|
|
|
+ until an honest r17 rebuild — r17-era deps staging unrecoverable, see
|
|
|
+ state doc).
|
|
|
+- 4x recipes need `tensor_parallel_size = 4` (world size must cover nnodes);
|
|
|
+ 2x→4x port of the r17abi recipe is the r18-4x family.
|
|
|
+
|
|
|
+## Reports
|
|
|
+
|
|
|
+Full per-lane reports: `recipes/<lane>/reports/` (r17abi: `2026-09-06_r18-rebaseline-r17abi-2x.md`,
|
|
|
+`2026-09-06_r18-image-ds4.md`; redhat/eugr: `2026-09-06_*.md`; r18-4x family:
|
|
|
+`2026-09-06_r18-4x*.md`). Analysis + open decisions:
|
|
|
+`specs/tech-architecture/glm53-recipe-state.md` → "Night-shift results".
|
