upstream watch 2026-10-14: b12x merged into FlashInfer 0.7.x (r40 path), gr8/DGPP/veloGB10 engines, MiMo 10-09 re-cuts + sparkring rebase, Qwen3.8-Flash-Next 2x EP lane, Ling-3.1-flash (MIT) as GLM-5.3-Flash competitor

specs/tech-architecture/upstream-watch-2026-10-14.md

@@ -0,0 +1,147 @@
+# Upstream watch — 2026-10-14
+
+Research pass over the sources we track (NVIDIA forums, engine repos, model
+repos, AA) since our last cuts (GLM-5.3-Flash r39 = 2026-09-21, kindling
+`spark-glm53:v10` = 2026-09-28, sparkring = 2026-09-29, qwen38 = vLLM
+0.27.1, dsv41 = tonyd2wild 09-10). Purpose: pick upgrades and new test
+candidates for the lanes.
+
+## 1. Engine stack upgrades
+
+### FlashInfer absorbs b12x (P1 — the big one)
+- `local-inference-lab/b12x` **archived 2026-10-06**; README replaced by a
+  FlashInfer migration guide. Kernels + preparation/autotune/compile/
+  workspace/checkpoint infra now live at `flashinfer/experimental/b12x`
+  (merge: flashinfer PR #5767, commit `fc8fdc17f70`, 2026-10-06).
+- **FlashInfer v0.7.1 (2026-10-09)** / v0.7.2rc3 (10-13) highlights: quantized
+  **expert-parallel MoE on SM12x** (GB10!), fused Kimi-K3 spec verification,
+  cuDNN linear-attention prefill on Blackwell, **DeepSeek-V4.1 dual-cache
+  sparse MLA**, **compact GLM53-NoPE caches**. `flashinfer-python[b12x]`
+  extra; needs torch ≥2.12 + CUTLASS DSL 4.7.1.
+- Our pins: flashinfer 0.6.15/0.6.17 + standalone b12x (r39: `c2dc1cf022`).
+- **Action:** r40 profile once the lab cuts a vLLM+flashinfer-0.7.x pairing
+  (jovian vLLM tip is 10-04 `ff292a9c63` — predates the merge, so their next
+  cut is the one to take). Watch `local-inference-lab/vllm` tags.
+- The `schema 22` tuning-decision invalidation (b12x 10-06) means autotune
+  caches from old cuts should be dropped on upgrade.
+
+### vLLM mainline (P2 for qwen38 lanes)
+- v0.28 (08-26) → v0.29 (09-09) → **v0.30 (09-22)** → **v0.31.0 (10-05)**.
+  We serve qwen38 on 0.27.1. Real recipes already run 0.30/0.31 on Spark:
+  MiaAI 2x Flash-Next (0.30, TP2+EP+MTP3), chisel-spark-recipes (0.31.0).
+- vLLM #58578 / PR #59740: reduced-vocabulary draft head (50,858-token
+  drafter slice, 0.5 GB instead of 2.5 GB) — packaged as a plugin in
+  yuanlin2004/chisel-spark-recipes for Qwen3.8-27B-FP8 (23.1 tok/s c1).
+- **Action:** rebase qwen38-flash-next / qwen38-27b lanes on 0.31.0 (or the
+  recipe's pinned stack) and re-bench; evaluate the draft-head plugin.
+
+### sparkring rebase (P2)
+- `ghcr.io/fujitsupolycom/sparkring` moved: `dev-20261013-dcp2prefill-
+  status038-sircl-tp6` (10-13!) — DCP2 prefill, **TP6**, sircl 0.6.0,
+  status038; a `kraken` line since 10-01. Our recipe pins the 09-29 digest
+  (`71d41057…`).
+- **MiMo-V2.6 model repos re-cut 2026-10-09** (Flash-MOPD, Pro-MOPD,
+  Flash-RL, Pro-RL). Our lanes serve the older checkpoints.
+- **Action:** re-pin the MiMo lanes to the newest image + 10-09 checkpoints,
+  re-bench (quality may move — RL is new for Flash).
+
+## 2. New engines (test candidates — evaluate, don't adopt blindly)
+
+### kindlingai/gr8 (P1 interest — same org as our kindling stack)
+- From-scratch CUDA engine, **full GLM-5.3 (600B, 256 experts) on 3-6
+  Sparks**, own RoCE-v2 verbs fabric (no vLLM/PyTorch/NCCL), OpenAI API,
+  one-line `./gr8 glm53 up`. Pushed 2026-10-14; beta.
+- Model: `kindlingai/glm-5.3-exlr8-k2-k3-k4` (updated 10-14, 300K dl).
+- Numbers (their RigMark runs, clocks capped 2200): TP=4 decode 49.3/30.0
+  t/s code/prose, aggregate 64 @4 streams, prefill 8K cold 1,374 t/s, KV
+  pool 310K tok (415K with int4 KV). DFlash2+MTP drafting.
+- The license-locked full model becomes servable — flag: Z.ai license is
+  `license:other`; quants redistribute weights, commercial use unclear.
+- **Action:** install + 4x lane test + tool-eval quality gate. Compare with
+  GLM-5.3-Flash (AA 41.8) vs full GLM-5.3 (AA 44.8, max-thinking 50.3).
+
+### DGPP (HawkBearPig/dgpp) (P2)
+- C++/CUDA engine for Spark, 120★, pushed 10-14, 107-post forum thread
+  (383406). OpenAI API, 1/2/4-node templates.
+- Matrix: GLM-5.3-Flash unsloth FP8 w4; GLM-5.3-Flash NVFP4-FP8 hybrid w2/w4;
+  **full GLM-5.3 int4/int8 w4** (272K FP8 KV); **MiMo-V2.6-Flash-RL w2/w4**;
+  DS4.1-Flash w4; Qwen3.8-Flash-Next NVFP4 w1/w2 (mmap n-gram, decode
+  graphs, fp8 dense — +40% over published vLLM on Qwen single-node); GLM-4.7.
+- Numbers (5-class spread, warm decode): Flash-Next NVFP4 w2 62-75 t/s;
+  GLM-5.3-Flash NVFP4 w4 50-58 t/s; full GLM-5.3 w4 25-29 t/s (42-47 @C4).
+- **Action:** bench DGPP vs our vLLM lanes on (a) Qwen3.8-Flash-Next w1,
+  (b) MiMo-RL w2 — quality via tool-eval-bench, same harness both sides.
+
+### veloGB10 v0.7.0 (sf-stav) (P3 — watch)
+- Rust engine, EXL3-focused; Qwen3.8-Flash-Next EXL3 3.05bpw TP=2 @186 t/s
+  (their claim); NVFP4 supported but unoptimized; TP=4 pending; no vision.
+- Model: doth4580/Qwen3.8-Flash-Next-EXL3-3.05bpw. Re-test when TP4/NVFP4
+  mature.
+
+### dsgx10 (forum 385803) (P3 — watch)
+- From-scratch DS4.1-Flash engine for 4×GB10 switchless ring; +5-70% over
+  knapcio v2.2/v2.4 baselines, bit-exact, built with Claude. One post so
+  far (10-13); no repo link yet. Track the thread.
+
+## 3. Model landscape (AA / HF)
+
+| Model | AA index | License | Size | Note |
+|---|---|---|---|---|
+| MiMo-V2.6-Pro | 46.3 | MIT | ~1TB | overkill, one-shot (unchanged) |
+| **Muse Wonder Full (Meta)** | 56 | ? | ? | 10-12, likely closed/large — check |
+| **Kimi K3 (max)** | (top open) | other | 2.78 TB | too big |
+| **Grok 4.7** | 46 | closed | — | — |
+| **Qwen3.8 Max (0902)** | 45 | closed | — | — |
+| GLM-5.3 (full) | 44.8 | license:other | 600B | gr8/DGPP make it servable; license q. |
+| GLM-5.3-Flash | 41.8 | MIT | 320B | our flagship; zai repo touched 10-12 (minor) |
+| **Ling 3.1 Flash (InclusionAI)** | 41 | **MIT** | ~350-400B? (512/8 experts, 262K ctx) | released ~10-13; watch for day-0 quants/recipes |
+| MiMo-V2.6-Flash | 38 | MIT | 311B | re-cut 10-09; RL variant new |
+| DeepSeek V4.1 Flash | 39 (max) | MIT | — | our dsv41 lanes |
+| Qwen3.8-Flash-Next | 40 | Apache | 125B A6B | the coder model; see new lanes |
+
+### Qwen3.8-Flash-Next 2x TP2+EP lane (P2 — new lane)
+- Forum 385731 (MiaAI recipe, vLLM 0.30): 43-46 t/s single, **174 t/s
+  aggregate @8-16 conc**, 2.9K t/s prefill, 158/164 HumanEval, **74.2%
+  aider polyglot**, 262K ctx, MTP=3, fp8 KV. Soak: 2-h agent run zero errors.
+- Our flash-next recipes are single-node. **Action:** 2x lane with
+  `--tensor-parallel-size 2 --enable-expert-parallel`, MTP=3; bench with
+  tool-eval-bench + concurrency sweep.
+
+### Full GLM-5.3 on the 4-node lane (P2 — new capability)
+- Via gr8 (exlr8) or DGPP (int4/int8-mix). The presentation's "license-locked,
+  unusable" stance should be revisited for personal/research use; commercial
+  remains questionable. The 4x cluster is idle — this is the highest-ceiling
+  quality upgrade available without new hardware.
+
+## 4. Meta / infra notes
+
+- **MikroTik CRS812 200G thread (385776, 10-13):** 4× Spark via CRS812 with
+  2×400G→2×200G breakout: NCCL AG 20.4 GB/s vs 25 theoretical (MTU 9000,
+  NCCL 2.28.9 RoCE, FEC on; logical-interface bonding attempted). We run the
+  same class of switch (SW2-MUC) — compare our own fabric-check AG numbers;
+  the thread's resolution may apply to us.
+- **tool-eval-bench moved again:** "with the most recent TEB, `high` reasoning
+  is no longer enough for 90+" (forum 381350 #827, 10-12); community reports
+  90-93 on current TEB where older pins read lower. Our pin: v2.7.0. Any
+  cross-engine A/B must share a harness version; consider a re-baseline of
+  the anchor lanes before comparing against community numbers.
+- **RigMark** (github.com/alexellis/rigmark) — the benchmark gr8 uses
+  (reasoning_effort low, clock-capped, machine-readable kindling.json).
+  Worth a look as a cross-engine comparison harness.
+- eugr/spark-vllm-b12x nightlies continue (10-14); our qwen38-flash-next
+  lane pins the 09-25 digest because of the garbling bug — consider testing
+  a fresh pinned digest for the fix.
+- The 16-node operator thread (385107) wants GLM-5.3-full NVFP4 16x,
+  DS4.1+GLM-5.3-Flash 4+4, MiMo-Pro 16x on spark-vllm-docker — our shared
+  recipe repo in the wild; their TP-over-switch questions may produce
+  reusable data.
+
+## Recommended order
+
+1. **r40 cut watch** (flashinfer-0.7.x pairing) — wait for the lab's tag, then
+   cut the profile for kindling/flash lanes.
+2. **gr8 on the 4x lane** — full GLM-5.3; quality-gate with tool-eval-bench.
+3. **sparkring rebase + MiMo 10-09 re-cuts** — cheap, existing lanes.
+4. **Qwen3.8-Flash-Next 2x EP lane** — the coder-endpoint use case.
+5. **DGPP bench** (Qwen w1, MiMo-RL w2) — cross-engine comparison.
+6. Watch: veloGB10 TP4, dsgx10 repo, Ling-3.1-flash quants, TEB re-baseline.