Upstream watch — 2026-10-14

Research pass over the sources we track (NVIDIA forums, engine repos, model repos, AA) since our last cuts (GLM-5.3-Flash r39 = 2026-09-21, kindling spark-glm53:v10 = 2026-09-28, sparkring = 2026-09-29, qwen38 = vLLM 0.27.1, dsv41 = tonyd2wild 09-10). Purpose: pick upgrades and new test candidates for the lanes.

1. Engine stack upgrades

FlashInfer absorbs b12x (P1 — the big one)

local-inference-lab/b12x archived 2026-10-06; README replaced by a FlashInfer migration guide. Kernels + preparation/autotune/compile/ workspace/checkpoint infra now live at flashinfer/experimental/b12x (merge: flashinfer PR #5767, commit fc8fdc17f70, 2026-10-06).
FlashInfer v0.7.1 (2026-10-09) / v0.7.2rc3 (10-13) highlights: quantized expert-parallel MoE on SM12x (GB10!), fused Kimi-K3 spec verification, cuDNN linear-attention prefill on Blackwell, DeepSeek-V4.1 dual-cache sparse MLA, compact GLM53-NoPE caches. flashinfer-python[b12x] extra; needs torch ≥2.12 + CUTLASS DSL 4.7.1.
Our pins: flashinfer 0.6.15/0.6.17 + standalone b12x (r39: c2dc1cf022).
Action: r40 profile once the lab cuts a vLLM+flashinfer-0.7.x pairing (jovian vLLM tip is 10-04 ff292a9c63 — predates the merge, so their next cut is the one to take). Watch local-inference-lab/vllm tags.
The schema 22 tuning-decision invalidation (b12x 10-06) means autotune caches from old cuts should be dropped on upgrade.

vLLM mainline (P2 for qwen38 lanes)

v0.28 (08-26) → v0.29 (09-09) → v0.30 (09-22) → v0.31.0 (10-05). We serve qwen38 on 0.27.1. Real recipes already run 0.30/0.31 on Spark: MiaAI 2x Flash-Next (0.30, TP2+EP+MTP3), chisel-spark-recipes (0.31.0).
vLLM #58578 / PR #59740: reduced-vocabulary draft head (50,858-token drafter slice, 0.5 GB instead of 2.5 GB) — packaged as a plugin in yuanlin2004/chisel-spark-recipes for Qwen3.8-27B-FP8 (23.1 tok/s c1).
Action: rebase qwen38-flash-next / qwen38-27b lanes on 0.31.0 (or the recipe's pinned stack) and re-bench; evaluate the draft-head plugin.

sparkring rebase (P2)

ghcr.io/fujitsupolycom/sparkring moved: dev-20261013-dcp2prefill- status038-sircl-tp6 (10-13!) — DCP2 prefill, TP6, sircl 0.6.0, status038; a kraken line since 10-01. Our recipe pins the 09-29 digest (71d41057…).
MiMo-V2.6 model repos re-cut 2026-10-09 (Flash-MOPD, Pro-MOPD, Flash-RL, Pro-RL). Our lanes serve the older checkpoints.
Action: re-pin the MiMo lanes to the newest image + 10-09 checkpoints, re-bench (quality may move — RL is new for Flash).

2. New engines (test candidates — evaluate, don't adopt blindly)

kindlingai/gr8 (P1 interest — same org as our kindling stack)

From-scratch CUDA engine, full GLM-5.3 (600B, 256 experts) on 3-6 Sparks, own RoCE-v2 verbs fabric (no vLLM/PyTorch/NCCL), OpenAI API, one-line ./gr8 glm53 up. Pushed 2026-10-14; beta.
Model: kindlingai/glm-5.3-exlr8-k2-k3-k4 (updated 10-14, 300K dl).
Numbers (their RigMark runs, clocks capped 2200): TP=4 decode 49.3/30.0 t/s code/prose, aggregate 64 @4 streams, prefill 8K cold 1,374 t/s, KV pool 310K tok (415K with int4 KV). DFlash2+MTP drafting.
The license-locked full model becomes servable — flag: Z.ai license is license:other; quants redistribute weights, commercial use unclear.
Action: install + 4x lane test + tool-eval quality gate. Compare with GLM-5.3-Flash (AA 41.8) vs full GLM-5.3 (AA 44.8, max-thinking 50.3).

DGPP (HawkBearPig/dgpp) (P2)

C++/CUDA engine for Spark, 120★, pushed 10-14, 107-post forum thread (383406). OpenAI API, 1/2/4-node templates.
Matrix: GLM-5.3-Flash unsloth FP8 w4; GLM-5.3-Flash NVFP4-FP8 hybrid w2/w4; full GLM-5.3 int4/int8 w4 (272K FP8 KV); MiMo-V2.6-Flash-RL w2/w4; DS4.1-Flash w4; Qwen3.8-Flash-Next NVFP4 w1/w2 (mmap n-gram, decode graphs, fp8 dense — +40% over published vLLM on Qwen single-node); GLM-4.7.
Numbers (5-class spread, warm decode): Flash-Next NVFP4 w2 62-75 t/s; GLM-5.3-Flash NVFP4 w4 50-58 t/s; full GLM-5.3 w4 25-29 t/s (42-47 @C4).
Action: bench DGPP vs our vLLM lanes on (a) Qwen3.8-Flash-Next w1, (b) MiMo-RL w2 — quality via tool-eval-bench, same harness both sides.

veloGB10 v0.7.0 (sf-stav) (P3 — watch)

Rust engine, EXL3-focused; Qwen3.8-Flash-Next EXL3 3.05bpw TP=2 @186 t/s (their claim); NVFP4 supported but unoptimized; TP=4 pending; no vision.
Model: doth4580/Qwen3.8-Flash-Next-EXL3-3.05bpw. Re-test when TP4/NVFP4 mature.

dsgx10 (forum 385803) (P3 — watch)

From-scratch DS4.1-Flash engine for 4×GB10 switchless ring; +5-70% over knapcio v2.2/v2.4 baselines, bit-exact, built with Claude. One post so far (10-13); no repo link yet. Track the thread.

3. Model landscape (AA / HF)

Model	AA index	License	Size	Note
MiMo-V2.6-Pro	46.3	MIT	~1TB	overkill, one-shot (unchanged)
Muse Wonder Full (Meta)	56	?	?	10-12, likely closed/large — check
Kimi K3 (max)	(top open)	other	2.78 TB	too big
Grok 4.7	46	closed	—	—
Qwen3.8 Max (0902)	45	closed	—	—
GLM-5.3 (full)	44.8	license:other	600B	gr8/DGPP make it servable; license q.
GLM-5.3-Flash	41.8	MIT	320B	our flagship; zai repo touched 10-12 (minor)
Ling 3.1 Flash (InclusionAI)	41	MIT	~350-400B? (512/8 experts, 262K ctx)	released ~10-13; watch for day-0 quants/recipes
MiMo-V2.6-Flash	38	MIT	311B	re-cut 10-09; RL variant new
DeepSeek V4.1 Flash	39 (max)	MIT	—	our dsv41 lanes
Qwen3.8-Flash-Next	40	Apache	125B A6B	the coder model; see new lanes

Qwen3.8-Flash-Next 2x TP2+EP lane (P2 — new lane)

Forum 385731 (MiaAI recipe, vLLM 0.30): 43-46 t/s single, 174 t/s aggregate @8-16 conc, 2.9K t/s prefill, 158/164 HumanEval, 74.2% aider polyglot, 262K ctx, MTP=3, fp8 KV. Soak: 2-h agent run zero errors.
Our flash-next recipes are single-node. Action: 2x lane with --tensor-parallel-size 2 --enable-expert-parallel, MTP=3; bench with tool-eval-bench + concurrency sweep.

Full GLM-5.3 on the 4-node lane (P2 — new capability)

Via gr8 (exlr8) or DGPP (int4/int8-mix). The presentation's "license-locked, unusable" stance should be revisited for personal/research use; commercial remains questionable. The 4x cluster is idle — this is the highest-ceiling quality upgrade available without new hardware.

4. Meta / infra notes

MikroTik CRS812 200G thread (385776, 10-13): 4× Spark via CRS812 with 2×400G→2×200G breakout: NCCL AG 20.4 GB/s vs 25 theoretical (MTU 9000, NCCL 2.28.9 RoCE, FEC on; logical-interface bonding attempted). We run the same class of switch (SW2-MUC) — compare our own fabric-check AG numbers; the thread's resolution may apply to us.
tool-eval-bench moved again: "with the most recent TEB, high reasoning is no longer enough for 90+" (forum 381350 #827, 10-12); community reports 90-93 on current TEB where older pins read lower. Our pin: v2.7.0. Any cross-engine A/B must share a harness version; consider a re-baseline of the anchor lanes before comparing against community numbers.
RigMark (github.com/alexellis/rigmark) — the benchmark gr8 uses (reasoning_effort low, clock-capped, machine-readable kindling.json). Worth a look as a cross-engine comparison harness.
eugr/spark-vllm-b12x nightlies continue (10-14); our qwen38-flash-next lane pins the 09-25 digest because of the garbling bug — consider testing a fresh pinned digest for the fix.
The 16-node operator thread (385107) wants GLM-5.3-full NVFP4 16x, DS4.1+GLM-5.3-Flash 4+4, MiMo-Pro 16x on spark-vllm-docker — our shared recipe repo in the wild; their TP-over-switch questions may produce reusable data.

Recommended order

r40 cut watch (flashinfer-0.7.x pairing) — wait for the lab's tag, then cut the profile for kindling/flash lanes.
gr8 on the 4x lane — full GLM-5.3; quality-gate with tool-eval-bench.
sparkring rebase + MiMo 10-09 re-cuts — cheap, existing lanes.
Qwen3.8-Flash-Next 2x EP lane — the coder-endpoint use case.
DGPP bench (Qwen w1, MiMo-RL w2) — cross-engine comparison.
Watch: veloGB10 TP4, dsgx10 repo, Ling-3.1-flash quants, TEB re-baseline.

upstream-watch-2026-10-14.md 8.3 KB Riwayat Mentahan