# Upstream watch — 2026-10-14

Research pass over the sources we track (NVIDIA forums, engine repos, model
repos, AA) since our last cuts (GLM-5.3-Flash r39 = 2026-09-21, kindling
`spark-glm53:v10` = 2026-09-28, sparkring = 2026-09-29, qwen38 = vLLM
0.27.1, dsv41 = tonyd2wild 09-10). Purpose: pick upgrades and new test
candidates for the lanes.

## 1. Engine stack upgrades

### FlashInfer absorbs b12x (P1 — the big one)
- `local-inference-lab/b12x` **archived 2026-10-06**; README replaced by a
  FlashInfer migration guide. Kernels + preparation/autotune/compile/
  workspace/checkpoint infra now live at `flashinfer/experimental/b12x`
  (merge: flashinfer PR #5767, commit `fc8fdc17f70`, 2026-10-06).
- **FlashInfer v0.7.1 (2026-10-09)** / v0.7.2rc3 (10-13) highlights: quantized
  **expert-parallel MoE on SM12x** (GB10!), fused Kimi-K3 spec verification,
  cuDNN linear-attention prefill on Blackwell, **DeepSeek-V4.1 dual-cache
  sparse MLA**, **compact GLM53-NoPE caches**. `flashinfer-python[b12x]`
  extra; needs torch ≥2.12 + CUTLASS DSL 4.7.1.
- Our pins: flashinfer 0.6.15/0.6.17 + standalone b12x (r39: `c2dc1cf022`).
- **Action:** r40 profile once the lab cuts a vLLM+flashinfer-0.7.x pairing
  (jovian vLLM tip is 10-04 `ff292a9c63` — predates the merge, so their next
  cut is the one to take). Watch `local-inference-lab/vllm` tags.
- The `schema 22` tuning-decision invalidation (b12x 10-06) means autotune
  caches from old cuts should be dropped on upgrade.

### vLLM mainline (P2 for qwen38 lanes)
- v0.28 (08-26) → v0.29 (09-09) → **v0.30 (09-22)** → **v0.31.0 (10-05)**.
  We serve qwen38 on 0.27.1. Real recipes already run 0.30/0.31 on Spark:
  MiaAI 2x Flash-Next (0.30, TP2+EP+MTP3), chisel-spark-recipes (0.31.0).
- vLLM #58578 / PR #59740: reduced-vocabulary draft head (50,858-token
  drafter slice, 0.5 GB instead of 2.5 GB) — packaged as a plugin in
  yuanlin2004/chisel-spark-recipes for Qwen3.8-27B-FP8 (23.1 tok/s c1).
- **Action:** rebase qwen38-flash-next / qwen38-27b lanes on 0.31.0 (or the
  recipe's pinned stack) and re-bench; evaluate the draft-head plugin.

### sparkring rebase (P2)
- `ghcr.io/fujitsupolycom/sparkring` moved: `dev-20261013-dcp2prefill-
  status038-sircl-tp6` (10-13!) — DCP2 prefill, **TP6**, sircl 0.6.0,
  status038; a `kraken` line since 10-01. Our recipe pins the 09-29 digest
  (`71d41057…`).
- **MiMo-V2.6 model repos re-cut 2026-10-09** (Flash-MOPD, Pro-MOPD,
  Flash-RL, Pro-RL). Our lanes serve the older checkpoints.
- **Action:** re-pin the MiMo lanes to the newest image + 10-09 checkpoints,
  re-bench (quality may move — RL is new for Flash).

## 2. New engines (test candidates — evaluate, don't adopt blindly)

### kindlingai/gr8 (P1 interest — same org as our kindling stack)
- From-scratch CUDA engine, **full GLM-5.3 (600B, 256 experts) on 3-6
  Sparks**, own RoCE-v2 verbs fabric (no vLLM/PyTorch/NCCL), OpenAI API,
  one-line `./gr8 glm53 up`. Pushed 2026-10-14; beta.
- Model: `kindlingai/glm-5.3-exlr8-k2-k3-k4` (updated 10-14, 300K dl).
- Numbers (their RigMark runs, clocks capped 2200): TP=4 decode 49.3/30.0
  t/s code/prose, aggregate 64 @4 streams, prefill 8K cold 1,374 t/s, KV
  pool 310K tok (415K with int4 KV). DFlash2+MTP drafting.
- The license-locked full model becomes servable — flag: Z.ai license is
  `license:other`; quants redistribute weights, commercial use unclear.
- **Action:** install + 4x lane test + tool-eval quality gate. Compare with
  GLM-5.3-Flash (AA 41.8) vs full GLM-5.3 (AA 44.8, max-thinking 50.3).

### DGPP (HawkBearPig/dgpp) (P2)
- C++/CUDA engine for Spark, 120★, pushed 10-14, 107-post forum thread
  (383406). OpenAI API, 1/2/4-node templates.
- Matrix: GLM-5.3-Flash unsloth FP8 w4; GLM-5.3-Flash NVFP4-FP8 hybrid w2/w4;
  **full GLM-5.3 int4/int8 w4** (272K FP8 KV); **MiMo-V2.6-Flash-RL w2/w4**;
  DS4.1-Flash w4; Qwen3.8-Flash-Next NVFP4 w1/w2 (mmap n-gram, decode
  graphs, fp8 dense — +40% over published vLLM on Qwen single-node); GLM-4.7.
- Numbers (5-class spread, warm decode): Flash-Next NVFP4 w2 62-75 t/s;
  GLM-5.3-Flash NVFP4 w4 50-58 t/s; full GLM-5.3 w4 25-29 t/s (42-47 @C4).
- **Action:** bench DGPP vs our vLLM lanes on (a) Qwen3.8-Flash-Next w1,
  (b) MiMo-RL w2 — quality via tool-eval-bench, same harness both sides.

### veloGB10 v0.7.0 (sf-stav) (P3 — watch)
- Rust engine, EXL3-focused; Qwen3.8-Flash-Next EXL3 3.05bpw TP=2 @186 t/s
  (their claim); NVFP4 supported but unoptimized; TP=4 pending; no vision.
- Model: doth4580/Qwen3.8-Flash-Next-EXL3-3.05bpw. Re-test when TP4/NVFP4
  mature.

### dsgx10 (forum 385803) (P3 — watch)
- From-scratch DS4.1-Flash engine for 4×GB10 switchless ring; +5-70% over
  knapcio v2.2/v2.4 baselines, bit-exact, built with Claude. One post so
  far (10-13); no repo link yet. Track the thread.

## 3. Model landscape (AA / HF)

| Model | AA index | License | Size | Note |
|---|---|---|---|---|
| MiMo-V2.6-Pro | 46.3 | MIT | ~1TB | overkill, one-shot (unchanged) |
| **Muse Wonder Full (Meta)** | 56 | ? | ? | 10-12, likely closed/large — check |
| **Kimi K3 (max)** | (top open) | other | 2.78 TB | too big |
| **Grok 4.7** | 46 | closed | — | — |
| **Qwen3.8 Max (0902)** | 45 | closed | — | — |
| GLM-5.3 (full) | 44.8 | license:other | 600B | gr8/DGPP make it servable; license q. |
| GLM-5.3-Flash | 41.8 | MIT | 320B | our flagship; zai repo touched 10-12 (minor) |
| **Ling 3.1 Flash (InclusionAI)** | 41 | **MIT** | ~350-400B? (512/8 experts, 262K ctx) | released ~10-13; watch for day-0 quants/recipes |
| MiMo-V2.6-Flash | 38 | MIT | 311B | re-cut 10-09; RL variant new |
| DeepSeek V4.1 Flash | 39 (max) | MIT | — | our dsv41 lanes |
| Qwen3.8-Flash-Next | 40 | Apache | 125B A6B | the coder model; see new lanes |

### Qwen3.8-Flash-Next 2x TP2+EP lane (P2 — new lane)
- Forum 385731 (MiaAI recipe, vLLM 0.30): 43-46 t/s single, **174 t/s
  aggregate @8-16 conc**, 2.9K t/s prefill, 158/164 HumanEval, **74.2%
  aider polyglot**, 262K ctx, MTP=3, fp8 KV. Soak: 2-h agent run zero errors.
- Our flash-next recipes are single-node. **Action:** 2x lane with
  `--tensor-parallel-size 2 --enable-expert-parallel`, MTP=3; bench with
  tool-eval-bench + concurrency sweep.

### Full GLM-5.3 on the 4-node lane (P2 — new capability)
- Via gr8 (exlr8) or DGPP (int4/int8-mix). The presentation's "license-locked,
  unusable" stance should be revisited for personal/research use; commercial
  remains questionable. The 4x cluster is idle — this is the highest-ceiling
  quality upgrade available without new hardware.

## 4. Meta / infra notes

- **MikroTik CRS812 200G thread (385776, 10-13):** 4× Spark via CRS812 with
  2×400G→2×200G breakout: NCCL AG 20.4 GB/s vs 25 theoretical (MTU 9000,
  NCCL 2.28.9 RoCE, FEC on; logical-interface bonding attempted). We run the
  same class of switch (SW2-MUC) — compare our own fabric-check AG numbers;
  the thread's resolution may apply to us.
- **tool-eval-bench moved again:** "with the most recent TEB, `high` reasoning
  is no longer enough for 90+" (forum 381350 #827, 10-12); community reports
  90-93 on current TEB where older pins read lower. Our pin: v2.7.0. Any
  cross-engine A/B must share a harness version; consider a re-baseline of
  the anchor lanes before comparing against community numbers.
- **RigMark** (github.com/alexellis/rigmark) — the benchmark gr8 uses
  (reasoning_effort low, clock-capped, machine-readable kindling.json).
  Worth a look as a cross-engine comparison harness.
- eugr/spark-vllm-b12x nightlies continue (10-14); our qwen38-flash-next
  lane pins the 09-25 digest because of the garbling bug — consider testing
  a fresh pinned digest for the fix.
- The 16-node operator thread (385107) wants GLM-5.3-full NVFP4 16x,
  DS4.1+GLM-5.3-Flash 4+4, MiMo-Pro 16x on spark-vllm-docker — our shared
  recipe repo in the wild; their TP-over-switch questions may produce
  reusable data.

## Recommended order

1. **r40 cut watch** (flashinfer-0.7.x pairing) — wait for the lab's tag, then
   cut the profile for kindling/flash lanes.
2. **gr8 on the 4x lane** — full GLM-5.3; quality-gate with tool-eval-bench.
3. **sparkring rebase + MiMo 10-09 re-cuts** — cheap, existing lanes.
4. **Qwen3.8-Flash-Next 2x EP lane** — the coder-endpoint use case.
5. **DGPP bench** (Qwen w1, MiMo-RL w2) — cross-engine comparison.
6. Watch: veloGB10 TP4, dsgx10 repo, Ling-3.1-flash quants, TEB re-baseline.