|
|
@@ -0,0 +1,126 @@
|
|
|
+# Upstream watch — 2026-10-15
|
|
|
+
|
|
|
+Delta pass over the same sources as [`upstream-watch-2026-10-14.md`](upstream-watch-2026-10-14.md)
|
|
|
+(one day, but the ecosystem is moving fast). Baseline changed since then:
|
|
|
+kindling 2x re-pinned to **vLLM v0.31.0** (`spark-glm53:v10`, bench 95/100),
|
|
|
+gr8 + DGPP already deployed and quality-gated (89 / 88 def / 86 low),
|
|
|
+TEB pin now **v2.8.0** (97 scenarios).
|
|
|
+
|
|
|
+## 1. Engine stack
|
|
|
+
|
|
|
+### vLLM — v0.32.0 is imminent (the upgrade to plan)
|
|
|
+- Stable is still **v0.31.0** (our base — current). `v0.32.0rc2` cut
|
|
|
+ **2026-10-15 09:50 UTC** (`7960eeb9de29`); 0.31.1 skipped (rc0 only, 10-06).
|
|
|
+- What 0.32.0 carries that matters to GB10/our lanes (merged since 0.31.0):
|
|
|
+ - **#60948 FlashInfer → 0.7.1** (10-14) — the pairing we were waiting for;
|
|
|
+ 0.7.1 has quantized expert-parallel MoE on SM12x, compact GLM53-NoPE
|
|
|
+ caches, DS4.1 dual-cache sparse MLA (see 10-14 spec).
|
|
|
+ - **#46963 NVFP4 KV cache on SM12x via FlashInfer** (10-07) +
|
|
|
+ **#60452 NVFP4 KV decode with XQA on SM12x** (10-10) — KV capacity lever
|
|
|
+ for the kindling lanes (NVFP4 KV ≈ 2× FP8 pool).
|
|
|
+ - **#59468 don't reserve DFlash/DSpark draft slots in the scheduler** (10-15)
|
|
|
+ — scheduler fix on our exact spec-decode path.
|
|
|
+ - **#59511 [Perf][DFlash] capture context combine + anchor in draft CUDA
|
|
|
+ graph** (10-14) — decode perf on our drafting path.
|
|
|
+ - #61437 SM121 TP=2 QSA projection plans; #61274 auto-disable DeepGEMM on
|
|
|
+ SM12x; #57378 NVML-memory fallback (UMA-adjacent); #58128 FP8 KV for
|
|
|
+ MiMo-V2.6-Flash.
|
|
|
+ - Structured-output hardening: #60359/#61047/#60748/#59061/#58862.
|
|
|
+- **Action:** when v0.32.0 stable lands (days), re-pin the kindling base
|
|
|
+ v0.31.0 → v0.32.0 and rebase the 38 experimental overlays (delta is much
|
|
|
+ smaller than the nightly→0.31.0 jump — same procedure, now checklist-ized
|
|
|
+ in `docker-images/glm53-kindling/AGENTS.md`). Expect flashinfer
|
|
|
+ 0.7.0.post1 → 0.7.1 pin bump too. Bench-gate vs the 95/100 anchor.
|
|
|
+
|
|
|
+### flashinfer
|
|
|
+- Stable **0.7.1** (10-09); 0.7.2 still rc3 (10-13). vLLM main pins 0.7.1.
|
|
|
+
|
|
|
+### lab (local-inference-lab) — r40 still pending, but watch the new channel
|
|
|
+- No new source.lock rev: vllm fork tip still `ff292a9c63` (10-04), LMCache
|
|
|
+ fork 09-14. Their flashinfer fork is where the action moved (b12x paged
|
|
|
+ attention, DSA top-k, BF16-GEMM paths — commits 10-15).
|
|
|
+- **New info source: `rtx6kpro` → `daily-summaries/YYYY-MM/*.md`** — the lab
|
|
|
+ publishes daily community/Discord digests (latest covers 10-14). Notable
|
|
|
+ from the 10-14 issue: LIL stable build broke on `b12x.loader._csf_source`
|
|
|
+ (fixed next pull); LMCache L1/L2 occasional misses force full prefill on
|
|
|
+ GLM-5.3-Flash; dealignai UNCENSORED-NVFP4 needs
|
|
|
+ `*.self_attn.in_proj_qkvgfab` added to `quantization_config.ignore`.
|
|
|
+
|
|
|
+### SparkRing 2026.10.4 (10-14) — validates the MiMo rebase, adds DCP2
|
|
|
+- Release image = `dev-20261013-dcp2prefill-status038-sircl-tp6`
|
|
|
+ (`@sha256:e21ba936d4fd…`) — **exactly the image pre-pulled on dgx3/4/6/8**
|
|
|
+ for the MiMo rebase. All 2026.10.2 profiles admitted unchanged.
|
|
|
+- New research-only profiles: **`glm53-flash-nvfp4-spark-tp2-dcp2`** (GLM-5.3-Flash
|
|
|
+ CSF on 2 Sparks with decode-context-parallel 2 — KV split across the pair,
|
|
|
+ ~2× context, 16 concurrent) and `glm53-nvfp4-csf-tp6` (full GLM-5.3 on 6
|
|
|
+ Sparks, not yet run on hardware). They use LIL **CSF checkpoints**
|
|
|
+ (`GLM-5.3-Flash-NVFP4-MXFP8-CSF-QAD`, `GLM-5.3-NVFP4-CSF`,
|
|
|
+ `Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD`) under the LIL license.
|
|
|
+- **Action:** proceed with the MiMo 4x rebase on this image (plan unchanged).
|
|
|
+ DCP2 is a candidate datapoint for the 2x lane (context ceiling) — but it
|
|
|
+ needs the CSF checkpoint + their toolchain, so treat as a separate
|
|
|
+ experiment lane, not a kindling change.
|
|
|
+
|
|
|
+### net-sircl (new, via lab digest 10-14)
|
|
|
+- Plugin: NCCL between Sparks over **direct cables** (no switch) via a .so +
|
|
|
+ topology file + 4 env vars + one fabric-tool NIC-forwarding setup.
|
|
|
+ Relevant to `2nodes-dac` — worth a fabric experiment when the pair is free.
|
|
|
+
|
|
|
+## 2. Engines we've tested — deltas
|
|
|
+
|
|
|
+- **gr8** (kindlingai): active daily — S2 scheduler (prefill quantum to the
|
|
|
+ shortest prompt, notice-target sizing, cancel attribution), **host-heap
|
|
|
+ return after weight load** (their analogue of our UMA/page-cache issue),
|
|
|
+ boot-time drafting-cost overrides (#25, 10-15). Re-test after a quiet week.
|
|
|
+- **DGPP** (HawkBearPig): now actively adding **Qwen3.8-27B** — YaRN rope
|
|
|
+ ramp + 768K template check, prefill-attention TMA rewrites (10-15).
|
|
|
+ Direct A/B candidate for `qwen38-27b-nvfp4-unsloth` (our 21.7 t/s winner);
|
|
|
+ their earlier claim: +40% over published vLLM on Qwen single-node.
|
|
|
+- **TensorFold** (new stack family, thread 384789 + MiaAI/gilesco repos):
|
|
|
+ EXL3 GLM-5.3-Flash on 2× Spark with NVMe weight streaming + SSD cache
|
|
|
+ offload; "1B tokens, several days, no issues" (ajvazan); MiaAI stack pins
|
|
|
+ an old TensorFold, gilesco fork backports the latest; python branch frozen
|
|
|
+ pending CUDA engine v1.0.10 (PR 804). Full GLM-5.3 (753B) 3-bit EXL3 TP4
|
|
|
+ @43 t/s code single-stream (thread 385042). Watch PR 804; not adoptable
|
|
|
+ yet (frozen python branch, EXL3 quality untested by us).
|
|
|
+- **TAI engine**: 96/100 on current TEB (sparkdash standard settings,
|
|
|
+ jcagle #840) — highest community quality report on the GLM-5.3 line;
|
|
|
+ engine/repo not identified in-thread — track.
|
|
|
+
|
|
|
+## 3. Fabric (CRS812 thread 385776 — resolved)
|
|
|
+- 185 Gbps achieved after **rebooting with the cables connected** (the
|
|
|
+ regression was kernel 7.0.0 + cold-plugged ports); 176/200 after
|
|
|
+ `ethtool` rate fix. 4 logical interfaces (separate subnets) only needed to
|
|
|
+ max PCIe — with ≤4 nodes one interface per node is right.
|
|
|
+- **Action:** compare against our own fabric-check AG numbers; if a Spark
|
|
|
+ pair was cabled while powered off, a reboot is the first fix.
|
|
|
+
|
|
|
+## 4. Models
|
|
|
+
|
|
|
+| Model | Note | Action |
|
|
|
+|---|---|---|
|
|
|
+| **NVIDIA Nemotron-3.5-Bolt-74B-A9B** (10-15) | 74B/A9B; official **24 GB GGUF** fits ONE Spark; DSpark-GGUF drafts exist; "intended for 2-bit" | cheap single-node test lane (A9B decode should be fast); quality unknown vs Qwen3.8-Flash-Next |
|
|
|
+| **Ling-3.1-flash** (inclusionAI) | on HF now; **572B** total (bigger than the 10-14 estimate), AA ~41, MIT; day-0 `cbfight/Ling-3.1-flash-NVFP4-lossless8` | 4x-lane candidate (NVFP4 ≈ GLM-5.3-Flash class size); watch for Spark recipes |
|
|
|
+| GLM-5.3-Flash **EXL3 wave** | turboderp 2.10bpw self-calibrated (2.5K dl), yamz-labs (+uncensored), cbert33 "DGX-Sliced", AxonZeta Focus 2.05bpw | feeds TensorFold/veloGB10; quality-gate before any lane swap |
|
|
|
+| LIL **CSF checkpoints** (3) | GLM-5.3-Flash / GLM-5.3 / Qwen3.8-Flash-Next CSF variants for sparkring profiles | LIL-licensed; only via sparkring profiles |
|
|
|
+| MiMo-V2.6 re-cuts (10-09) | unchanged since 10-14 spec | covered by the rebase task |
|
|
|
+
|
|
|
+## 5. Harness
|
|
|
+- Community on current TEB: "high reasoning no longer enough for 90+"
|
|
|
+ (re-baseline discipline matters; our anchors are v2.8.0/97-scenario).
|
|
|
+- TAI 96/100 and MiaAI C8 numbers are same-harness-comparable only if they
|
|
|
+ stamp the version — ask before cross-comparing.
|
|
|
+
|
|
|
+## Recommended order (updated)
|
|
|
+
|
|
|
+1. **MiMo 4x rebase** — 2026.10.4 validates the pre-pulled image; do it when
|
|
|
+ dgx3/4/6/8 free (they're off now).
|
|
|
+2. **v0.32.0 stable → kindling base re-pin** — flashinfer 0.7.1 + SM12x
|
|
|
+ NVFP4 KV + DSpark scheduler/DFlash graph fixes in one bump; bench-gate
|
|
|
+ vs 95/100.
|
|
|
+3. **DGPP Qwen3.8-27B A/B** vs `qwen38-27b-nvfp4-unsloth` (their 768K YaRN
|
|
|
+ lands this week; single-node, cheap).
|
|
|
+4. **Nemotron-3.5-Bolt 24GB** single-Spark quick lane (new, free capacity).
|
|
|
+5. **net-sircl** DAC experiment on `2nodes-dac` (fabric headroom check).
|
|
|
+6. Watch: lab r40 cut, TensorFold CUDA PR 804, TAI engine identity,
|
|
|
+ Ling-3.1-flash Spark recipes, v0.32.0rc notes until stable.
|
