Página Principal Explorar Ajuda
Iniciar Sessão
sm121
/
launcher
2
0
Fork 0
Ficheiros Problemas 0 Pull Requests 0 Wiki
Árvore: ef7298e1ee
Ramos Etiquetas
launcher
/
specs
/
tech-architecture
/
upstream-watch-2026-10-15.md

upstream-watch-2026-10-15.md 7.7 KB
Histórico Em bruto

Upstream watch — 2026-10-15

Delta pass over the same sources as upstream-watch-2026-10-14.md (one day, but the ecosystem is moving fast). Baseline changed since then: kindling 2x re-pinned to vLLM v0.31.0 (spark-glm53:v10, bench 95/100), gr8 + DGPP already deployed and quality-gated (89 / 88 def / 86 low), TEB pin now v2.8.0 (97 scenarios).

1. Engine stack

vLLM — v0.32.0 is imminent (the upgrade to plan)

  • Stable is still v0.31.0 (our base — current). v0.32.0rc2 cut 2026-10-15 09:50 UTC (7960eeb9de29); 0.31.1 skipped (rc0 only, 10-06).
  • What 0.32.0 carries that matters to GB10/our lanes (merged since 0.31.0):
    • #60948 FlashInfer → 0.7.1 (10-14) — the pairing we were waiting for; 0.7.1 has quantized expert-parallel MoE on SM12x, compact GLM53-NoPE caches, DS4.1 dual-cache sparse MLA (see 10-14 spec).
    • #46963 NVFP4 KV cache on SM12x via FlashInfer (10-07) + #60452 NVFP4 KV decode with XQA on SM12x (10-10) — KV capacity lever for the kindling lanes (NVFP4 KV ≈ 2× FP8 pool).
    • #59468 don't reserve DFlash/DSpark draft slots in the scheduler (10-15) — scheduler fix on our exact spec-decode path.
    • #59511 [Perf][DFlash] capture context combine + anchor in draft CUDA graph (10-14) — decode perf on our drafting path.
    • #61437 SM121 TP=2 QSA projection plans; #61274 auto-disable DeepGEMM on SM12x; #57378 NVML-memory fallback (UMA-adjacent); #58128 FP8 KV for MiMo-V2.6-Flash.
    • Structured-output hardening: #60359/#61047/#60748/#59061/#58862.
  • Action: when v0.32.0 stable lands (days), re-pin the kindling base v0.31.0 → v0.32.0 and rebase the 38 experimental overlays (delta is much smaller than the nightly→0.31.0 jump — same procedure, now checklist-ized in docker-images/glm53-kindling/AGENTS.md). Expect flashinfer 0.7.0.post1 → 0.7.1 pin bump too. Bench-gate vs the 95/100 anchor.

flashinfer

  • Stable 0.7.1 (10-09); 0.7.2 still rc3 (10-13). vLLM main pins 0.7.1.

lab (local-inference-lab) — r40 still pending, but watch the new channel

  • No new source.lock rev: vllm fork tip still ff292a9c63 (10-04), LMCache fork 09-14. Their flashinfer fork is where the action moved (b12x paged attention, DSA top-k, BF16-GEMM paths — commits 10-15).
  • New info source: rtx6kpro → daily-summaries/YYYY-MM/*.md — the lab publishes daily community/Discord digests (latest covers 10-14). Notable from the 10-14 issue: LIL stable build broke on b12x.loader._csf_source (fixed next pull); LMCache L1/L2 occasional misses force full prefill on GLM-5.3-Flash; dealignai UNCENSORED-NVFP4 needs *.self_attn.in_proj_qkvgfab added to quantization_config.ignore.

SparkRing 2026.10.4 (10-14) — validates the MiMo rebase, adds DCP2

  • Release image = dev-20261013-dcp2prefill-status038-sircl-tp6 (@sha256:e21ba936d4fd…) — exactly the image pre-pulled on dgx3/4/6/8 for the MiMo rebase. All 2026.10.2 profiles admitted unchanged.
  • New research-only profiles: glm53-flash-nvfp4-spark-tp2-dcp2 (GLM-5.3-Flash CSF on 2 Sparks with decode-context-parallel 2 — KV split across the pair, ~2× context, 16 concurrent) and glm53-nvfp4-csf-tp6 (full GLM-5.3 on 6 Sparks, not yet run on hardware). They use LIL CSF checkpoints (GLM-5.3-Flash-NVFP4-MXFP8-CSF-QAD, GLM-5.3-NVFP4-CSF, Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD) under the LIL license.
  • Action: proceed with the MiMo 4x rebase on this image (plan unchanged). DCP2 is a candidate datapoint for the 2x lane (context ceiling) — but it needs the CSF checkpoint + their toolchain, so treat as a separate experiment lane, not a kindling change.

net-sircl (new, via lab digest 10-14)

  • Plugin: NCCL between Sparks over direct cables (no switch) via a .so + topology file + 4 env vars + one fabric-tool NIC-forwarding setup. Relevant to 2nodes-dac — worth a fabric experiment when the pair is free.

2. Engines we've tested — deltas

  • gr8 (kindlingai): active daily — S2 scheduler (prefill quantum to the shortest prompt, notice-target sizing, cancel attribution), host-heap return after weight load (their analogue of our UMA/page-cache issue), boot-time drafting-cost overrides (#25, 10-15). Re-test after a quiet week.
  • DGPP (HawkBearPig): now actively adding Qwen3.8-27B — YaRN rope ramp + 768K template check, prefill-attention TMA rewrites (10-15). Direct A/B candidate for qwen38-27b-nvfp4-unsloth (our 21.7 t/s winner); their earlier claim: +40% over published vLLM on Qwen single-node.
  • TensorFold (new stack family, thread 384789 + MiaAI/gilesco repos): EXL3 GLM-5.3-Flash on 2× Spark with NVMe weight streaming + SSD cache offload; "1B tokens, several days, no issues" (ajvazan); MiaAI stack pins an old TensorFold, gilesco fork backports the latest; python branch frozen pending CUDA engine v1.0.10 (PR 804). Full GLM-5.3 (753B) 3-bit EXL3 TP4 @43 t/s code single-stream (thread 385042). Watch PR 804; not adoptable yet (frozen python branch, EXL3 quality untested by us).
  • TAI engine: 96/100 on current TEB (sparkdash standard settings, jcagle #840) — highest community quality report on the GLM-5.3 line; engine/repo not identified in-thread — track.

3. Fabric (CRS812 thread 385776 — resolved)

  • 185 Gbps achieved after rebooting with the cables connected (the regression was kernel 7.0.0 + cold-plugged ports); 176/200 after ethtool rate fix. 4 logical interfaces (separate subnets) only needed to max PCIe — with ≤4 nodes one interface per node is right.
  • Action: compare against our own fabric-check AG numbers; if a Spark pair was cabled while powered off, a reboot is the first fix.

4. Models

Model Note Action
NVIDIA Nemotron-3.5-Bolt-74B-A9B (10-15) 74B/A9B; official 24 GB GGUF fits ONE Spark; DSpark-GGUF drafts exist; "intended for 2-bit" cheap single-node test lane (A9B decode should be fast); quality unknown vs Qwen3.8-Flash-Next
Ling-3.1-flash (inclusionAI) on HF now; 572B total (bigger than the 10-14 estimate), AA ~41, MIT; day-0 cbfight/Ling-3.1-flash-NVFP4-lossless8 4x-lane candidate (NVFP4 ≈ GLM-5.3-Flash class size); watch for Spark recipes
GLM-5.3-Flash EXL3 wave turboderp 2.10bpw self-calibrated (2.5K dl), yamz-labs (+uncensored), cbert33 "DGX-Sliced", AxonZeta Focus 2.05bpw feeds TensorFold/veloGB10; quality-gate before any lane swap
LIL CSF checkpoints (3) GLM-5.3-Flash / GLM-5.3 / Qwen3.8-Flash-Next CSF variants for sparkring profiles LIL-licensed; only via sparkring profiles
MiMo-V2.6 re-cuts (10-09) unchanged since 10-14 spec covered by the rebase task

5. Harness

  • Community on current TEB: "high reasoning no longer enough for 90+" (re-baseline discipline matters; our anchors are v2.8.0/97-scenario).
  • TAI 96/100 and MiaAI C8 numbers are same-harness-comparable only if they stamp the version — ask before cross-comparing.

Recommended order (updated)

  1. MiMo 4x rebase — 2026.10.4 validates the pre-pulled image; do it when dgx3/4/6/8 free (they're off now).
  2. v0.32.0 stable → kindling base re-pin — flashinfer 0.7.1 + SM12x NVFP4 KV + DSpark scheduler/DFlash graph fixes in one bump; bench-gate vs 95/100.
  3. DGPP Qwen3.8-27B A/B vs qwen38-27b-nvfp4-unsloth (their 768K YaRN lands this week; single-node, cheap).
  4. Nemotron-3.5-Bolt 24GB single-Spark quick lane (new, free capacity).
  5. net-sircl DAC experiment on 2nodes-dac (fabric headroom check).
  6. Watch: lab r40 cut, TensorFold CUDA PR 804, TAI engine identity, Ling-3.1-flash Spark recipes, v0.32.0rc notes until stable.