Upstream watch — 2026-10-17 (night-shift research pass)

Pass across vLLM releases, the NVIDIA forums, and HF model cards, feeding the night-shift agenda (history/night-shifts/2026-10-17-night-plan.md).

vLLM release status — base stays v0.31.0

Latest release is still v0.31.0 (2026-10-05). v0.32.0 is only at rc2 (rc0/rc1/rc2 tags, no stable). No re-pin of the kindling base yet.
Keep watching for v0.32.0 stable (would trigger the base re-pin + overlay rebase + flashinfer bump, bench-gated vs the 95/100 kindling 2x anchor).

New NVIDIA-official FP4 checkpoints (all need the b12x container)

NVIDIA shipped a family of "…-FP4-Local" QAD checkpoints quantized with nvidia-modelopt v0.47.0 via quantization-aware distillation (QAD). Their NVFP4 scale factors use a new lossless-compression format that upstream vLLM does NOT yet support — you MUST run the eugr/spark-vllm-b12x container (b12x backend + --load-format b12x), not stock vLLM. Digest-pinned eugr/spark-vllm-b12x@sha256:03a95654…bb3595 (arm64, built 2026-10-15).

nvidia/GLM-5.3-Flash-FP4-Local — official NVFP4 GLM-5.3-Flash. 2x DGX Spark, 500K ctx, MTP n=5 (moe_backend:humming, attention_backend:B12X), FP8 KV (~2M tokens), --attention-backend B12X --block-size 256 --moe/--linear/--load-format b12x, --reasoning-parser glm45 --tool-call-parser glm47, GMU 0.87. Card quality 91.3% / 92.4%. This is the official NVFP4 counterpart to our kindling 2x lane (our own r35 build) — a direct A/B vs the 95/100 anchor. Recipe: glm53-flash-fp4-local-2x (recipes d5276db). NOTE: 2x → needs 2 free Sparks; 2nodes-dac is busy with the live kindling serve.
nvidia/Qwen3.8-Flash-Next-FP4-Local — 1x Spark, b12x, MTP n=4, FP8 KV, 262K, GMU 0.8, VLLM_ENGINE_READY_TIMEOUT_S=3600 (b12x autotune > 600 s). Forum hardmode 89/100, ~49 t/s decode. Recipe qwen38-flash-next-fp4-local.
nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 (75 GB, 1x Spark, VL, 1M ctx, LiLiCorr dflash-7 drafter, mamba flags, qwen3_coder/nemotron_v3) and Nemotron-3.5-Bolt-74B-A9B-24GB (24 GB, QAD 2-bit experts, 1x Spark, ultra_v3 parser plugin). Recipes nemotron-35-super-vl-120b-nvfp4, nemotron-35-bolt-74b-24gb. Forum: Super-VL 90/100, Bolt 88/100.

GB10 hardware gotchas (both directly bit us this session)

Stuck GPU clock (forum 386060, confirms 376039/376239/366944/…): GB10 SM clock can stick at ~660 MHz (max 3003, app clock 2418) at ~16 W with throttle reasons 0x0; nvidia-smi -lgc is accepted but ignored and a reboot does not clear it. Symptom = anomalously low throughput that looks like "hot state" but is a hard clock cap. Remedy (community, not NVIDIA- documented): power off, unplug AC + USB, hold the power button ~10 s, wait a few minutes, reconnect (one report cleared to 2080 MHz / 77 TFLOPS). → When a lane underperforms, check SM clock under load (expect ~2000–2400, not ~660) before blaming the recipe; a smart-plug cycle may not be enough (needs the hold-power-button drain).
GPU/unified memory is NOT cgroup-contained on GB10 (forum 386057): with the 580 driver nvidia-smi reports Memory-Usage: Not Supported on the iGPU, and docker --memory / MemoryMax= do not cap cudaMalloc/unified memory. When the unified pool is exhausted the whole host stalls (not just the container). This is exactly why the Ling fp4 converter OOM-killed on dgx1 while the GLM-5.3 kindling 2x serve held ~119/121 GB. → Never run a heavy host job (conversion, big load) on a node that has a live serve. Use a clean node (we moved to dgx5, 117 GB free). The reporter backported 615's memacct to 580 for per-cgroup GPU-memory accounting (not adopted; noted).

Other forum items (noted, not lanes)

Muse Spark Open 816B on 2 Sparks (386058): llama.cpp RPC over 200 GbE (RoCEv2), UD-IQ1_M 194.6 GiB / 6 shards, 32K ctx, q8_0 K/V, 6.8–7.0 tok/s single request. Two nodes is the floor (one can't hold weights; a third buys only context). Batch-only, llama.cpp stack — not a vLLM lane.
DGPP thread 383406 at 108 posts (1 new since the 10-16 pass; no material change). Our pin 3ad63d6b…, HEAD observed d9311e72… — bump pending a supervised re-bench.

Ling-3.1-flash fp8→fp4 (our own converter — in progress)

No official 3.1-flash fp4 exists (inclusionAI has fp4 only for 3.0). We wrote a converter (fp8-block routed experts → MXFP4, "even" scale mode, byte-exact vs vLLM reference_mxfp4.py dq). Prototype validated (my dq == reference dq, relerr 0.0000; ~10.8% mxfp4 loss vs bf16; experts ~53% of fp8 bytes). Full 54-shard conversion running on dgx5 → NAS models--inclusionAI--Ling-3.1-flash-fp4. Target ~110 GB → one Spark (vs 534 GB fp8 → 8). Serve + bench next. Format + math spec: specs/tech-architecture/upstream-watch-2026-10-16.md ("making 3.1-flash smaller").

upstream-watch-2026-10-17.md 4.9 KB 履歴 Raw