spec: stock vLLM/SGLang vs ours for DeepSeek-V4.1-Flash-1017

Records where our pinned engines sit (prod SGLang base 2026-09-29 = ~3 weeks
behind v0.5.22; our vLLM lanes 0.26.1rc0 vs 0.32.0) and which upstream work
bears on the production model: decoder-side SWA bounded replay, NVFP4 KV cache
on SM12x (a 4x smaller page than fp8), fused multi-step draft decode, DCP/PCP,
HiSparse, the CUTLASS NVFP4/MXFP4 MoE corruption fix, and SGLang's DSpark +
HiSparse-with-spec batch.

Also profiles MiniMax-M3.1-Flash (252 GB mxfp8, 63 GB/rank at TP4 - it fits
four Sparks comfortably) and notes that FLARE upstream already ships a recipe
for it with packed DSpark speculation.

specs/tech-architecture/stock-stack-gaps-1017.md

@@ -0,0 +1,86 @@
+# Stock stacks vs ours — DeepSeek-V4.1-Flash-1017 (2026-10-23)
+
+Focus since 2026-10-23: **`deepseek-ai/DeepSeek-V4.1-Flash-1017`** — the
+production model. Everything else is secondary. This note records where our
+pinned engines sit against upstream and which upstream work matters for 1017
+on four Sparks.
+
+## Where we are
+
+| lane | engine build | upstream | gap |
+|---|---|---|---|
+| production `dsv41-flash-1017-4x-sglang` | `aidendle94/sparkrun-sglang-dsv41-gb10@sha256:f617afd8…` (2026-09-29), `sglang 0.0.0.dev1+gda64c5cbb`, torch 2.13+cu130 | SGLang **v0.5.22** (2026-10-19) | ~3 weeks of DSpark/HiSparse work |
+| `glm53-mxfp4-fp8-4x-r39` (our clone-at-pin) | `jovian-judgement:r39-glm53`, vLLM `0.26.1rc0+glm53.r39.vllm9e5d1793` | vLLM **0.32.0** (2026-10-18) | six minor releases, plus the fork's own b12x |
+| eugr nightly (staged on dgx5–7) | `eugr/spark-vllm-b12x:nightly-20261015`, vLLM main `8244d1c44` | vLLM main | essentially current |
+
+## What upstream added that bears on 1017
+
+**vLLM 0.32.0** (release notes, 2026-10-18):
+
+- **DeepSeek-V4 and V4.1: decoder-side SWA bounded replay** (#58132) — replay
+  layers captured into CUDA graphs and trimmed in PIECEWISE graphs
+  (`compilation_config`). This is the mechanism that lets a bounded-window model
+  replay long prefixes instead of recomputing them; it is exactly the
+  "long prefill is our pain" problem we hit on the GLM lane.
+- **NVFP4 KV cache on SM12x** (#46963) and with **XQA decode on SM12x**
+  (#60452): a 4× smaller KV page on GB10 than fp8 → more context per pin.
+  Worth probing on 1017 first: it changes the context/token economics more than
+  any kernel.
+- **Speculative decoding**: fused multi-step draft decode on FlashInfer
+  trtllm-gen (#58371), in-graph metric collection; GLM-5.3 gets sparse-MLA index
+  conversion reuse (#59464) and fused multi-step draft decode (#57443).
+- **Context parallelism**: decode requests sharded across PCP ranks (#52162),
+  block-interleaved DCP (#52162 area) — the DCP that our r39 fork refused with
+  `DCP not supported` may exist upstream now.
+- **HiSparse**: per-request residency cache (up to 58% at high concurrency,
+  #60083), lower scheduler overhead, GPU-resident contexts.
+- **Startup/memory**: shared-memory tensor arena on by default (#51207).
+- **Fixes**: silent CUTLASS NVFP4/MXFP4 MoE corruption under EP (#59042) — that
+  one is a correctness bug in the exact kernel family our mxfp4 lanes use.
+
+**SGLang v0.5.22** (2026-10-19):
+
+- **DeepSeek-V4.1: up to 2.3× more throughput per GPU**, plus DSpark on new
+  hardware; **DSpark support** (#41623) with a run of correctness fixes:
+  mask uncommitted slots before commit-inject cache lookups (#42624), fold draft
+  sampling for DSv4.1 heads (#42987), one step interface for markov heads
+  (#43620/#43621), top-p mask capture for DFlash/DSpark (#34201).
+- **HiSparse with speculative decoding** (multi-step KV swapping, #37771) —
+  long-context serving with the KV cache in CPU memory, which is a different
+  answer to our GB10 memory wall than TP8.
+- Hybrid-linear models no longer resume from wrong mamba states when spec +
+  structured output are combined (relevant to our agentic benchmarks).
+
+## FLARE (different runtime, same model)
+
+`NVIDIA/flare-local-inference` serves the same checkpoint from prepared CUDA
+graphs in a Rust runtime; first boot was ~2.4× slower on decode than our SGLang
+lane with nothing tuned. It moves several commits a day — see
+[`flare-bringup.md`](flare-bringup.md#upstream-watch-flare-moves-several-times-a-day)
+and `scripts/flare-upstream-watch.sh`. The 2026-10-23 batch added
+`Preset DeepSeek V4.1 packed GPU speculation`, `adaptive verification scoring
+alpha`, `resume preparation checkpoints`, `Serve and prepare over hosts cabled
+as a chain` and a **MiniMax-M3.1-Flash recipe with packed DSpark speculation**.
+
+## MiniMax-M3.1-Flash (new on the NAS)
+
+`MiniMaxAI/MiniMax-M3.1-Flash` — 252.4 GB, 72 files, downloaded 2026-10-23.
+`MiniMaxM3SparseForCausalLM` (plus a vision tower: `minimax_m3_vl`), 60 layers,
+hidden 6144, 64 heads, MoE top-4, 1M context, **mxfp8 weights (block [1,32])**.
+At TP4 that is ~63 GB per rank — comfortable on four Sparks, unlike the GLM-5.3
+full lane's 99 GiB. Candidate for the next "big model that actually fits" slot;
+FLARE upstream already has the recipe with packed DSpark speculation.
+
+## The queue that follows from this
+
+1. **1017 on stock vLLM 0.32** (image already staged): boot TP4, measure
+   pp/tg, then probe `--kv-cache-dtype nvfp4` on SM121 — the single biggest
+   potential context win.
+2. **SGLang 0.5.22 for the production lane**: our base is 2026-09-29; the DSpark
+   fixes since then are correctness fixes for our exact configuration.
+3. **b12x A4 prefill** into our clone-at-pin build (+10% prefill claimed, todo
+   #23) — prefill is what limits usable prompt length on GB10.
+4. **FLARE tuning on 1017** (scheduler policy, concurrency, adaptive
+   verification) to see whether the Rust runtime can close the 2.4× gap.
+5. **MiniMax-M3.1-Flash**: serve it once, measure, and only then compare it
+   with 1017 on quality.