# Stock stacks vs ours — DeepSeek-V4.1-Flash-1017 (2026-10-23)

Focus since 2026-10-23: **`deepseek-ai/DeepSeek-V4.1-Flash-1017`** — the
production model. Everything else is secondary. This note records where our
pinned engines sit against upstream and which upstream work matters for 1017
on four Sparks.

## Where we are

| lane | engine build | upstream | gap |
|---|---|---|---|
| production `dsv41-flash-1017-4x-sglang` | `aidendle94/sparkrun-sglang-dsv41-gb10@sha256:f617afd8…` (2026-09-29), `sglang 0.0.0.dev1+gda64c5cbb`, torch 2.13+cu130 | SGLang **v0.5.22** (2026-10-19) | ~3 weeks of DSpark/HiSparse work |
| `glm53-mxfp4-fp8-4x-r39` (our clone-at-pin) | `jovian-judgement:r39-glm53`, vLLM `0.26.1rc0+glm53.r39.vllm9e5d1793` | vLLM **0.32.0** (2026-10-18) | six minor releases, plus the fork's own b12x |
| eugr nightly (staged on dgx5–7) | `eugr/spark-vllm-b12x:nightly-20261015`, vLLM main `8244d1c44` | vLLM main | essentially current |

## What upstream added that bears on 1017

**vLLM 0.32.0** (release notes, 2026-10-18):

- **DeepSeek-V4 and V4.1: decoder-side SWA bounded replay** (#58132) — replay
  layers captured into CUDA graphs and trimmed in PIECEWISE graphs
  (`compilation_config`). This is the mechanism that lets a bounded-window model
  replay long prefixes instead of recomputing them; it is exactly the
  "long prefill is our pain" problem we hit on the GLM lane.
- **NVFP4 KV cache on SM12x** (#46963) and with **XQA decode on SM12x**
  (#60452): a 4× smaller KV page on GB10 than fp8 → more context per pin.
  Worth probing on 1017 first: it changes the context/token economics more than
  any kernel.
- **Speculative decoding**: fused multi-step draft decode on FlashInfer
  trtllm-gen (#58371), in-graph metric collection; GLM-5.3 gets sparse-MLA index
  conversion reuse (#59464) and fused multi-step draft decode (#57443).
- **Context parallelism**: decode requests sharded across PCP ranks (#52162),
  block-interleaved DCP (#52162 area) — the DCP that our r39 fork refused with
  `DCP not supported` may exist upstream now.
- **HiSparse**: per-request residency cache (up to 58% at high concurrency,
  #60083), lower scheduler overhead, GPU-resident contexts.
- **Startup/memory**: shared-memory tensor arena on by default (#51207).
- **Fixes**: silent CUTLASS NVFP4/MXFP4 MoE corruption under EP (#59042) — that
  one is a correctness bug in the exact kernel family our mxfp4 lanes use.

**SGLang v0.5.22** (2026-10-19):

- **DeepSeek-V4.1: up to 2.3× more throughput per GPU**, plus DSpark on new
  hardware; **DSpark support** (#41623) with a run of correctness fixes:
  mask uncommitted slots before commit-inject cache lookups (#42624), fold draft
  sampling for DSv4.1 heads (#42987), one step interface for markov heads
  (#43620/#43621), top-p mask capture for DFlash/DSpark (#34201).
- **HiSparse with speculative decoding** (multi-step KV swapping, #37771) —
  long-context serving with the KV cache in CPU memory, which is a different
  answer to our GB10 memory wall than TP8.
- Hybrid-linear models no longer resume from wrong mamba states when spec +
  structured output are combined (relevant to our agentic benchmarks).

## FLARE (different runtime, same model)

`NVIDIA/flare-local-inference` serves the same checkpoint from prepared CUDA
graphs in a Rust runtime; first boot was ~2.4× slower on decode than our SGLang
lane with nothing tuned. It moves several commits a day — see
[`flare-bringup.md`](flare-bringup.md#upstream-watch-flare-moves-several-times-a-day)
and `scripts/flare-upstream-watch.sh`. The 2026-10-23 batch added
`Preset DeepSeek V4.1 packed GPU speculation`, `adaptive verification scoring
alpha`, `resume preparation checkpoints`, `Serve and prepare over hosts cabled
as a chain` and a **MiniMax-M3.1-Flash recipe with packed DSpark speculation**.

## MiniMax-M3.1-Flash (new on the NAS)

`MiniMaxAI/MiniMax-M3.1-Flash` — 252.4 GB, 72 files, downloaded 2026-10-23.
`MiniMaxM3SparseForCausalLM` (plus a vision tower: `minimax_m3_vl`), 60 layers,
hidden 6144, 64 heads, MoE top-4, 1M context, **mxfp8 weights (block [1,32])**.
At TP4 that is ~63 GB per rank — comfortable on four Sparks, unlike the GLM-5.3
full lane's 99 GiB. Candidate for the next "big model that actually fits" slot;
FLARE upstream already has the recipe with packed DSpark speculation.

## The queue that follows from this

1. **1017 on stock vLLM 0.32** (image already staged): boot TP4, measure
   pp/tg, then probe `--kv-cache-dtype nvfp4` on SM121 — the single biggest
   potential context win.
2. **SGLang 0.5.22 for the production lane**: our base is 2026-09-29; the DSpark
   fixes since then are correctness fixes for our exact configuration.
3. **b12x A4 prefill** into our clone-at-pin build (+10% prefill claimed, todo
   #23) — prefill is what limits usable prompt length on GB10.
4. **FLARE tuning on 1017** (scheduler policy, concurrency, adaptive
   verification) to see whether the Rust runtime can close the 2.4× gap.
5. **MiniMax-M3.1-Flash**: serve it once, measure, and only then compare it
   with 1017 on quality.