dsv41-flash-1017-4x-vllm032: stock vLLM baseline for 1017 — 29.7 t/s c1, 45.6 c2, prefill ~750

Same checkpoint and four Sparks as the production SGLang lane (47.8-50.7 c1,
64.2-68.3 c2, pp 2.8-3.1k). Un-speculated and untuned vLLM main sits at ~60-70%
of prod decode and a quarter of its prefill; KV pool 8.73M tokens at a 131072
window with fp8 KV and engram tables in disk mode.

Two boot notes recorded: the image's InstantTensor lacks
_determine_buffer_size (use the default loader), and the engram disk config is
accepted as-is.

recipes/dsv41-flash-1017-4x-vllm032/NOTE.md

@@ -0,0 +1,32 @@
+# DeepSeek-V4.1-Flash-1017 on stock-era vLLM (eugr nightly 20261015)
+
+Purpose: the vLLM side of the "how far has the stock stack come on OUR model"
+question (`specs/tech-architecture/stock-stack-gaps-1017.md`). Same checkpoint,
+same four Sparks (dgx5–8, TP4), same cluster as the production SGLang lane.
+
+## Baseline, no speculation (2026-10-23)
+
+Boot: 476 GB checkpoint streamed from the NAS with the plain safetensors
+loader (~10 min), `GPU KV cache size: 8,726,823` tokens at a 131072 window
+(fp8 KV, `--gpu-memory-utilization 0.88`, engram tables in `disk` mode).
+
+| measure | stock vLLM (vLLM main 8244d1c44) | production SGLang EP2 4x |
+|---|---|---|
+| decode c1 | **29.7 t/s** (20.0 on the first, cold turn) | 47.8–50.7 t/s |
+| decode c2 (aggregate) | **45.6 t/s** | 64.2–68.3 t/s |
+| prefill (~4K prompt) | **~750 t/s** | 2,800–3,127 t/s |
+| KV pool | 8.73M tokens @131072 | — |
+
+So the engine alone, un-speculated and untuned, lands at ~60–70% of the
+production lane's decode and ~25% of its prefill. Production runs DSpark K5
+speculation on a tuned SGLang build; the next boot adds DSpark to this lane to
+see how much of the gap is speculation and how much is the engine.
+
+## Boot notes
+
+- `--load-format instanttensor` fails on this image
+  (`Installed InstantTensor does not expose the metadata layout needed for
+  index-aware loading (missing: _determine_buffer_size)`) — use the default
+  loader.
+- `--engram-config '{"cpu_offload":false,"table_memory":"disk"}'` is accepted
+  and the engine starts with the tables streamed from the checkpoint.

recipes/dsv41-flash-1017-4x-vllm032/recipe.toml

@@ -0,0 +1,74 @@
+# DeepSeek-V4.1-Flash-1017 on a STOCK-era vLLM (eugr nightly, vLLM main), 4x TP4.
+#
+# Purpose: answer "how far has stock vLLM come on OUR model?" — the production
+# lane is SGLang from a 2026-09-29 base, our vLLM lanes are a 0.26.1rc0 fork,
+# and upstream vLLM is 0.32.0 (see specs/tech-architecture/stock-stack-gaps-1017.md).
+# This lane is the vLLM side of that A/B: same checkpoint, same four Sparks,
+# same cluster as `dsv41-flash-1017-4x-sglang`.
+#
+# Image: `eugr/spark-vllm-b12x:nightly-20261015` — vLLM main `8244d1c44`, a
+# GB10/arm64 build (already on all four nodes). Verified by inspection:
+# registers DeepseekV41/GlmMoeDsa/Glm53DSpark, compressed-tensors 0.17.0 with
+# the mxfp4/mxfp8 schemes, and ships the mxfp4 linear/MoE kernel families.
+# It carries no sm121.checksum label, so `serve` warns STALE — expected for a
+# prebuilt pull, not a build we own.
+#
+# Deliberately minimal first boot: no speculation, no B12X* backend pins, no
+# engram override beyond `disk` — every one of those is a lab-fork-specific flag
+# whose presence in this image we have not verified, and each is a 25-minute
+# reload if wrong. Measure the baseline, then add DSpark k=5.
+#
+# Checkpoint geometry (config.json): DeepseekV41ForCausalLM, 40 layers,
+# hidden 5120, MTP heads in-checkpoint (num_nextn_predict_layers=3), 1M native
+# window, fp8 block [32,32]; 48 shards, 476 GB on the NAS including the Engram
+# tables.
+
+engine = "vllm"
+name = "DeepSeek-V4.1-Flash-1017 on vLLM main (eugr nightly 20261015), 4x"
+docker_image = "eugr/spark-vllm-b12x:nightly-20261015"  # prebuilt (no docker_image_dir)
+model = "deepseek-ai/DeepSeek-V4.1-Flash-1017"
+parent_model = "deepseek-ai/DeepSeek-V4.1-Flash"
+required_nodes = 4
+tensor_parallel_size = 4
+pipeline_parallel_size = 1
+distributed_backend = "mp"
+shm_size = "32gb"
+cluster = "dgx58-switch"
+entrypoint = ["vllm", "serve"]
+
+args = [
+    "--served-model-name", "dsv41",
+    "--trust-remote-code",
+    "--distributed-executor-backend", "mp",
+    "--gpu-memory-utilization", "0.88",
+    # 131072 first: the production lane runs 524288 behind SGLang's engram
+    # staging, but a vLLM boot that OOMs teaches us nothing. Raise after the
+    # baseline numbers exist.
+    "--max-model-len", "131072",
+    "--max-num-seqs", "4",
+    "--max-num-batched-tokens", "4096",
+    "--enable-chunked-prefill",
+    "--enable-prefix-caching",
+    "--kv-cache-dtype", "fp8",
+    "--block-size", "256",
+    # Engram: the 1017's learned n-gram tables live in the checkpoint; disk
+    # mode streams rows instead of pinning them (the lab fork's flag).
+    "--engram-config", """
+'{"cpu_offload":false,"table_memory":"disk"}'""",
+    # 2026-10-23 13:58: the image's InstantTensor lacks
+    # `_determine_buffer_size` (index-aware loading), so the fast loader is out
+    # and the default safetensors path is used — slower load, same result.
+    # "--load-format", "instanttensor",
+    "--generation-config", "vllm",
+    "--reasoning-parser", "deepseek_v41",
+    "--tool-call-parser", "deepseek_v41",
+    "--enable-auto-tool-choice",
+    # No --speculative-config on the first boot: MTP heads are in the
+    # checkpoint, add {"method":"dspark","num_speculative_tokens":5} after a
+    # clean baseline.
+]
+
+[env]
+HF_HUB_OFFLINE = "1"
+VLLM_ENGINE_READY_TIMEOUT_S = "5400"
+MAX_JOBS = "2"