1017 vllm032: engram staged locally via [[staging]] local-view — prefill 750 -> 3,957 t/s

The Engram tables are read at ~1 MB/token and live in two 101.5 GB shards.
Serving them from the shared NAS held this lane's prefill at 750 t/s while
production ran 2,800-3,100; staging the two shards on local NVMe lifted prefill
to 3,957 t/s (unique 12.6K prompt) and dropped shared-mount reads to 0 MB
during a run that used to pull 258 MB per short completion. Decode c1 52.9 ->
59.7 t/s too.

The recipe declares the staging (kind=local-view); serve measures free space on
every node first, builds the view (real copies + symlinks for the rest), mounts
it at /root/.cache/hf-view and rewrites the model path. Verified by content
hashes, so later serves are verification-only.

recipes/dsv41-flash-1017-4x-vllm032/NOTE.md

@@ -53,3 +53,40 @@ Next levers for the prefill side, in order of cheapness: `--max-num-batched-toke
 (4096 now — SGLang chunks larger), the lab fork's `--prefill-compute-share`, and
 the b12x A4-prefill patch (todo #23) if `B12X_W4A16_MXFP4_PREFILL_MIN_TOKENS`
 exists in this image.
+
+## Engram locality (2026-10-23) — the prefill gap was I/O, not the engine
+
+The Engram n-gram tables are read **at ~1 MB per generated token** and live in
+exactly two shards (`model-00047/48-of-00048.safetensors`, 101.5 GB each).
+Serving them off the shared NAS cost the lane its prefill:
+
+| setup | prefill (12.6K unique prompt) | shared-NFS reads during a run |
+|---|---|---|
+| Engram on the NAS | 750 t/s | +258 MB per short completion |
+| Engram staged locally | **3,957 t/s** | **0 MB** |
+
+Decode c1 also improved (52.9 → 59.7 t/s). Production's 2,800–3,100 t/s was
+mostly *this*, not a different engine stack.
+
+The recipe now declares it, and `serve` does the rest:
+
+```toml
+[[staging]]
+kind = "local-view"
+repo = "deepseek-ai/DeepSeek-V4.1-Flash-1017"
+dst = "/home/kilork/models/dsv41-1017-local"
+local = ["model-00047-of-00048.safetensors", "model-00048-of-00048.safetensors"]
+min_free = "10G"
+```
+
+`serve` measures every launch node first (need vs free, and refuses before
+starting anything if one is short), then builds the view: real copies for the
+two engram shards, symlinks into the shared cache for the other 46 files, and
+mounts it at `/root/.cache/hf-view` (the model path is rewritten to it). The
+view is verified by sampled content hashes, so the second serve is
+verification-only, and a tree already rsync'd by hand is adopted.
+
+Notes for the next lane: 190 GB per node is the cost — check `df -h /home`
+first (dgx7 sits at ~49 GB free with the shards in place, so it is the binding
+node). First staging pass over the NAS took ~8 min per node at ~470 MB/s
+(fabric, node-to-node).

recipes/dsv41-flash-1017-4x-vllm032/recipe.toml

@@ -72,4 +72,17 @@ args = [
 [env]
 HF_HUB_OFFLINE = "1"
 VLLM_ENGINE_READY_TIMEOUT_S = "5400"
-MAX_JOBS = "2"
+MAX_JOBS = "2"
+
+# Engram tables are read at ~1 MB per TOKEN (measured 2026-10-23: one 250-token
+# completion pulled 258 MB off the shared NFS mount). They live in exactly two
+# shards, so those two are staged to local NVMe on every launch node and the
+# rest of the 476 GB stays on shared storage — the launcher builds the view,
+# verifies it by content hash and refuses to start if a node lacks the space.
+[[staging]]
+kind = "local-view"
+repo = "deepseek-ai/DeepSeek-V4.1-Flash-1017"
+dst = "/home/kilork/models/dsv41-1017-local"
+local = ["model-00047-of-00048.safetensors", "model-00048-of-00048.safetensors"]
+min_free = "10G"
+verify = "sample-hash"

recipes/dsv41-flash-1017-4x-vllm032/reports/2026-10-23_143823_1017-vllm032-dspark5.log

@@ -0,0 +1,40 @@
+^D{"event": "backend_detected", "backend": "vllm"}
+{"event": "scenario_start", "scenario_id": "TC-01", "title": "Direct Specialist Match", "category": "A", "index": 0, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-01", "status": "pass", "points": 2, "index": 0, "total": 97, "duration_seconds": 20.29}
+{"event": "scenario_start", "scenario_id": "TC-02", "title": "Distractor Resistance", "category": "A", "index": 1, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-02", "status": "pass", "points": 2, "index": 1, "total": 97, "duration_seconds": 22.09}
+{"event": "scenario_start", "scenario_id": "TC-03", "title": "Implicit Tool Need", "category": "A", "index": 2, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-03", "status": "pass", "points": 2, "index": 2, "total": 97, "duration_seconds": 24.05}
+{"event": "scenario_start", "scenario_id": "TC-04", "title": "Unit Handling", "category": "B", "index": 3, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-04", "status": "pass", "points": 2, "index": 3, "total": 97, "duration_seconds": 5.81}
+{"event": "scenario_start", "scenario_id": "TC-05", "title": "Date and Time Parsing", "category": "B", "index": 4, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-05", "status": "pass", "points": 2, "index": 4, "total": 97, "duration_seconds": 13.26}
+{"event": "scenario_start", "scenario_id": "TC-06", "title": "Multi-Value Extraction", "category": "B", "index": 5, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-06", "status": "pass", "points": 2, "index": 5, "total": 97, "duration_seconds": 9.1}
+{"event": "scenario_start", "scenario_id": "TC-07", "title": "Search \u2192 Read \u2192 Act", "category": "C", "index": 6, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-07", "status": "pass", "points": 2, "index": 6, "total": 97, "duration_seconds": 19.51}
+{"event": "scenario_start", "scenario_id": "TC-08", "title": "Conditional Branching", "category": "C", "index": 7, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-08", "status": "pass", "points": 2, "index": 7, "total": 97, "duration_seconds": 11.94}
+{"event": "scenario_start", "scenario_id": "TC-09", "title": "Parallel Independence", "category": "C", "index": 8, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-09", "status": "pass", "points": 2, "index": 8, "total": 97, "duration_seconds": 11.29}
+{"event": "scenario_start", "scenario_id": "TC-10", "title": "Trivial Knowledge", "category": "D", "index": 9, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-10", "status": "pass", "points": 2, "index": 9, "total": 97, "duration_seconds": 2.59}
+{"event": "scenario_start", "scenario_id": "TC-11", "title": "Simple Math", "category": "D", "index": 10, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-11", "status": "pass", "points": 2, "index": 10, "total": 97, "duration_seconds": 2.12}
+{"event": "scenario_start", "scenario_id": "TC-12", "title": "Impossible Request", "category": "D", "index": 11, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-12", "status": "pass", "points": 2, "index": 11, "total": 97, "duration_seconds": 11.16}
+{"event": "scenario_start", "scenario_id": "TC-13", "title": "Empty Results", "category": "E", "index": 12, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-13", "status": "pass", "points": 2, "index": 12, "total": 97, "duration_seconds": 17.15}
+{"event": "scenario_start", "scenario_id": "TC-14", "title": "Malformed Response", "category": "E", "index": 13, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-14", "status": "pass", "points": 2, "index": 13, "total": 97, "duration_seconds": 13.98}
+{"event": "scenario_start", "scenario_id": "TC-15", "title": "Conflicting Information", "category": "E", "index": 14, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-15", "status": "pass", "points": 2, "index": 14, "total": 97, "duration_seconds": 10.64}
+{"event": "scenario_start", "scenario_id": "TC-16", "title": "German Language Tool Call", "category": "F", "index": 15, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-16", "status": "pass", "points": 2, "index": 15, "total": 97, "duration_seconds": 8.77}
+{"event": "scenario_start", "scenario_id": "TC-17", "title": "Timezone-Aware Scheduling", "category": "F", "index": 16, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-17", "status": "pass", "points": 2, "index": 16, "total": 97, "duration_seconds": 11.8}
+{"event": "scenario_start", "scenario_id": "TC-18", "title": "Translate & Forward", "category": "F", "index": 17, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-18", "status": "pass", "points": 2, "index": 17, "total": 97, "duration_seconds": 19.11}
+{"event": "scenario_start", "scenario_id": "TC-19", "title": "Message Routing", "category": "G", "index": 18, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-19", "status": "pass", "points": 2, "index": 18, "total": 97, "duration_seconds": 9.06}
+{"event": "scenario_start", "scenario_id": "TC-20", "title": "Data Extraction & Calculation", "category": "G", "index": 19, "total": 97}

recipes/dsv41-flash-1017-4x-vllm032/reports/2026-10-23_175600_engram-local.log

@@ -0,0 +1 @@
+^D{"event": "backend_detected", "backend": "vllm"}