# BUG-2026-09-14: r37 tree garbles DeepSeek-V4.1-Flash text on 4x GB10 Status: bisect A-C complete (flags excluded), D pending (cluster outage). ## Symptom On `jovian-r37-ds41:local` (vLLM `c687594b8a8`, tree `3fb88fdf`, image label `9350429b...`), DeepSeek-V4.1-Flash (deepseek-ai/DeepSeek-V4.1-Flash, 552B MoE, 4x DGX Spark GB10, TP4) generates text with a high rate of single-letter corruptions — the model seems to miss the vocabulary: - "данних, Етот цыфровой оазис среди вечной мерзлоти, навикации, иследований, планети, критically важной точкой опори, вахтовим, халодом, видлели колосальное количество енергии" (300-word Russian generation, temp 0.7) - code comments: "возврощает", "валідный" (Ukrainian і), "должин быть словарём" - second sample: "метал, процесоров, информатсии, системи, мошной, постувает, эфективная енергия" - third sample (prod contract, below): "индекаторами, температу́рами, данние, халодный, масивов, потерьял, бистро" The corruption hits exactly the high-frequency, closely-spaced vocabulary; structure, identifiers, and long tokens stay intact (5/5 near-identical config keys emitted correctly in a code test). ## Control (clean) The same checkpoint + same 4 nodes + same weights served by the tonyd2wild-overlay image (`dsv41-flash-b12x:local`: `vllm/vllm-openai` digest-pinned arm64 nightly + dsv41-feat@e47aa780 + 7 boot-10 patches) produces clean text (production of record, ~4 weeks of agent traffic). ## Bisect (r37 image, one change per boot, same test prompt) | Boot | Change vs r37 contract | Result | |------|------------------------|--------| | A | remove `--linear-backend b12x` | garbled | | B | prod spec: K5 probabilistic/block, adaptive off (was K7 greedy/standard/adaptive) | garbled | | C | FULL prod contract: GMU 0.85, seqs 8, tokens 8192, block 128, K5-tuned capture sizes, no B12X attention/linear backends, no instanttensor | garbled | | D | C + engram disabled (no `--engram-config`) | **pending** (cluster outage mid-boot, 2026-09-14 ~07:27 UTC) | Excluded along the way: runtime LM-head MXFP8 quantization (`VLLM_MXFP8_LM_HEAD=0` verified in-container, no "Quantizing LM head" log line, KV pool shrank 10.39M -> 10.19M tokens proving the BF16 head) — the typos persist with the BF16 head. ## Implication The defect is in the r37 tree itself (not the contract flags): the default attention/dense path, the `deepseek_v32` model code, the b12x MoE kernels, or the engram disk-read implementation — or a combination. Boot D separates the engram hypothesis from the rest. ## Related (same tree, same week, already documented elsewhere) - DCP>1 sparse-decode crash (`deepseek_v32/attention.py:596`, both r35 and r37) — blocks 512K on the full GLM-5.3 model. - `FlashInferMxFp4LinearKernel.is_supported` accepts sm_121 while cute-dsl `mm_fp4` rejects it. - b12x mxfp4-linear warmup rank-3-vs-4 crash on RedHatAI GLM-5.3-MXFP4 shapes. ## Repro assets - Recipe: `recipes/dsv41-flash-jovian-r37-4x/recipe.toml` (bisect log in the args comments). - Test: greedy 300-word Russian generation, 3 samples above (temp 0.7/0.0). - Boot logs: node `/var/lib/docker/containers/*` (kept via `--keep` until the outage).