launcher/specs/bugs/BUG-2026-09-28-vllm-mimo-dflash-parked.md

MiMo-V2.6-Flash-MOPD on vLLM+r39 — the 2026-09-28 experiment (PARKED → RESOLVED)

UPDATE 2026-09-28 ~19:30, 4 hours later — RESOLVED by the ecosystem, not by us. The four walls below were exactly the gap between our r39 pin (last merge 2026-09-17) and local-inference-lab/vllm@dev/karmic-kraken (2026-09-22 series: MiMo aux loading, block-outermost KV layouts for mixed page sizes, b12x non-causal attention, eagle3 language-model unwrap — plus lazy mxfp8 guards). The lane now runs eugr/spark-vllm-b12x:latest (the arm64 nightly CI-built from that branch, v0.1.dev21546+g502d6cb5a) with the lab's own serve-mimo26-flash.sh flag set mirrored (b12x linear/moe, block 128, BF16 KV). Boot reached the wire first try: KV pool 8.57M tokens, DFlash graphs captured on B12X, API serving mimo-v2.6-flash. Our wrapper image (docker-images/mimo26-spark-vllm) and its two patches are RETIRED — upstream shipped the equivalents; delete the dir when the lane stabilizes.

Lane: recipes/mimo26-flash-mopd-4x-vllm + image docker-images/mimo26-spark-vllm (thin FROM wrapper on jovian-judgement:r39-glm53, two anchor patches) — historical, kept for the record.

Verdict for r39: dead end. Every layer of the vLLM DFlash path had to be forced onto a fork that was qualified for GLM-5.3, not MiMo, and each boot revealed one more incompatibility. None were ours to fix cleanly:

1. EAGLE3 hook vs Omni wrapper — MiMo-V2.6 resolves to MiMoV2OmniForCausalLM (mm wrapper); set_eagle3_aux_hidden_state_layers isinstance-checks the OUTER class and raises. Fixable (and fixed) with a language-model fallback patch; the protocol's own method unwraps the same way. Patched, works: "Eagle3 interface resolved on MiMoV2FlashForCausalLM inside MiMoV2OmniForCausalLM", aux layers (1,12,24,36,48) taken from the draft config.
2. B12X rejects non-causal attention — DFlash's block-diffusion draft runs non-causal; b12x in r39 declares it unsupported. Workaround: draft on TRITON_ATTN (target keeps B12X). Works, draft loads (2.7 GB, mask embedding loaded).
3. ModelOptMxFp8LinearMethod ImportError — qwen3_dflash.py hard-imports a class r39's modelopt doesn't ship (fork predates it). Patched to a None-tolerant getattr (bf16 draft unaffected).
4. KV-layout wall (fatal): during cudagraph-memory profiling, validate_kv_cache_layout refuses: "KV cache layout LBNHC cannot express this model's mixed page sizes ([131072, 163840]); set VLLM_KV_CACHE_LAYOUT=BLHNC". The hint is env-only, but BLHNC is a backend-declared-layout question — b12x in r39 declares LBNHC only. This is the architecture mismatch the whole experiment kept routing around: the MiMo hybrid stack (SWA-128 + 9 GA layers, different page sizes per group) needs a vLLM/b12x generation qualified for it; r39 is not.

External intel (2026-09-28)

• local-inference-lab/lil-catalog MiMo DFlash entry: "implemented; runtime qualification unverified" — they never ran it either.
• madeby561/vllm mimo-v26-flash-b12x-20260923-rc4: MiMo RL + b12x + FP8 KV + DFlash-7, with adaptive verification — but linux/amd64 only (RTX 6000 Pro). Not runnable here; its compose is a good flag corpus (enable_adaptive_verification, kv_cache_dtype: fp8 for target+draft, --max-num-scheduled-tokens 2048, prefill-compute-share, BLHNC-class layouts would all apply if a Spark-native equivalent appears).
• eugr/spark-vllm-b12x publishes arm64 nightlies daily (latest 2026-09-28) — first candidate when someone asks whether MiMo landed there.

Status now (2026-09-28 late)

1. DONE: lane switched to eugr/spark-vllm-b12x:latest nightly; boot is clean. Wrapper image + patches retired (upstream equivalents shipped).
2. Open: serial quality bench (vllm-kk-nightly, seed 42 hardmode+perf, conc 1,2, tokenizer /tmp/mimo-tok) vs sglang anchor 168/184 (91/100); then the d0→256K slope sweep (sglang: 77.6 → 23.5 t/s) to see whether b12x split-KV fixes the GA long-context cliff.
3. sglang 4x (mimo26-flash-mopd-4x) is the production lane in the meantime; the FROZEN_KV_MTP experiment is the next sglang-side lever for the long-context cliff.