|
|
@@ -1,9 +1,23 @@
|
|
|
-# MiMo-V2.6-Flash-MOPD on vLLM+r39 — the 2026-09-28 experiment (PARKED)
|
|
|
+# MiMo-V2.6-Flash-MOPD on vLLM+r39 — the 2026-09-28 experiment (PARKED → RESOLVED)
|
|
|
+
|
|
|
+**UPDATE 2026-09-28 ~19:30, 4 hours later — RESOLVED by the ecosystem, not by
|
|
|
+us.** The four walls below were exactly the gap between our r39 pin (last
|
|
|
+merge 2026-09-17) and `local-inference-lab/vllm@dev/karmic-kraken` (2026-09-22
|
|
|
+series: MiMo aux loading, block-outermost KV layouts for mixed page sizes,
|
|
|
+b12x non-causal attention, eagle3 language-model unwrap — plus lazy mxfp8
|
|
|
+guards). The lane now runs `eugr/spark-vllm-b12x:latest` (the arm64 nightly
|
|
|
+CI-built from that branch, v0.1.dev21546+g502d6cb5a) with the lab's own
|
|
|
+`serve-mimo26-flash.sh` flag set mirrored (b12x linear/moe, block 128, BF16
|
|
|
+KV). Boot reached the wire first try: KV pool 8.57M tokens, DFlash graphs
|
|
|
+captured on B12X, API serving `mimo-v2.6-flash`. Our wrapper image
|
|
|
+(`docker-images/mimo26-spark-vllm`) and its two patches are RETIRED —
|
|
|
+upstream shipped the equivalents; delete the dir when the lane stabilizes.
|
|
|
|
|
|
Lane: `recipes/mimo26-flash-mopd-4x-vllm` + image `docker-images/mimo26-spark-vllm`
|
|
|
-(thin FROM wrapper on `jovian-judgement:r39-glm53`, two anchor patches).
|
|
|
+(thin FROM wrapper on `jovian-judgement:r39-glm53`, two anchor patches) —
|
|
|
+historical, kept for the record.
|
|
|
|
|
|
-**Verdict: dead end for now.** Every layer of the vLLM DFlash path had to be
|
|
|
+**Verdict for r39: dead end.** Every layer of the vLLM DFlash path had to be
|
|
|
forced onto a fork that was qualified for GLM-5.3, not MiMo, and each boot
|
|
|
revealed one more incompatibility. None were ours to fix cleanly:
|
|
|
|
|
|
@@ -43,16 +57,14 @@ revealed one more incompatibility. None were ours to fix cleanly:
|
|
|
* `eugr/spark-vllm-b12x` publishes arm64 nightlies daily (latest 2026-09-28)
|
|
|
— first candidate when someone asks whether MiMo landed there.
|
|
|
|
|
|
-## How to reopen
|
|
|
+## Status now (2026-09-28 late)
|
|
|
|
|
|
-1. Rebase the wrapper onto a MiMo-qualified vLLM (eugr nightly that carries
|
|
|
- MiMo + b12x, or a new lab rev with upstream's MiMo/b12x reconciliation —
|
|
|
- the upstream PR #46104 lineage) and retry the same recipe. The two
|
|
|
- patches (eagle3-inner + mxfp8-optional) stay valid until that vLLM ships
|
|
|
- the equivalents; then delete them.
|
|
|
-2. If the new stack boots, run the same long-context slope sweep used for
|
|
|
- sglang (`~/.cache/sm121/bench/slope-28x`: 77.6 → 23.5 t/s d0→256K) and the
|
|
|
- serial quality bench (anchor: sglang 168/184, 91/100).
|
|
|
+1. DONE: lane switched to `eugr/spark-vllm-b12x:latest` nightly; boot is
|
|
|
+ clean. Wrapper image + patches retired (upstream equivalents shipped).
|
|
|
+2. Open: serial quality bench (`vllm-kk-nightly`, seed 42 hardmode+perf,
|
|
|
+ conc 1,2, tokenizer /tmp/mimo-tok) vs sglang anchor 168/184 (91/100);
|
|
|
+ then the d0→256K slope sweep (sglang: 77.6 → 23.5 t/s) to see whether
|
|
|
+ b12x split-KV fixes the GA long-context cliff.
|
|
|
3. sglang 4x (`mimo26-flash-mopd-4x`) is the production lane in the meantime;
|
|
|
the FROZEN_KV_MTP experiment is the next sglang-side lever for the
|
|
|
long-context cliff.
|
