|
|
@@ -431,3 +431,47 @@ decisively.
|
|
|
4. The b12x-mxfp4-linear warmup rank bug + the flashinfer is_supported
|
|
|
sm_12x false-positive (both workarounds are in the recipe; file both).
|
|
|
5. Push all local commits when git.kilork.org recovers.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Morning shift (2026-09-14, ~05:50-07:00 UTC)
|
|
|
+
|
|
|
+### Operator request: bring prod up; R37 makes suspicious typos
|
|
|
+"как будто оно не попадает в лексику" — check that nothing was exported/quantized/
|
|
|
+serialized wrong.
|
|
|
+
|
|
|
+### Finding 1: R37 tree runtime-quantizes the LM head by default (NOT the cause, but real)
|
|
|
+- `envs.py`: `VLLM_MXFP8_LM_HEAD` default **"1"** — "Quantize eligible unquantized
|
|
|
+ LM heads on b12x by default; =0 opts out." Implementation:
|
|
|
+ `model_executor/layers/vocab_parallel_embedding.py` (sets
|
|
|
+ `runtime_lm_head_quantization = "mxfp8"` for unquantized, untied LM heads).
|
|
|
+- The eugr/tonyd2wild prod image (nightly-20260910 base) has NO such feature —
|
|
|
+ prod runs a BF16 head. Added `VLLM_MXFP8_LM_HEAD = "0"` to the R37 recipe
|
|
|
+ (commit 902e989). Boot without the "Quantizing LM head shards to MXFP8" line
|
|
|
+ = opt-out confirmed; KV pool shrank 10.39M -> 10.19M tokens (head now BF16 =
|
|
|
+ more memory) — indirect confirmation.
|
|
|
+- **But the typos persist** with the BF16 head (verbatim/greedy probes below).
|
|
|
+
|
|
|
+### Typo evidence (R37 lane, temp=0 greedy, ~06:20 UTC)
|
|
|
+Long 300-word Russian generation: "данних. Етот цыфровой оазис среди вечной
|
|
|
+мерзлоти... навикации... иследований... критically важной точкой опори...
|
|
|
+видлели колосальное количество енергии" (10+ mangled words in ~150 words).
|
|
|
+Code sample: "возврощает", "валідный" (Ukrainian і), "должин быть словарём".
|
|
|
+Identifiers check: all 5 near-identical config keys emitted correctly (code
|
|
|
+structure intact; it's the vocab surface layer that wobbles).
|
|
|
+
|
|
|
+### Bisect (in progress at 06:30)
|
|
|
+Flag diff vs every clean lane: R37 contract uniquely adds `--linear-backend
|
|
|
+b12x` (prod: b12x MoE only, default linear; GLM r35 clean lane: B12X attention
|
|
|
++ instanttensor, default linear). Boot A = r37 image WITHOUT `--linear-backend
|
|
|
+b12x`. If typos persist -> Boot B: prod's spec config (K5 probabilistic/block,
|
|
|
+adaptive off) instead of K7 greedy/standard/adaptive-on; Boot C: full prod arg
|
|
|
+set on the r37 image (tree vs contract).
|
|
|
+
|
|
|
+### Naming correction (operator question)
|
|
|
+Our prod 4x image is NOT eugr: `dsv41-flash-b12x:local` = thin wrapper over
|
|
|
+`dsv41-flash:local` = digest-pinned `vllm/vllm-openai` arm64 nightly +
|
|
|
+tonyd2wild overlay chain (dsv41-feat@e47aa780 + 7 boot-10 patches + FlashInfer
|
|
|
+0.7.0rc1, all built in this repo). "eugr" was my sloppy label inherited from
|
|
|
+the `deepseek-v4-flash-eugr*`/`glm53-flash-*eugr*` lane naming; from now on:
|
|
|
+"DS4.1 prod" for the tonyd2wild lane, "eugr" only for the eugr-image lanes.
|
