Ver código fonte

r37 recipe: VLLM_MXFP8_LM_HEAD=0 - the lab tree's runtime LM-head MXFP8 quantization (default ON, absent from the eugr prod image) is the suspected cause of the typo regression

Alexander Korolev 1 mês atrás
pai
commit
902e989
1 arquivos alterados com 7 adições e 0 exclusões
  1. 7 0
      recipes/dsv41-flash-jovian-r37-4x/recipe.toml

+ 7 - 0
recipes/dsv41-flash-jovian-r37-4x/recipe.toml

@@ -111,6 +111,13 @@ args = [
 
 # Their serve-ds41-jovian.sh runtime env (multi-node-adapted; the launcher
 # injects NCCL_SOCKET_IFNAME/IB_HCA/GID + VLLM_HOST_IP per node).
+# 2026-09-14 (night shift, operator's typo report): the lab tree's runtime
+# LM-head quantization (vocab_parallel_embedding.py) quantizes an unquantized
+# LM head to MXFP8 BY DEFAULT (envs.py: VLLM_MXFP8_LM_HEAD, default "1").
+# The eugr 2026-09-10 nightly (our prod image) has NO such feature and keeps
+# the BF16 head - the coarsened 8-bit vocab-logit layer is the suspected cause
+# of the typo-rate regression (close-token logits distorted). Opting out.
+VLLM_MXFP8_LM_HEAD = "0"
 NCCL_NCHANNELS = "16"
 NCCL_MIN_NCHANNELS = "16"
 NCCL_MAX_NCHANNELS = "16"