Jelajahi Sumber

eugr GLM lane: document forum 381350#364 parity + DFlash2 alt spec (night queue 3b: k=7 incoai drafter A/B vs MTP k=5)

Alexander Korolev 1 bulan lalu
induk
melakukan
54dfffeb24

+ 5 - 0
recipes/glm53-flash-nvfp4-eugr/recipe.toml

@@ -45,6 +45,11 @@
 # REFERENCE:
 #   https://github.com/eugr/spark-vllm-docker  (recipes/glm-5.3-flash.yaml)
 #   https://hub.docker.com/r/eugr/spark-vllm-b12x/tags
+#   https://forums.developer.nvidia.com/t/381350/364 — public confirmation of
+#   this flag set (2026-09-05, tea24864 on 4x/TP4, perf-only: ~60 tok/s c1,
+#   ~73 c4 decode, prefill 2.3-2.8k tok/s) + the DFlash2 alternative spec
+#   ("method":"dflash" + "incoai/GLM-5.3-Flash-DFlash2" drafter, k=7, see
+#   night queue 3b)
 
 engine = "vllm"
 name = "GLM-5.3-Flash NVFP4 eugr B12X (vLLM, 2x Spark, MTP k=5, RoCE)"

+ 8 - 0
specs/tech-architecture/glm53-recipe-state.md

@@ -644,6 +644,14 @@ Order = value/duration; each item lists prereqs + expected verdict.
    local-inference-lab/GLM-5.3-Flash-NVFP4-Spark` on both nodes. Serve
    glm53-flash-nvfp4-eugr → A/B vs r18: full b12x cluster quality
    (linear-b12x toxin retest on their stack) + RoCE 2x decode numbers.
+   3b. **DFlash2 spec A/B** (same eugr image, after 3 boots). Swap the
+   spec to `"method":"dflash","model":"incoai/GLM-5.3-Flash-DFlash2",
+   "num_speculative_tokens":7,"draft_tensor_parallel_size":2,
+   "kv_cache_dtype":"auto"` (separate drafter repo — huggingface-cli
+   download on both nodes first; exists on HF, single safetensors). VS
+   the recipe's MTP k=5: decode at C1/C4 + quality spot-check (forum
+   numbers are perf-only: ~60 c1 / ~73 c4 tok/s on 4x — their 4x/TP4
+   config, so 2x numbers will differ).
 4. **DSV4F-Vision eugr lane** (2x, prefer dgx3/dgx4 — weight reuse).
    Reuse the per-node ~/dsv4vision-hf copies (verify the HF_HOME mapping
    vs the recipe's default HF cache) or re-download Vision-Exp (168 GB).