redhat lane: add MTP k=5 — model_mtp.safetensors (7.6 GB) now in the main NVFP4 repo; card 'Enable Speculative Decoding' section (2026-09-05)

recipes/glm53-flash-nvfp4-redhat/recipe.toml

@@ -11,10 +11,12 @@
 #   524288 ctx. GMU 0.90 matches the proven lab config; the pin overrides
 #   GMU-based KV sizing.
 #
-# ⚠ NO MTP DRAFTER IN THIS CHECKPOINT — the index has 146,745 tensors and
-#   zero layer-45/MTP weights. `--speculative-config mtp` WILL fail to load.
-#   This recipe is baseline decode only (~15 tok/s class on this arch —
-#   spec decode buys 1.3–2×; use glm53-flash-nvfp4[-lab] when MTP matters).
+# MTP (added 2026-09-05): the checkpoint now ships the MTP drafter as
+#   model_mtp.safetensors (7.6 GB, added to the repo after our 2026-08-28
+#   index scan that found zero MTP weights) — vendor card section "Enable
+#   Speculative Decoding" recommends k=5. Cost: +~7.6 GB weights/node on top
+#   of the 88 GB base; if the boot dies on free-memory or KV shrinks, drop
+#   the KV pin 10 → 8 GiB. k=3 is the fallback if k=5 regresses quality.
 #
 # Image: the -lab overlay (day-0 vLLM image + NoPE-MLA mod + FujitsuPolycom
 # arch patches). The sparse_attn_indexer CC-12.0 guards are REQUIRED at the
@@ -41,13 +43,16 @@
 #
 # PREREQUISITES: weights cached on both nodes:
 #   huggingface-cli download RedHatAI/GLM-5.3-Flash-NVFP4
+# (re-run the download on nodes cached before 2026-09-04: model_mtp.safetensors
+#  7.6 GB was added to the repo after the initial pulls; HF_HUB_OFFLINE=1 means
+#  a missing file is a hard load failure)
 #
 # REFERENCE:
 #   https://huggingface.co/RedHatAI/GLM-5.3-Flash-NVFP4 (vendor vLLM cmd)
 #   https://github.com/kingjones30/GLM-5.3-Flash-2x-DGX-Spark (mod)
 
 engine = "vllm"
-name = "GLM-5.3-Flash NVFP4-redhat (vLLM, 2x Spark, no MTP)"
+name = "GLM-5.3-Flash NVFP4-redhat (vLLM, 2x Spark, MTP k=5)"
 docker_image = "glm53-flash-nvfp4-lab:local"
 docker_image_dir = "glm53-flash-nvfp4-lab"
 model = "RedHatAI/GLM-5.3-Flash-NVFP4"
@@ -78,7 +83,10 @@ args = [
     "--kv-cache-dtype", "fp8_ds_mla",
     "--language-model-only",
     "--moe-backend", "marlin",
-    # NO --speculative-config: checkpoint ships no MTP weights (verified index)
+    # MTP drafter: model_mtp.safetensors in the main repo (vendor card
+    # "Enable Speculative Decoding": k=5; no separate drafter repo needed)
+    "--speculative-config", """
+'{"method":"mtp","num_speculative_tokens":5}'""",
     "--tool-call-parser", "glm47",
     "--enable-auto-tool-choice",
     "--reasoning-parser", "glm45",

specs/tech-architecture/glm53-recipe-state.md

@@ -126,14 +126,14 @@ retry serve (or the launcher refuses with "only 1 nodes").
 - Consider adding `--kernel-config` autotune-off to the LibertAIDAI recipe
   (boot minutes shorter; redhat now has it, see 512K section — verify before
   trusting on LibertAIDAI).
-- MTP for redhat: RedHatAI ships MTP as **separate variant repos** (proven
-  pattern: `RedHatAI/GLM-5.2-NVFP4-FP8-mtp-FP8` = full quant + `mtp.safetensors`
-  alongside main weights). Nothing for 5.3-Flash yet (org search 2026-08-28:
-  only NVFP4 + BF16). If/when `GLM-5.3-Flash-NVFP4-mtp-*` lands, the recipe
-  change is: swap `model` + add `--speculative-config` (k=3). Cross-grafting
-  the lab's MXFP8 MTP as an external `speculative-config.model` draft is
-  theoretically possible but untested in vLLM — don't chase before the
-  official drop.
+- MTP for redhat: RESOLVED (2026-09-05) — no variant repo needed: the main
+  `RedHatAI/GLM-5.3-Flash-NVFP4` repo gained `model_mtp.safetensors` (7.6 GB)
+  and the card an "Enable Speculative Decoding" section (k=5, plain
+  `--speculative-config {"method":"mtp",...}` — same model path, no swap).
+  Recipe `glm53-flash-nvfp4-redhat` updated to MTP k=5 (watch the +7.6 GB/node
+  memory cost vs the 10 GiB KV pin; k=3 fallback). The lab's 5.2 pattern
+  (`-mtp-FP8` variant repos) did not apply; the cross-graft MXFP8-MTP idea is
+  moot.
 
 ## 4p67: --enforce-eager vs CUDA graphs (2026-08-29, same seed/invocation)