|
|
@@ -68,5 +68,11 @@ algo=MXFP8` confirm the MTP fix is live; `algo=None` there = broken load.
|
|
|
- Vendor instanttensor loader into the lab image (~30 s vs 700 s weight load).
|
|
|
- Consider adding `--kernel-config` autotune-off + pinned KV to the redhat and
|
|
|
LibertAIDAI recipes (their boot could be minutes shorter; verify before trusting).
|
|
|
-- MTP for redhat is impossible unless someone trains/exports a drafter into
|
|
|
- the checkpoint — don't chase it.
|
|
|
+- MTP for redhat: RedHatAI ships MTP as **separate variant repos** (proven
|
|
|
+ pattern: `RedHatAI/GLM-5.2-NVFP4-FP8-mtp-FP8` = full quant + `mtp.safetensors`
|
|
|
+ alongside main weights). Nothing for 5.3-Flash yet (org search 2026-08-28:
|
|
|
+ only NVFP4 + BF16). If/when `GLM-5.3-Flash-NVFP4-mtp-*` lands, the recipe
|
|
|
+ change is: swap `model` + add `--speculative-config` (k=3). Cross-grafting
|
|
|
+ the lab's MXFP8 MTP as an external `speculative-config.model` draft is
|
|
|
+ theoretically possible but untested in vLLM — don't chase before the
|
|
|
+ official drop.
|
