Переглянути джерело

glm53-mxfp4-fp8-4x-r39: DSpark speculator attempt recorded — dies in the draft KV builder on r39 (needs vLLM 0.32)

RedHatAI/GLM-5.3-speculator.dspark (2.5 GB, 3-layer Qwen3 draft, hidden states
from target layers 2/20/39/58/75, up to 8 drafts, 1M ctx, 41% acceptance /
4.29 accepted length at 524K-1M) is the right drafter for this model, but on our
r39 fork it passes config validation and draft-backend selection and then dies
inside qwen3_dflash._build_context_kv_buffers while loading draft weights. The
speculator targets vLLM 0.32 / speculators 0.7.x — retry on a vLLM-main image.
Lane restored to the working MTP k=3 at 131072 context.
Alexander Korolev 17 годин тому
батько
коміт
559e8d4db9
1 змінених файлів з 7 додано та 0 видалено
  1. 7 0
      recipes/glm53-mxfp4-fp8-4x-r39/recipe.toml

+ 7 - 0
recipes/glm53-mxfp4-fp8-4x-r39/recipe.toml

@@ -108,6 +108,13 @@ args = [
     "--enable-auto-tool-choice",
     "--tool-call-parser", "glm47",
     "--reasoning-parser", "glm45",
+    # 2026-10-22 23:26: the DSpark route (RedHatAI/GLM-5.3-speculator.dspark,
+    # k=8, draft backend FLASHINFER + bfloat16 draft KV) gets through config
+    # validation and the draft backend selection, then dies inside
+    # qwen3_dflash._build_context_kv_buffers during draft weight load — the
+    # speculator is built for vLLM 0.32 + speculators 0.7.x, our r39 fork is
+    # older. Retry on a vLLM-main image (eugr nightly), not here.
+    #
     # MTP speculation (2026-10-22, 2nd boot): the head is in the checkpoint
     # (num_nextn_predict_layers=1); k=3 is our proven Flash-lane depth. The
     # window drops to 32768 to pay for the draft KV + extra graph shapes —