glm53-mxfp4-fp8-4x-r39: MTP k=3 breaks the decode ceiling — 19.0 -> 33-35 t/s c1, 47.6 t/s c2

The full model is bandwidth-bound at ~99 GiB of weights per rank, so kernel
swaps did nothing (karmic b12x MoE: 19.2). Speculation does: MTP k=3 (head is
in the checkpoint, no drafter loaded) gives acceptance length 3.2-3.6 of 3,
decode 33.4-34.9 t/s single-request and 47.6 t/s aggregate at c2. Weights
96.34 -> 98.86 GiB/rank, window drops to 32768 to pay for the draft cache.

Note also records the measurement caveat: with speculation SSE chunks bundle
several tokens, so count usage.completion_tokens, not chunks.

recipes/glm53-mxfp4-fp8-4x-r39/NOTE.md

@@ -48,3 +48,28 @@ fp8_block modules keep their own backend.
 Older siblings that had already run at 4x: `glm53-full-mxfp4-4x` /
 `-probe` (2026-09-13/14, pure MXFP4 cut) — see their headers for the memory
 plan, DCP4 ladder and the 2026-09-08 OOM-thrash hazard.
+
+## MTP speculation — the 20 t/s ceiling breaks (2026-10-22, second boot)
+
+Decode was bandwidth-bound at 96–99 GiB of weights per rank, so no kernel swap
+helped (karmic's native b12x MoE experts: 19.2 vs 19.0 t/s). Speculation does:
+
+| config | decode c1 | decode c2 (aggregate) | acceptance len |
+|---|---|---|---|
+| no spec (first boot) | 19.0 t/s | — | — |
+| **MTP k=3** | **33.4–34.9 t/s** | **47.6 t/s** | 3.2–3.6 / 3 |
+| MTP k=5 | see below | | |
+
+`--speculative-config '{"method":"mtp","num_speculative_tokens":3}'` — the MTP
+head is in the checkpoint (`num_nextn_predict_layers=1`), so no draft model is
+loaded; vLLM iterates that head per step (it warns that depth > 1 runs the same
+MTP layer several times, which is exactly the trade we want here). Weights grow
+96.34 → 98.86 GiB/rank, and the window drops to 32768 to pay for the draft
+cache and the extra graph shapes (KV pin stays 4 GiB).
+
+Engine metrics in the same window: `Mean acceptance length: 3.41–3.63,
+verification depth 3.00/3`.
+
+Caveat for streaming clients: with speculation the stream bundles several
+tokens into one chunk, so counting SSE chunks under-reports t/s; measure with
+`usage.completion_tokens` (600-token generations here).

recipes/glm53-mxfp4-fp8-4x-r39/recipe.toml

@@ -88,7 +88,7 @@ args = [
     # fp8_block layers keep their own (working) backend.)
     "--load-format", "instanttensor",
     "--block-size", "256",
-    "--max-model-len", "65536",
+    "--max-model-len", "32768",
     "--max-num-seqs", "4",
     "--max-num-batched-tokens", "4096",
     "--enable-chunked-prefill",
@@ -102,9 +102,12 @@ args = [
     "--enable-auto-tool-choice",
     "--tool-call-parser", "glm47",
     "--reasoning-parser", "glm45",
-    # No --speculative-config on the first boot: the MTP head is real
-    # (num_nextn_predict_layers=1) but draft KV + graph shapes need headroom
-    # this lane does not have yet. Add mtp k=3 after a clean boot.
+    # MTP speculation (2026-10-22, 2nd boot): the head is in the checkpoint
+    # (num_nextn_predict_layers=1); k=3 is our proven Flash-lane depth. The
+    # window drops to 32768 to pay for the draft KV + extra graph shapes —
+    # decode is bandwidth-bound at 96 GiB/rank and spec is the only lever that
+    # moves it without more nodes.
+    "--speculative-config", """'{"method":"mtp","num_speculative_tokens":5}'""",
 ]
 
 [env]