Преглед на файлове

glm53-mxfp4-fp8-4x-r39: MTP k=5 measured — same throughput as k=3 (33-37 c1, 47.9 c2), acceptance 4.4/5

Verification, not acceptance, is the limit: 87-93% of drafts are accepted at
depth 5 and the ladder does not add t/s over depth 3. Both double the
un-speculated decode (19.0 -> ~35 t/s).
Alexander Korolev преди 20 часа
родител
ревизия
866d05a06c
променени са 1 файла, в които са добавени 8 реда и са изтрити 2 реда
  1. 8 2
      recipes/glm53-mxfp4-fp8-4x-r39/NOTE.md

+ 8 - 2
recipes/glm53-mxfp4-fp8-4x-r39/NOTE.md

@@ -57,8 +57,14 @@ helped (karmic's native b12x MoE experts: 19.2 vs 19.0 t/s). Speculation does:
 | config | decode c1 | decode c2 (aggregate) | acceptance len |
 |---|---|---|---|
 | no spec (first boot) | 19.0 t/s | — | — |
-| **MTP k=3** | **33.4–34.9 t/s** | **47.6 t/s** | 3.2–3.6 / 3 |
-| MTP k=5 | see below | | |
+| **MTP k=3** | **33.4–34.9 t/s** | **47.6 t/s** | 3.2–3.6 of 3 |
+| **MTP k=5** | **33.1–37.3 t/s** | **47.9 t/s** | 4.36–4.66 of 5 |
+
+k=5 buys nothing over k=3 in throughput (the drafts are accepted 87–93% of the
+time, so the ladder is not the limit — verification is). Both roughly **double**
+the un-speculated decode: 19.0 → ~35 t/s single request. The recipe keeps k=5
+because it is what was last measured; k=3 is the safer default if memory is
+tighter. |
 
 `--speculative-config '{"method":"mtp","num_speculative_tokens":3}'` — the MTP
 head is in the checkpoint (`num_nextn_predict_layers=1`), so no draft model is