|
|
@@ -605,16 +605,32 @@ aiden lane during the day — stop it first, then all 4 nodes are free).
|
|
|
Order = value/duration; each item lists prereqs + expected verdict.
|
|
|
|
|
|
1. **r18 fork build** (docker-images/glm53-jovian-r12, ~1-2 h build).
|
|
|
- Stage on the build node: /tmp/vllm-src.tar.gz (93 MB) +
|
|
|
+ Stage on the build node: /tmp/vllm-src.tar.gz (94 MB) +
|
|
|
/tmp/b12x-src.tar.gz (29 MB) — pins per versions.env (vllm
|
|
|
- f564dffe9a66+pr582 → 725e36d; b12x 613cfd9b+pr284 → c594d44), then
|
|
|
+ a8c796f3+pr582+pr646 → 343661fe; b12x b58f34ea+pr284 → 1adb8a38),
|
|
|
+ then
|
|
|
`TAG=glm53-jovian-r12:local bash build.sh`. Test campaign:
|
|
|
(a) quality anchor glm53-flash-nvfp4-r17abi (expect ≥88/100 hardmode),
|
|
|
(b) **re-bisect `--linear-backend b12x`** — new b12x A16 precision
|
|
|
selection may clear the 44/100 toxin (29.6 c1 unlock if clean),
|
|
|
(c) RoCE one-shot on the 4x lane (VLLM_ENABLE_ROCE_ALLREDUCE=1;
|
|
|
expect −5–19% decode per-step per their #597 numbers),
|
|
|
- (d) `--kv-cache-dtype nvfp4_ds_mla` probe (new B12X KV path).
|
|
|
+ (d) `--kv-cache-dtype nvfp4_ds_mla` probe (new B12X KV path),
|
|
|
+ (e) #646 geometry check: 2048/256/256 split blocks selected +
|
|
|
+ repeat-request prefix resume at the fine boundary (their regression
|
|
|
+ scenario: retention 0, states 32/16, resume at 32) — if broken, drop
|
|
|
+ the 646 pin from versions.env and re-cut (rest of r18 is independent).
|
|
|
+ 1b. **DCP4 probe** (4x lane, no build dependency — run on the current
|
|
|
+ r17abi image first, re-run on r18 if positive). Add
|
|
|
+ `--decode-context-parallel-size 4` (stock vLLM flag, verified in the
|
|
|
+ r18 tree: vllm/config/parallel.py; DCP>1 requires MLA — GLM53
|
|
|
+ qualifies). Reference (sparkring MTP3 mesh profile, 4 Sparks, DCP4 +
|
|
|
+ 24 GiB fp8 KV/rank): ~4.32M logical KV tokens (vs our ~1.25M at
|
|
|
+ 12 GiB DCP1), 231.3 aggregate decode tok/s peak, prefill ~2.7k flat
|
|
|
+ 8K→128K. Verdict: KV-token capacity per node at our pin, decode
|
|
|
+ C1/C8/C16 vs baseline, prefill 8K/64K. Their fabric is a direct
|
|
|
+ HW-forwarded mesh (cheaper all-reduce than our switch) — measure, do
|
|
|
+ not extrapolate.
|
|
|
2. **RedHatAI MTP lane** (2x, ~30 min). Re-run `huggingface-cli download
|
|
|
RedHatAI/GLM-5.3-Flash-NVFP4` on both nodes (model_mtp.safetensors
|
|
|
7.6 GB is new since the initial pulls; HF_HUB_OFFLINE=1). Serve
|
