|
|
@@ -976,26 +976,39 @@ latency lower in 14/15 cells (c=1 62–64 vs 65–66 ms; c=16 194–212 vs
|
|
|
kernels — i.e. it attacks exactly our measured decode profile
|
|
|
(bandwidth-bound, scales to ~N=4; prefill untouched).
|
|
|
|
|
|
-Integration surface — verified already in our tree:
|
|
|
-- the vLLM adapter `b12x_roce_all_reduce.py` is **already present in our
|
|
|
- r27 pin** (63a82f8d, blob a311ae52) — the voipmonitor fork mirrors
|
|
|
- local-inference-lab/vllm (our pin resolves there), and the A/B receipt's
|
|
|
- adapter (aef3576) predates our pin, so this is a zero-port item;
|
|
|
+Integration surface — **verified already in our r27 image** (2026-09-07,
|
|
|
+dgx5, `docker run -i` import check, all OK):
|
|
|
+- our r27 b12x pin (e8ad299b) is NOT just the original #295 merge — the r27
|
|
|
+ line carries it plus follow-ups: two-rail striping (aa90a277 — our
|
|
|
+ `rocep1s0f0,roceP2p1s0f0` fabric is exactly what it targets), DSV4
|
|
|
+ graph-collective opt (1a7e3ec2), PR #315 gather-geometry (b58f34ea). The
|
|
|
+ r27 variant is ahead of the public line, not behind (5 of 8 roce files
|
|
|
+ differ from the LIL A/B revision; the r27 ones add per-stripe completion
|
|
|
+ flags + power-of-two grid counters);
|
|
|
+- the vLLM adapter `b12x_roce_all_reduce.py` is in our vLLM pin (63a82f8d,
|
|
|
+ blob a311ae52); `import b12x.comm.roce` + `roce_oneshot` + the adapter all
|
|
|
+ import cleanly in the built image;
|
|
|
- b12x env falls back to `NCCL_IB_HCA` / `NCCL_IB_GID_INDEX` — our config
|
|
|
- already pins both CX-7 functions (`rocep1s0f0,roceP2p1s0f0`) and the
|
|
|
- per-node GID auto-detect propagates automatically;
|
|
|
+ already pins both CX-7 functions and the per-node GID auto-detect
|
|
|
+ propagates automatically;
|
|
|
- enable = `VLLM_ENABLE_ROCE_ALLREDUCE=1` in recipe `[env]` (caps
|
|
|
`VLLM_ROCE_ALLREDUCE_MAX_SIZE`=2M / `VLLM_ROCE_ALLGATHER_MAX_SIZE`=16M).
|
|
|
|
|
|
+Repo note: **voipmonitor/b12x is the operational repo** (holds the r27/r28
|
|
|
+qualification lines; LIL is the public community repo where PR #295 merged
|
|
|
+and 00280f2 sits on a dangling branch of). The first bump attempt
|
|
|
+(e8ad299b → 00280f2) was a wrong move — 00280f2 is a diverged mainline
|
|
|
+commit (no r27 GLM line, no `integration/vllm/loader.py`), broke the
|
|
|
+b12x-managed build gate, and was reverted; no rebuild was needed.
|
|
|
+
|
|
|
Remaining work (r27 context, production untouched):
|
|
|
-1. Bump `B12X_PIN` e8ad299b → 00280f2 (resolve full SHA + tree hash,
|
|
|
- one versions.env line) + dgx5 clone-at-pin rebuild (~1–2 h).
|
|
|
-2. A/B on the r27 2x lane (2 free nodes): decode c1/c2 vs the anchors
|
|
|
- (24.5–27 t/s tg, 1,356 pp c1); re-verify quality — bf16 sums are within
|
|
|
- 6.8e-3 rel of NCCL (float32 accumulate), so expect the 156 to hold.
|
|
|
+1. **Done (as revised):** no pin bump — RoCEnante is already baked in.
|
|
|
+2. A/B on the r27 2x lane (2 free nodes): `VLLM_ENABLE_ROCE_ALLREDUCE=1` vs
|
|
|
+ the existing anchors (24.5–27 t/s tg, 1,356 pp c1); re-verify quality —
|
|
|
+ bf16 sums are within 6.8e-3 rel of NCCL (float32 accumulate), so expect
|
|
|
+ the 156 to hold.
|
|
|
3. A 4x A/B would mean production sign-off (the toggle is an env var only).
|
|
|
|
|
|
-Queue: item #4, after the r27 DFlash2 fork patch + the r18 16 GiB
|
|
|
-boot-verify. Sources: local-inference-lab/b12x PR #295, docs/rocenante.md,
|
|
|
-receipts docs/evidence/rocenante/ (20260902 serving A/B + 20260903
|
|
|
-standalone, b12x 00280f2).
|
|
|
+Queue: item #2 is now the only remaining piece. Sources:
|
|
|
+local-inference-lab/b12x PR #295 (+ follow-up PRs #315, r27-line commits
|
|
|
+aa90a277/1a7e3ec2), docs/rocenante.md, receipts docs/evidence/rocenante/.
|
