Răsfoiți Sursa

mimo26-flash-mopd-4x: RoCEnante live on TP4 (a15), tg +20-36%

SGLang port's SG17 guard (world==8) relaxed to b12x core's 2..16; vLLM
lanes proved world=4 years-ago. First live sglang TP4 one-shot all-reduce:
tg d0 75.9 t/s (+20% vs NCCL), d4096 c1 76.3 (+36%), TTFT -10%. GMU 0.88,
pool 3.68M. Perf bench: 2026-09-28_152839_roce-perf.
Alexander Korolev 3 săptămâni în urmă
părinte
comite
615e526ecd

+ 12 - 0
history/night-shifts/2026-09-27.md

@@ -404,3 +404,15 @@ trial diff is page-cache margin, not a blocker). Pool 4,137,472; 515K probe
   everything lives under /root/.cache/sglang — 35 triton dirs, inductor,
   flashinfer; b12x-roce unused on TP<8): bind to ~/.cache/mimo26-jit/sglang
   on local NVMe. Takes effect next serve (production NOT restarted).
+
+## RoCEnante on TP4 — a15 (28 Sep, ~15:00 UTC)
+
+The vLLM lanes had RoCEnante live on 4 nodes since Sep-07 (b12x core takes
+world 2..16); the sglang port's SG17 guard had it pinned to world==8. a15
+relaxes the guard to the core's range (2..16, Gloo-exchange-group check and
+2-HCA check intact) and `mimo26-flash-mopd-4x` turns it on with
+SGLANG8_ROCE_ALLREDUCE=1 + GMU 0.88.
+- Boot: `RoCEnante ready: world=4 hcas=rocep1s0f0,roceP2p1s0f0`, ROUTE fires
+  on first decode all-reduce. Pool 3,677,056 @0.88.
+- Perf vs the 0.94 NCCL run: tg d0 c1 75.9 (+20%), d4096 c1 76.3 (+36%),
+  c2 +15%, d8192 flat, TTFT ~10% better, pp unchanged.

+ 34 - 0
recipes/mimo26-flash-mopd-4x/reports/2026-09-28_152839_roce-perf.log

@@ -0,0 +1,34 @@
+^D{"event": "backend_detected", "backend": "sglang"}
+
+╭──────────────────── ⚡ llama-benchy Throughput Benchmark ────────────────────╮
+│ mimo-v2.6-flash                                                              │
+│ pp=[2048]  tg=[128]  depth=[0, 4096, 8192]  concurrency=[1, 2]  runs=3       │
+│ latency=generation                                                           │
+╰──────────────────────────────────────────────────────────────────────────────╯
+
+  ✓ Complete ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18/18 0:01:34
+
+  llama-benchy 0.4.0
+  Estimated latency: 102.0 ms
+
+                               llama-benchy Results                             
+┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━
+┃                      ┃    ┃         ┃         ┃     TTFT ┃    Total ┃         
+┃ Test                 ┃ c  ┃  pp t/s ┃  tg t/s ┃     (ms) ┃     (ms) ┃   Tokens
+┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━
+│ pp2048 tg128 @ d0    │ c1 │   3,321 │    75.9 │      743 │    2,307 │ 2048+128
+│ pp2048 tg128 @ d0    │ c2 │   2,718 │    95.7 │    1,521 │    3,857 │ 2048+128
+│ pp2048 tg128 @ d4096 │ c1 │   3,388 │    76.3 │    1,945 │    3,495 │ 2048+128
+│ pp2048 tg128 @ d4096 │ c2 │   3,244 │   103.4 │    3,795 │    6,052 │ 2048+128
+│ pp2048 tg128 @ d8192 │ c1 │   3,273 │    61.6 │    3,262 │    5,211 │ 2048+128
+│ pp2048 tg128 @ d8192 │ c2 │   3,247 │    91.1 │    6,314 │    8,743 │ 2048+128
+└──────────────────────┴────┴─────────┴─────────┴──────────┴──────────┴─────────
+
+  ℹ Metrics sourced from llama-benchy — see https://github.com/eugr/llama-benchy
+for methodology.
+
+
+  Report saved to 
+/Users/kilork/.cache/sm121/bench/2026-09-28_152839_roce-perf/runs/2026/09/2026-0
+9-28T15-30-14.756157Z_aec19423.md
+

+ 44 - 0
recipes/mimo26-flash-mopd-4x/reports/2026-09-28_152839_roce-perf.md

@@ -0,0 +1,44 @@
+# Throughput Benchmark — mimo-v2.6-flash
+
+- **Run ID**: `2026-09-28T15-30-14.756157Z_aec19423`
+- **Date**: `2026-09-28T15:30:14.756885+00:00`
+- **Mode**: throughput-only
+- **tool-eval-bench**: `v2.7.1.dev7+gbd35ba91b`
+
+## Run Context
+
+| Parameter | Value |
+|---|---|
+| Backend | sglang |
+| Server | `http://***:30000/v1` |
+| Model (API) | `mimo-v2.6-flash` |
+| Temperature | 0.0 |
+| Seed | 42 |
+| Max Turns | 8 |
+| Timeout | 120.0s |
+| Scenarios | all (92) |
+| Parallel | 1 (sequential) |
+| Error Rate | 0.0 |
+| Thinking | enabled |
+| Max Tokens | 16384 |
+
+## Inference Engine
+
+| Property | Value |
+|---|---|
+| Engine | SGLang |
+| Max Model Length | 524,288 |
+| Host | `localhost` |
+| Platform | `macOS-26.6.2-arm64-arm-64bit-Mach-O` |
+| Python | 3.14.7 |
+
+## Results
+
+| Test | pp t/s | tg t/s | TTFT (ms) | Total (ms) | Tokens |
+|---|---:|---:|---:|---:|---:|
+| pp2048 tg128 @ d0 | 3,321 | 75.9 | 743 | 2,307 | 2048+128 |
+| pp2048 tg128 @ d0 c2 | 2,718 | 95.7 | 1,521 | 3,857 | 2048+128 |
+| pp2048 tg128 @ d4096 | 3,388 | 76.3 | 1,945 | 3,495 | 2048+128 |
+| pp2048 tg128 @ d4096 c2 | 3,244 | 103.4 | 3,795 | 6,052 | 2048+128 |
+| pp2048 tg128 @ d8192 | 3,273 | 61.6 | 3,262 | 5,211 | 2048+128 |
+| pp2048 tg128 @ d8192 c2 | 3,247 | 91.1 | 6,314 | 8,743 | 2048+128 |