|
|
@@ -709,6 +709,35 @@ tree `67e7e6e` (base `a8c796f3` + #582 + #646), 16.1 GB, on all 4 nodes.
|
|
|
| r18 4x **DCP4** (TP4+`--decode-context-parallel-size 4`+`--cp-kv-cache-interleave-size 4`) | **161** | **91** | 75 | **4.84M** | **BEST** |
|
|
|
| r18 4x **nvfp4_ds_mla KV** | **161** | **91** | **77** | 2.35M | PASS (best dep) |
|
|
|
|
|
|
+**--perf results** (llama-benchy, pp2048/tg128, runs=3, from each report's
|
|
|
+Throughput Metrics table; c1 = single, c2 = 2 concurrent; t/s = tokens/s):
|
|
|
+
|
|
|
+| Lane (img) | pp c1 | tg c1 | TTFT c1 | pp c2 | tg c2 | TTFT c2 | tg c2 @8k-ctx |
|
|
|
+|---|---:|---:|---:|---:|---:|---:|---:|
|
|
|
+| r17abi 2x (baseline, r17) | 1,577 | 29.0 | 1,362ms | 1,595 | 41.7 | 2,093ms | 29.7 |
|
|
|
+| r18 2x (r18) | 1,473 | 30.7 | 1,481ms | 1,374 | 40.2 | 2,268ms | 30.6 |
|
|
|
+| RedHat MTP k=5 2x (lab) | 1,340 | 26.8 | 1,638ms | **460** | 22.4 | 4,667ms | 14.7 |
|
|
|
+| eugr MTP k=5 2x (eugr) | 1,292 | 31.1 | 2,397ms | 1,388 | 41.6 | 2,751ms | 17.7 |
|
|
|
+| r18 4x RoCE (r18) | **2,904** | 47.7 | **762ms** | 2,413 | **70.9** | 1,399ms | 35.9 |
|
|
|
+| r18 4x DCP4 (r18) | 1,639 | 43.1 | 1,521ms | 1,756 | 51.1 | 1,637ms | 30.0 |
|
|
|
+| r18 4x nvfp4 KV (r18) | 2,614 | 45.3 | 873ms | 2,250 | 62.6 | 1,368ms | 35.6 |
|
|
|
+
|
|
|
+Perf readings:
|
|
|
+- **4x RoCE is the throughput king**: 2× prefill of 2x (2.9k vs 1.6k), 1.7×
|
|
|
+ decode (70.9 c2 vs 41.7), half the TTFT (762ms). 4x wins on both axes vs
|
|
|
+ 2x — the case for the 4x production cell.
|
|
|
+- **DCP4 pays a prefill tax**: 1.6k pp (vs 2.9k RoCE) — the DCP ring
|
|
|
+ exchange costs prefill; decode is comparable (43–51). Its edge is the
|
|
|
+ 4.84M KV capacity + quality, not raw throughput.
|
|
|
+- **nvfp4 KV ≈ RoCE** (2.6k pp, 62.6 c2): the KV dtype costs nothing
|
|
|
+ measurable — combine candidates: DCP4 + nvfp4 KV.
|
|
|
+- **RedHat MTP k=5 collapses at c2**: pp 460 t/s (vs ~1.4–1.6k everywhere
|
|
|
+ else), TTFT 4.7s — quality is fine (156) but the lab image's MTP-k=5 path
|
|
|
+ is perf-weak under concurrency. r18 2x dominates it on both axes.
|
|
|
+- **eugr 2x**: c1/c2 short-ctx throughput matches the baseline, but decode
|
|
|
+ degrades at depth (15.8–17.7 t/s @4k/8k-ctx vs 28–30) — consistent with
|
|
|
+ the quality failure (81/100).
|
|
|
+
|
|
|
Key findings:
|
|
|
- **DCP4 works on our switch fabric** (sparkring steal confirmed): 4.84M KV
|
|
|
tokens at the SAME 10 GiB/node pin — 3.3× plain 4x, and beats their
|
