|
|
@@ -326,3 +326,41 @@ known-good configuration class (a boot is low-risk, ~50 min) and answers the
|
|
|
+43% prefill claim on GB10; then **P0 (MXFP4)** with the whole remaining
|
|
|
night, whose probe boot decides the capability question. Teardown/restore is
|
|
|
P0.5.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## P0 in flight (interim, ~23:55 UTC) — full MXFP4 on 4x: PROBE PASSED, DCP4 ladder
|
|
|
+
|
|
|
+Boot ladder for `glm53-full-mxfp4-4x-probe` (no DCP, 131K, r35 img):
|
|
|
+1. `--moe-backend flashinfer_cutlass_afp8` → argparse rejects it (not a CLI
|
|
|
+ choice on r35; the CLI family is `flashinfer_cutlass`, the `_afp8` key is
|
|
|
+ an oracle-level thing).
|
|
|
+2. `--moe-backend flashinfer_cutlass` → accepted, but the flag is INERT for
|
|
|
+ this checkpoint: the compressed-tensors W4A4-MXFP4 MoE scheme picks
|
|
|
+ **MarlinExperts** on its own. The boot then died in the mxfp4 LINEAR
|
|
|
+ kernel: the default priority [FlashInfer, Marlin, Humming, B12x] starts
|
|
|
+ with the FlashInfer kernel, whose `is_supported()` wrongly accepts
|
|
|
+ capability 121 (numeric `has_device_capability(100)` passes on GB10)
|
|
|
+ while the cute-dsl mm_fp4 GEMM rejects it at runtime:
|
|
|
+ `mm_fp4 does not support backend 'cute-dsl' with capability 121`
|
|
|
+ (kv_b_proj dequant, mla_attention.py).
|
|
|
+3. `--linear-backend b12x` → correct selection ("Using
|
|
|
+ B12xMxFp4LinearKernel for MXFP4 GEMM"; verified per-component: family(120)
|
|
|
+ True, b12x blockscaled OK, cute-dsl input-quantize works on 121) — but the
|
|
|
+ UNCONDITIONAL b12x warmup crashes on this checkpoint's shapes:
|
|
|
+ "Expected source and destination tensors to have the same rank, but got 3
|
|
|
+ and 4" (b12x dense_gemm via synthetic zero source). Filed-worthy for the lab.
|
|
|
+4. **`--linear-backend marlin` + 8 GiB KV → BOOTED.** (A 4 GiB pin was
|
|
|
+ rejected: the engine needs 6.58 GiB/rank for ONE 131072-token request →
|
|
|
+ the full model's KV = **51.6 KB/token/rank**, fp8_ds_mla.)
|
|
|
+ Pool: **159,232 tokens per 8 GiB pin** (52.7 KB/token), max concurrency
|
|
|
+ 1.21x at 131K. Smoke: 27×43=1161 ✓, tool call exact in 4.2 s ✓,
|
|
|
+ 35,157-token needle answered "73451" in 5.1 s (≈7.8K tok/s prefill) ✓,
|
|
|
+ no degenerate repetition. **The 740B model works on 4× GB10.**
|
|
|
+5. DCP4 first boot: `DCP sparse indexer currently supports only
|
|
|
+ cp_kv_cache_interleave_size=1 (got 4)` — the x4 interleave is a Flash
|
|
|
+ dense-MLA rule, not the DSA top-k path. Recipe fixed to interleave 1,
|
|
|
+ rebooting.
|
|
|
+
|
|
|
+DCP4 512K pool estimate: 8 GiB/rank ÷ (52.7 KB/token ÷ 4) ≈ 650K tokens
|
|
|
+≥ 524288 ✓. Then smoke + dev65 bench `dev65-full-mxfp4-dcp4`, then P2.
|
