Ver Fonte

mxfp4 recipes: --linear-backend marlin - the b12x mxfp4 warmup (ungated) crashes on this checkpoint's shapes (rank 3 vs 4 in b12x dense_gemm); marlin A16 is the guaranteed path

Alexander Korolev há 1 mês atrás
pai
commit
675ccdb307

+ 12 - 1
recipes/glm53-full-mxfp4-4x-probe/recipe.toml

@@ -207,7 +207,18 @@ args = [
     # (The lab's GLM-5.1 runbook describes the same kernel family for the
     # quant_method=mxfp4 ingestion path - this checkpoint goes through the
     # compressed-tensors path, which is why --moe-backend was the wrong lever.)
-    "--linear-backend", "b12x",
+    # First attempt: b12x (the native SM120 GEMM) - correct selection, but the
+    # unconditional b12x warmup (b12x_warmup.py, no config gate) crashes on
+    # this checkpoint's layer shapes during kernel compile:
+    #   RuntimeError: Expected source and destination tensors to have the same
+    #   rank, but got 3 and 4  (b12x/gemm/blockscaled/_linear.py dense_gemm,
+    #   via _apply_b12x_mxfp4_linear on a synthetic zero source)
+    # so the mxfp4 LINEAR layers run on Marlin (weight-only A16 dequant GEMM -
+    # the same family as the MarlinExperts the MoE scheme picked). Slower than
+    # native FP4 for the attention projections, but the MoE experts (the
+    # dominant cost) are Marlin either way. The b12x warmup rank bug is
+    # filed-worthy for the lab (RedHatAI GLM-5.3-MXFP4 on GB10/sm_121a).
+    "--linear-backend", "marlin",
     # pure I/O loader (kept across every GLM config)
     "--load-format", "instanttensor",
     "--block-size", "256",

+ 12 - 1
recipes/glm53-full-mxfp4-4x/recipe.toml

@@ -200,7 +200,18 @@ args = [
     # (The lab's GLM-5.1 runbook describes the same kernel family for the
     # quant_method=mxfp4 ingestion path - this checkpoint goes through the
     # compressed-tensors path, which is why --moe-backend was the wrong lever.)
-    "--linear-backend", "b12x",
+    # First attempt: b12x (the native SM120 GEMM) - correct selection, but the
+    # unconditional b12x warmup (b12x_warmup.py, no config gate) crashes on
+    # this checkpoint's layer shapes during kernel compile:
+    #   RuntimeError: Expected source and destination tensors to have the same
+    #   rank, but got 3 and 4  (b12x/gemm/blockscaled/_linear.py dense_gemm,
+    #   via _apply_b12x_mxfp4_linear on a synthetic zero source)
+    # so the mxfp4 LINEAR layers run on Marlin (weight-only A16 dequant GEMM -
+    # the same family as the MarlinExperts the MoE scheme picked). Slower than
+    # native FP4 for the attention projections, but the MoE experts (the
+    # dominant cost) are Marlin either way. The b12x warmup rank bug is
+    # filed-worthy for the lab (RedHatAI GLM-5.3-MXFP4 on GB10/sm_121a).
+    "--linear-backend", "marlin",
     # pure I/O loader (kept across every GLM config)
     "--load-format", "instanttensor",
     "--block-size", "256",