Quellcode durchsuchen

live 8x probe results: Ling fp8 feasible on GB10 (serves+generates; bench needs tool-call config + thinking control); Step-5 ruled out (Step5 DSA needs SM90/SM103, GB10=SM121)

Alexander Korolev vor 1 Woche
Ursprung
Commit
b4bcb389ec
2 geänderte Dateien mit 21 neuen und 0 gelöschten Zeilen
  1. 9 0
      recipes/ling-31-flash-nvfp4/recipe.toml
  2. 12 0
      recipes/step5-preview-nvfp4/recipe.toml

+ 9 - 0
recipes/ling-31-flash-nvfp4/recipe.toml

@@ -14,6 +14,15 @@
 #
 # GB10/sm_121a fp8-MoE feasibility is UNVERIFIED for this arch — first live
 # probe. See specs/tech-architecture/upstream-watch-2026-10-16.md.
+#
+# RESULT 2026-10-16 (live 8x serve, stock v0.31.0): FEASIBILITY PROVEN — all 8
+# ranks load 534 GB over NFS (~40 min), reach readiness on :30000, and generate
+# (vLLM picks FP8 MoE DeepGEMM/TRITON + TRITON_MLA + FLASH_ATTN prefill; no UMA
+# boot gate at max-model-len 65536 / GMU 0.78). A valid tool-eval-bench score
+# still needs serving config: `--enable-auto-tool-choice --tool-call-parser
+# <bailing parser>` (else tool scenarios 400 on tool_choice=auto) and thinking
+# control (`--no-think` in the harness, or the model's long default reasoning
+# blows the 120 s/turn timeout → 94/97 scenarios excluded). Follow-up.
 
 engine = "vllm"
 name = "Ling-3.1-flash fp8 (8x, stock v0.31.0)"

+ 12 - 0
recipes/step5-preview-nvfp4/recipe.toml

@@ -1,5 +1,17 @@
 # Step-5-Preview-NVFP4 (stepfun-ai) on 8x DGX Spark — PREP recipe (GB10 probe).
 #
+# ⚠ RESULT 2026-10-16 (live 8x probe): CANNOT run on DGX Spark. The dedicated
+# image resolves `Step5ForConditionalGeneration` + `modelopt_fp4` correctly, but
+# the engine aborts at model init:
+#   NotImplementedError: Step5 DSA requires SM90 or SM103; got
+#   DeviceCapability(major=12, minor=1)
+# GB10 is SM121 (CC 12.1); Step-5's DeepSeek Sparse Attention kernel is built
+# only for SM90 (Hopper) / SM103 (B-series Blackwell). Hard architectural
+# blocker — not a config/flag issue. Kept as a recipe for the record; do NOT
+# retry on Spark until upstream ships an SM121 DSA path.
+#
+# Original notes below.
+#
 # Status 2026-10-16: `Step5ForConditionalGeneration` (`model_type: step5_vl`) —
 # a VISION-LANGUAGE model (8-bit, modelopt/NVFP4), 604B, ~473 GB. vLLM v0.31.0
 # registers `StepVLForConditionalGeneration` (step_vl) but NOT