ソースを参照

feat(recipe): add GLM-5.2 NVFP4 recipe (4-node SGLang)

Recipe for Mapika/GLM-5.2-NVFP4 on 4x DGX Spark using
scitrera/dgx-spark-sglang-mm:v0 SGLang image. Based on:

- Model card serving command (quantization modelopt_fp4, moe-runner-backend
  flashinfer_cutlass, mem-fraction-static 0.85)
- vLLM recipe page flags (tool-call-parser glm47, reasoning-parser glm45)
- Existing minimax-m3 SGLang NVFP4 recipe pattern (cuda-graph tuning, env)
Alexander Korolev 4 ヶ月 前
親
コミット
3d35c82535
1 ファイル変更、27 行追加、0 行削除
  1. 27 0
      recipes/glm52/recipe.toml

+ 27 - 0
recipes/glm52/recipe.toml

@@ -0,0 +1,27 @@
+engine = "sglang"
+name = "GLM-5.2 NVFP4"
+docker_image = "scitrera/dgx-spark-sglang-mm:v0"
+model = "Mapika/GLM-5.2-NVFP4"
+required_nodes = 4
+tensor_parallel_size = 4
+distributed_backend = "mp"
+
+entrypoint = ["sglang", "serve"]
+
+args = [
+    "--quantization", "modelopt_fp4",
+    "--mem-fraction-static", "0.85",
+    "--context-length", "32768",
+    "--fp4-gemm-backend", "cutlass",
+    "--kv-cache-dtype", "auto",
+    "--reasoning-parser", "glm45",
+    "--tool-call-parser", "glm47",
+    "--moe-runner-backend", "flashinfer_cutlass",
+    "--trust-remote-code",
+    "--cuda-graph-max-bs", "16",
+    "--cuda-graph-bs", "1", "2", "4", "8", "12", "16",
+    "--served-model-name", "glm52",
+]
+
+[env]
+TORCH_NCCL_ASYNC_ERROR_HANDLING = "1"