Ver Fonte

glm53-flash-fp4-local-2x: add distributed_backend="mp" + pipeline_parallel_size=1 + shm_size=16g — the missing distributed_backend made the launcher skip ALL dist flags, so each node booted an independent single-node engine loading the FULL 166 GB checkpoint (the root cause of the four stalls); launcher now refuses this config (4029b28)

Alexander Korolev há 5 dias atrás
pai
commit
c5a8444bb5
1 ficheiros alterados com 8 adições e 0 exclusões
  1. 8 0
      recipes/glm53-flash-fp4-local-2x/recipe.toml

+ 8 - 0
recipes/glm53-flash-fp4-local-2x/recipe.toml

@@ -24,6 +24,14 @@ parent_model = "zai-org/GLM-5.3-Flash"
 port = 30000
 required_nodes = 2
 tensor_parallel_size = 2
+pipeline_parallel_size = 1
+distributed_backend = "mp"
+shm_size = "16g"
+# distributed_backend is MANDATORY for multi-node: without it the launcher
+# skips ALL dist flags (--tensor-parallel-size/--nnodes/--node-rank/--master-addr)
+# and each node boots an INDEPENDENT single-node engine that tries to load the
+# full 166 GB checkpoint into a 121 GiB box — the root cause of the four
+# "stalls" (b12x routing OOM) before this was caught.
 # GB10 adaptation of the card's DGX-Spark-x2 block: the card ships
 # --max-model-len 500000 + --gpu-memory-utilization 0.87 + --max-num-seqs 4.
 # On GB10 (unified memory, not cgroup-contained — forum 386057) that first