Explorar o código

feat: set GLOO_SOCKET_IFNAME and TP_SOCKET_IFNAME for vLLM

vLLM's distributed executor relies on Gloo and tensor-parallel
coordinator/collectives, both of which select interfaces by name.  By
default they try to autodetect (and fall back to garbage like "enP7s7"
on DGX Spark), causing failures like:

  RuntimeError: Unable to find address for: enP7s7

Pin them to the cluster's configured `socket_ifname` so they match
the NCCL setting.  Always set; no opt-in needed since both libraries
ignore env vars they don't need.
Alexander Korolev hai 4 meses
pai
achega
1b02f53b07
Modificáronse 2 ficheiros con 10 adicións e 0 borrados
  1. 2 0
      src/docker/mod.rs
  2. 8 0
      src/docker/tests.rs

+ 2 - 0
src/docker/mod.rs

@@ -375,6 +375,8 @@ fn build_docker_command(
         format!("-e NCCL_IB_DISABLE={}", nccl.ib_disable),
         format!("-e NCCL_IB_HCA={}", nccl.ib_hca),
         format!("-e NCCL_SOCKET_IFNAME={}", nccl.socket_ifname),
+        format!("-e GLOO_SOCKET_IFNAME={}", nccl.socket_ifname),
+        format!("-e TP_SOCKET_IFNAME={}", nccl.socket_ifname),
         format!("-e NCCL_IB_GID_INDEX={}", nccl.ib_gid_index),
         format!("-e HF_HOME=/cache/huggingface"),
         format!("-v {}:/cache/huggingface", cache_dir),

+ 8 - 0
src/docker/tests.rs

@@ -115,6 +115,14 @@ fn mp() {
         cmd.contains("-e NCCL_CUMEM_ENABLE=0"),
         "default NCCL_CUMEM_ENABLE missing"
     );
+    assert!(
+        cmd.contains("-e GLOO_SOCKET_IFNAME=e"),
+        "default GLOO_SOCKET_IFNAME missing"
+    );
+    assert!(
+        cmd.contains("-e TP_SOCKET_IFNAME=e"),
+        "default TP_SOCKET_IFNAME missing"
+    );
     assert!(
         cmd.contains("-e OMP_NUM_THREADS=4"),
         "default OMP_NUM_THREADS missing"