|
@@ -219,6 +219,50 @@ NCCL env vars come from the cluster's `[nccl."<name>"]` section. All containers
|
|
|
|
|
|
|
|
For Ray backend, nodes start sequentially (head first with API readiness poll up to 150s). For MP backend, all nodes start in parallel.
|
|
For Ray backend, nodes start sequentially (head first with API readiness poll up to 150s). For MP backend, all nodes start in parallel.
|
|
|
|
|
|
|
|
|
|
+## Environment Variables
|
|
|
|
|
+
|
|
|
|
|
+The launcher passes the following env vars to every container. Recipe-level
|
|
|
|
|
+`[env]` entries are appended after the defaults and can override them.
|
|
|
|
|
+
|
|
|
|
|
+### Cluster-driven (from `[nccl."<cluster>"]`)
|
|
|
|
|
+
|
|
|
|
|
+| Env var | Source | Purpose |
|
|
|
|
|
+|---|---|---|
|
|
|
|
|
+| `NCCL_IB_DISABLE` | `nccl.<cluster>.ib_disable` | Disable NCCL's InfiniBand path (0 = use IB, 1 = disable) |
|
|
|
|
|
+| `NCCL_IB_HCA` | `nccl.<cluster>.ib_hca` | RDMA devices for NCCL (comma-separated, e.g. `rocep1s0f0,roceP2p1s0f0`) |
|
|
|
|
|
+| `NCCL_SOCKET_IFNAME` | `nccl.<cluster>.socket_ifname` | Network interface NCCL uses for socket-based control (e.g. `enp1s0f0np0`) |
|
|
|
|
|
+| `NCCL_IB_GID_INDEX` | `nccl.<cluster>.ib_gid_index` | RoCE GID index for InfiniBand routing |
|
|
|
|
|
+| `GLOO_SOCKET_IFNAME` | `nccl.<cluster>.socket_ifname` | Network interface for Gloo (vLLM distributed + PyTorch CPU collectives) |
|
|
|
|
|
+| `TP_SOCKET_IFNAME` | `nccl.<cluster>.socket_ifname` | vLLM tensor-parallel coordinator interface |
|
|
|
|
|
+
|
|
|
|
|
+### Hard-coded defaults
|
|
|
|
|
+
|
|
|
|
|
+| Env var | Value | Purpose |
|
|
|
|
|
+|---|---|---|
|
|
|
|
|
+| `NCCL_CUMEM_ENABLE` | `0` | Disable CUDA-managed memory in NCCL (use system allocator) |
|
|
|
|
|
+| `OMP_NUM_THREADS` | `4` | OpenMP thread limit per process |
|
|
|
|
|
+| `HF_HOME` | `/cache/huggingface` | HuggingFace cache root (volume-mounted from `ssh_user`'s host cache) |
|
|
|
|
|
+
|
|
|
|
|
+### Engine-specific host IP
|
|
|
|
|
+
|
|
|
|
|
+| Env var | Engines | Value | Purpose |
|
|
|
|
|
+|---|---|---|---|
|
|
|
|
|
+| `VLLM_HOST_IP` | vLLM only | `node.ib_ip` | vLLM's own host-IP for distributed init |
|
|
|
|
|
+| `SGLANG_WORKER_IP` | SGLang only | `node.ib_ip` | SGLang's worker-IP for distributed init |
|
|
|
|
|
+
|
|
|
|
|
+### Ray-only (when `distributed_backend = "ray"`)
|
|
|
|
|
+
|
|
|
|
|
+| Env var | Value | Purpose |
|
|
|
|
|
+|---|---|---|
|
|
|
|
|
+| `RAY_ADDRESS` | `<master_ib_ip>:6379` | Ray head address (workers connect here) |
|
|
|
|
|
+| `RAY_PORT` | `recipe.port` | Ray gRPC port |
|
|
|
|
|
+
|
|
|
|
|
+### Recipe-level overrides
|
|
|
|
|
+
|
|
|
|
|
+Anything in `[env]` is appended after all of the above, so a recipe can
|
|
|
|
|
+override any default (e.g. `NCCL_CUMEM_ENABLE = "1"` to re-enable
|
|
|
|
|
+CUDA-managed memory).
|
|
|
|
|
+
|
|
|
## Doctor Mode
|
|
## Doctor Mode
|
|
|
|
|
|
|
|
```bash
|
|
```bash
|