# sm121-launcher

Rust CLI tool for launching vLLM/SGLang containers across a DGX Spark cluster via SSH.

## Features

- Launch distributed LLM inference clusters with a single command
- Support for vLLM and SGLang engines
- Multi-cluster support with auto-selection
- SSH-based provisioning — no agent required on target nodes
- Docker-in-Docker deployment via SSH exec
- Health checks, log tailing, and clean cluster teardown
- Dry-run mode to preview commands without execution

## Quick Start

```bash
# Build
cargo build --release

# Launch a recipe with auto-cluster selection
cargo run serve --recipe glm51

# Launch on a specific cluster
cargo run serve --recipe glm51 --cluster 4nodes-switch

# Dry-run (print SSH commands without executing)
cargo run -- -d serve --recipe glm51

# View cluster logs
cargo run logs --recipe glm51                # Last 100 lines per node
cargo run logs --recipe glm51 --follow       # Follow head node logs

# Stop all containers on the cluster
cargo run stop --recipe glm51

# Health check
cargo run health                                     # All clusters
cargo run health --cluster 2nodes-dac               # Single cluster
```

## Configuration

### config.toml

```toml
ssh_user = "kilork"
ssh_key_path = "/path/to/ssh/key"

[[clusters]]
name = "4nodes-switch"
master_ib_ip = "192.168.200.15"
nodes = [
    { name = "dgx1.lan", ssh_ip = "192.168.88.14", ib_ip = "192.168.200.15", rank = 0 },
    { name = "dgx2.lan", ssh_ip = "192.168.88.10", ib_ip = "192.168.200.13", rank = 1 },
]

[nccl."4nodes-switch"]
ib_hca = "rocep1s0f0,roceP2p1s0f0"
socket_ifname = "enp1s0f0np0"
ib_gid_index = 3
ib_disable = 0
```

### Recipes

Recipes live in `recipies/<name>/recipe.toml` (note the directory spelling).

```toml
engine = "vllm"
name = "GLM-5.1 NVFP8"
docker_image = "vllm/vllm-openai:glm51-cu130"
model = "lukealonso/GLM-5.1-NVFP4"
port = 30000
required_nodes = 4
pipeline_parallel_size = 0    # 0 = auto (equals nnodes)

args = [
    "--kv-cache-dtype", "fp8",
    "--max-model-len", "32768",
]

[env]
NCCL_DEBUG = "WARN"
```

## Launch Behavior

For each node, the tool builds and executes a `docker run` command via SSH with:
- Full GPU access (`--gpus all --ipc=host --privileged --network=host`)
- NCCL env vars from the cluster's `[nccl."<name>"]` section
- Recipe-defined env vars and args
- Appended multi-node flags: `--tensor-parallel-size 1`, `--pipeline-parallel-size`, `--nnodes`, `--node-rank`, `--master-addr`, `--distributed-executor-backend mp`

`--cluster auto` selects the first cluster in config with an NCCL section.

## Dependencies

| Crate | Purpose |
|---|---|
| `clap` 4.6 | CLI argument parsing (derive) |
| `russh` 0.60 | SSH client |
| `tokio` full | Async runtime |
| `toml` 1 + `serde` 1 | Config/recipe deserialization |
| `tracing` | Structured logging |
| `anyhow` | Error handling |