launcher/DEVELOPMENT.md

Development — sm121-launcher

Rust CLI tool for launching vLLM/SGLang containers across a DGX Spark cluster. SSH-based provisioning over russh.

Project Layout

Cargo.toml                 # Binary crate, Edition 2024
config.toml                # User config (gitignored)
locales/                   # Fluent i18n files (en, ru, es, it, fr, de)
recipes/                   # Recipe directories (<name>/recipe.toml)
src/
├── main.rs                # CLI entrypoint, command dispatch, i18n init
├── cli.rs                 # clap derive CLI definitions
├── config.rs              # Config deserialization, cluster resolution;
│                          #   config/types.rs (Node/Cluster/Config),
│                          #   config/recipe.rs (Recipe + [env] EnvValue refs)
├── ssh.rs                 # SSH client, key loading, host key verification
├── docker/
│   ├── mod.rs             # Docker command builder: serve, stop, health, logs
│   └── tests.rs           # Command builder tests
├── tmux/
│   ├── mod.rs             # Tmux module root
│   ├── serve.rs           # Tmux serve (window/panel layout)
│   └── ops.rs             # Tmux stop/status/logs/attach
├── power.rs               # Power on/off/switch flows
├── home_assistant.rs      # HA REST API client
├── host_keys.rs           # init-host-keys command
├── exec.rs                # Exec subcommand + alias resolution
├── doctor.rs              # Doctor mode + --repair
├── pre_serve.rs           # PreServeCommand types
├── before_serve.rs        # Pre-serve hook execution
└── health.rs              # Tmux health-check helpers

Quality Gates

make check            # All gates: fmt → clippy → test → line-count
make fmt              # cargo fmt
make check-fmt        # cargo fmt --check
make check-clippy     # cargo clippy --all-targets -- -D warnings
make check-test       # cargo test
make check-lines      # File-size ≤500 lines per src/**/*.rs

Architecture

Docker launch flow

1. serve reads the recipe TOML
2. Resolves cluster (exact/prefix match → smallest with NCCL + matching node count)
3. Runs pre-serve hooks (cluster-level + recipe-level)
4. Checks for repaired image (via doctor --repair)
5. For each node: builds docker run command via build_docker_command(), runs via SSH
6. Waits 10s, then shows tail logs

build_docker_command() in src/docker/mod.rs is engine-aware:

Aspect	vLLM	SGLang
Host IP env var	VLLM_HOST_IP	SGLANG_WORKER_IP
Model argument	bare name	--model-path
TP flag	--tensor-parallel-size	--tp
PP flag	--pipeline-parallel-size	—
Backend flag	--distributed-executor-backend (ray only)	—
Init addr	--master-addr + --master-port	--dist-init-addr
Workers	--headless	—

Auto-generated flags: --host 0.0.0.0, --port <recipe.port>, --nnodes, --node-rank.

Cluster selection

• Exact match: --cluster 4nodes-switch
• Prefix match: --cluster 4 → resolves to 4nodes-switch (unique prefix, fails on ambiguous)
• Auto: --cluster auto (or omitted) → picks the smallest cluster with NCCL config matching required_nodes

Env var precedence

1. Cluster-driven (from [nccl]): NCCL_IB_DISABLE, NCCL_IB_HCA, NCCL_SOCKET_IFNAME, etc.
2. Network interface (from nccl.socket_ifname): MN_IF_NAME, UCX_NET_DEVICES, OMPI_MCA_btl_tcp_if_include
3. Defaults: OMP_NUM_THREADS=4, NCCL_IGNORE_CPU_AFFINITY=1, NCCL_NET=IB, NCCL_DEBUG=WARN
4. Engine IP: VLLM_HOST_IP / SGLANG_WORKER_IP
5. Head node: VLLM_LOGGING_COLOR=1, TERM=xterm-256color, -p <port>:<port>
6. Ray backend: RAY_ADDRESS, RAY_PORT, RAY_NODE_IP_ADDRESS, RAY_OVERRIDE_NODE_IP_ADDRESS
7. Recipe [env]: appended last, overrides everything above

Power management

• power off: SSH sudo shutdown now → wait shutdown_delay_seconds → HA turn_off per-node entities + cluster extras
• power on: HA turn_on per-node entities + cluster extras (no SSH)
• power switch: disable nodes/extras unique to from → enable all in to

i18n

Mozilla Fluent via fluent-i18n crate. Locale auto-detected from system. FTL files in locales/{lang}/main.ftl.

Module size limit

Every src/**/*.rs file must stay ≤500 lines. Enforced by make check-lines.

Dev Workflow

# Build
cargo build

# Run locally with dry-run (no SSH execution)
cargo run -- -d serve --recipe deepseek-v4-flash

# Run tests
cargo test

# All gates (must pass before commit)
make check

# Add a new recipe
mkdir -p recipes/my-model/
# create recipes/my-model/recipe.toml
cargo run -- -d serve --recipe my-model --cluster <name>

# Add a new locale
cp locales/en/main.ftl locales/xx/main.ftl
# translate strings, commit

Dependencies

Crate	Version	Purpose
clap	4.6	CLI parsing (derive)
russh	0.61	SSH client
russh-keys	0.50-beta	SSH key loading
tokio	1 (full)	Async runtime
toml	1	Config deserialization
serde	1	Serialization
tracing	0.1	Logging
anyhow	1.0	Error handling
futures	0.3	Concurrent SSH
sha2	0.11	Host key fingerprints
base64	0.22	Base64 encoding
reqwest	0.13	HTTP client (HA)
serde_json	1	JSON (HA)
dirs	6	OS config dir
fluent-i18n	—	Mozilla Fluent i18n