# Development — sm121-launcher Rust CLI tool for launching vLLM/SGLang containers across a DGX Spark cluster. SSH-based provisioning over `russh`. ## Project Layout ``` Cargo.toml # Binary crate, Edition 2024 config.toml # User config (gitignored) locales/ # Fluent i18n files (en, ru, es, it, fr, de) recipes/ # Recipe directories (/recipe.toml) src/ ├── main.rs # CLI entrypoint, command dispatch, i18n init ├── cli.rs # clap derive CLI definitions ├── config.rs # Config deserialization, cluster resolution; │ # config/types.rs (Node/Cluster/Config), │ # config/recipe.rs (Recipe + [env] EnvValue refs) ├── ssh.rs # SSH client, key loading, host key verification ├── docker/ │ ├── mod.rs # Docker command builder: serve, stop, health, logs │ └── tests.rs # Command builder tests ├── tmux/ │ ├── mod.rs # Tmux module root │ ├── serve.rs # Tmux serve (window/panel layout) │ └── ops.rs # Tmux stop/status/logs/attach ├── power.rs # Power on/off/switch flows ├── home_assistant.rs # HA REST API client ├── host_keys.rs # init-host-keys command ├── exec.rs # Exec subcommand + alias resolution ├── doctor.rs # Doctor mode + --repair ├── pre_serve.rs # PreServeCommand types ├── before_serve.rs # Pre-serve hook execution ├── health.rs # Tmux health-check helpers ├── recipe_repos.rs # [[recipe_repos]] git sync (clone/pull into │ # ~/.cache/sm121/recipe-repos/), │ # the repos.json archive registry, sm121 repos/ │ # archive/unarchive ├── promote.rs # sm121 promote (cross-repo git mv: target add + │ # push first, source remove + push second) └── docker/registry_pull.rs # --image-source registry (pull the prebuilt # {base}/{name}:{checksum} tag, verify the # sm121.checksum label) ``` ## Quality Gates ```bash make check # All gates: fmt → clippy → test → line-count make fmt # cargo fmt make check-fmt # cargo fmt --check make check-clippy # cargo clippy --all-targets -- -D warnings make check-test # cargo test make check-lines # File-size ≤500 lines per src/**/*.rs ``` ## Architecture ### Docker launch flow 1. `serve` reads the recipe TOML 2. Resolves cluster (exact/prefix match → smallest with NCCL + matching node count) 3. Runs pre-serve hooks (cluster-level + recipe-level) 4. Checks for repaired image (via doctor --repair) 5. For each node: builds `docker run` command via `build_docker_command()`, runs via SSH 6. Waits 10s, then shows tail logs `build_docker_command()` in `src/docker/mod.rs` is engine-aware: | Aspect | vLLM | SGLang | |---|---|---| | Host IP env var | `VLLM_HOST_IP` | `SGLANG_WORKER_IP` | | Model argument | bare name | `--model-path` | | TP flag | `--tensor-parallel-size` | `--tp` | | PP flag | `--pipeline-parallel-size` | — | | Backend flag | `--distributed-executor-backend` (ray only) | — | | Init addr | `--master-addr` + `--master-port` | `--dist-init-addr` | | Workers | `--headless` | — | Auto-generated flags: `--host 0.0.0.0`, `--port `, `--nnodes`, `--node-rank`. ### Cluster selection - **Exact match**: `--cluster 4nodes-switch` - **Prefix match**: `--cluster 4` → resolves to `4nodes-switch` (unique prefix, fails on ambiguous) - **Auto**: `--cluster auto` (or omitted) → picks the **smallest** cluster with NCCL config matching `required_nodes` ### Env var precedence 1. Cluster-driven (from `[nccl]`): `NCCL_IB_DISABLE`, `NCCL_IB_HCA`, `NCCL_SOCKET_IFNAME`, etc. 2. Network interface (from `nccl.socket_ifname`): `MN_IF_NAME`, `UCX_NET_DEVICES`, `OMPI_MCA_btl_tcp_if_include` 3. Defaults: `OMP_NUM_THREADS=4`, `NCCL_IGNORE_CPU_AFFINITY=1`, `NCCL_NET=IB`, `NCCL_DEBUG=WARN` 4. Engine IP: `VLLM_HOST_IP` / `SGLANG_WORKER_IP` 5. Head node: `VLLM_LOGGING_COLOR=1`, `TERM=xterm-256color`, `-p :` 6. Ray backend: `RAY_ADDRESS`, `RAY_PORT`, `RAY_NODE_IP_ADDRESS`, `RAY_OVERRIDE_NODE_IP_ADDRESS` 7. Recipe `[env]`: appended last, overrides everything above ### Power management - `power off`: SSH `sudo shutdown now` → wait `shutdown_delay_seconds` → HA `turn_off` per-node entities + cluster extras - `power on`: HA `turn_on` per-node entities + cluster extras (no SSH) - `power switch`: disable nodes/extras unique to `from` → enable all in `to` ### i18n Mozilla Fluent via `fluent-i18n` crate. Locale auto-detected from system. FTL files in `locales/{lang}/main.ftl`. ### Recipe repos & the HA add-on Full design: `specs/tech-architecture/ha-addon-recipe-repos.md`. - **`[[recipe_repos]]`** (ordered list; first = highest priority on a recipe-name collision): git repos holding `recipes/` (+ `docker-images/`), cloned/pulled by `recipe_repos::sync_recipe_repos` into `~/.cache/sm121/recipe-repos/`. Best-effort: an unreachable repo warns + is skipped (a dead playground must not take down production serving). The synced roots are appended to `recipe_sources` (as their `recipes/` dirs) and `docker_image_sources` (as repo roots) in list order — the existing search-order semantics do the shadowing. Skipped in dry-run (no git I/O). - **Archive registry** (`repos.json` next to the global config): separate from the config, so a repo can be archived without editing it (`sm121 archive ` / `unarchive`). Missing → fresh; corrupt → `.bak` + fresh + warning. - **`sm121 promote --from --to [--apply]`**: decomposed git mv — target checkout: `cp -a` + `git add/commit/push` FIRST; then source: `git rm -r` + commit + push. A failure in between leaves the recipe duplicated (safe under priority semantics), never deleted. Collision in the target is refused up front. `--dry-run` (the global `-d`) prints the exact commands. - **`--image-source {repo,registry}`** (serve/build): `repo` (default) = the current on-demand build from `docker-images/`; `registry` pulls the prebuilt `{base}/{name}:{checksum12}` from the `[registry]` base on every launch node, tags it as the recipe's `docker_image`, and verifies the `sm121.checksum` label (a missing tag names the tag + points at build/push or the `repo` fallback). Mutually exclusive with `--build-node`. - **The add-on image** (`docker-images/sm121-addon/`): multi-stage (the binary is baked in by CI — a pinned rust:alpine build, the Pi never compiles), alpine + git + jq, entrypoint writes the effective config from /data/options.json (the HA add-on options — env injection is legacy-app only), drops the SSH key (base64 option), `sm121 repos sync`, then `sm121 web --port 3100 --bind 0.0.0.0` (Bearer token; `sm121_insecure: true` disables auth) with a re-sync supervisor. `make addon-image` / `addon-push`; smoke: `scripts/smoke-addon.sh`. The HA add-on manifest lives in the `github.com/kilork/sm121-ha-integration` repo (its `custom_components/` prototype is superseded — entities are a phase-2 nicety over the add-on's REST API). - **Single-launcher rule**: the per-recipe run ledger (`.lock` in the recipe repo) serializes serving — run ONE launcher as the source of truth (the add-on); the Mac launcher is for dev. ### Module size limit Every `src/**/*.rs` file must stay ≤500 lines. Enforced by `make check-lines`. ## Dev Workflow ```bash # Build cargo build # Run locally with dry-run (no SSH execution) cargo run -- -d serve --recipe deepseek-v4-flash # Run tests cargo test # All gates (must pass before commit) make check # Add a new recipe mkdir -p recipes/my-model/ # create recipes/my-model/recipe.toml cargo run -- -d serve --recipe my-model --cluster # Add a new locale cp locales/en/main.ftl locales/xx/main.ftl # translate strings, commit ``` ## Dependencies | Crate | Version | Purpose | |---|---|---| | `clap` | 4.6 | CLI parsing (derive) | | `russh` | 0.61 | SSH client | | `russh-keys` | 0.50-beta | SSH key loading | | `tokio` | 1 (full) | Async runtime | | `toml` | 1 | Config deserialization | | `serde` | 1 | Serialization | | `tracing` | 0.1 | Logging | | `anyhow` | 1.0 | Error handling | | `futures` | 0.3 | Concurrent SSH | | `sha2` | 0.11 | Host key fingerprints | | `base64` | 0.22 | Base64 encoding | | `reqwest` | 0.13 | HTTP client (HA) | | `serde_json` | 1 | JSON (HA) | | `dirs` | 6 | OS config dir | | `fluent-i18n` | — | Mozilla Fluent i18n |