---
name: sm121-launcher
description: Operate the sm121 Rust CLI launcher for this repo — serve/stop/logs/health/doctor/power/tmux on the DGX Spark cluster, cluster+recipe selection, image builds, and dev gates. Use whenever working with sm121 commands, recipes, config, or the DGX Spark cluster.
---

# sm121 Launcher

`sm121` is the Rust CLI (`edition 2024`) that launches vLLM/SGLang engine
containers across the DGX Spark cluster over SSH (`russh`, Docker default,
tmux alternative). A model may be **serving live traffic at any time** — read
the Safety section before anything destructive.

## Quick reference

| Action | Command |
|---|---|
| Build | `cargo build` |
| All gates (before commit) | `make check` |
| Dry-run (safe, no SSH) | `cargo run -- -d serve --recipe <name>` |
| Tests | `cargo test` (65) |
| l10n | `locales/{en,ru,es,it,fr,de}/main.ftl` |

## Commands

`serve` / `stop` / `logs` / `health` / `doctor` / `exec` / `power` / `tmux` /
`init-host-keys`.

- `sm121 health --cluster <c>` — safe; ALWAYS run before stop/power (see Safety).
- `sm121 logs --recipe <r> [--cluster <c>]` — safe.
- `sm121 doctor [--repair]` — inspect/repair node+image state; safe.
- `sm121 exec --recipe <r> -- <cmd>` — run a command in the serving container;
  honors `exec_aliases` from `config.toml`.
- `sm121 serve --recipe <r> [--cluster <c>] [--keep] [--follow]` — main launch;
  `--keep` keeps the container after engine exit, `--follow` tails logs.
- `sm121 stop --recipe <r> [--cluster <c>]` — teardown (Docker stop/rm, or tmux
  session kill; runs `after_stop` hooks). ⚠️ see Safety.
- `sm121 power off|on|switch --cluster <c>` — node power via HA. ⚠️ see Safety.
- `sm121 tmux ...` — tmux ops (`serve` uses tmux when `deploy = "tmux"`).
- `sm121 build --recipe <r>` — build/upload the recipe's docker image.
- `sm121 init-host-keys` — seed SSH host keys on nodes.

## Cluster selection

- **Exact**: `--cluster 4nodes-switch`
- **Prefix**: `--cluster 4` → `4nodes-switch` (must be unambiguous)
- **Auto**: `--cluster auto` (or omit) → smallest cluster with NCCL config
  matching `required_nodes`

`serve --solo` overrides cluster selection: first free node across all clusters,
single-node container (no `--nnodes/--tp/--master-addr` — engine defaults).
`--no-solo` forces multi-node even when a recipe sets `solo = true`. `--force`
skips the occupancy check (keeps occupied nodes in the available set; reachable
still required) — dangerous for GPU recipes (OOM / port conflicts).

## Recipe anatomy (`recipes/<name>/recipe.toml`)

```toml
engine = "vllm"                 # or "sglang"
docker_image = "img:tag"
docker_image_dir = "recipe-name" # build dir under docker-images/ (default: recipe name)
model = "org/model"
port = 30000
required_nodes = 2
tensor_parallel_size = 2
pipeline_parallel_size = 1
distributed_backend = "mp"      # or "ray"
shm_size = "32gb"
deploy = "docker"               # or "tmux"
venv = "~/venvs/vllm025"        # tmux only
venv_setup = ["..." ]           # tmux only; runs if venv missing
entrypoint = ["vllm", "serve"]
before_serve = [{ command = "echo", on_failure = "abort" }]
after_stop = [{ script = "restore_dgx_services" }]
args = ["--kv-cache-dtype", "fp8"]

[[extra_mounts]]                # alias | host+container, or repo-based
alias = "jit_cache"             # resolved via [mount_aliases] in config.toml
container = "/root/.cache/jit"
env = "XDG_CACHE_HOME"          # injects -e NAME=<container path>

# repo-based mount (resolves HF cache path per-node at serve time):
[[extra_mounts]]
repo = "org/model"
subfolder = "mtp-draft"         # optional
container = "mtp_draft"

[mods]                          # docker only; runs inside container before engine
commands = ["pip install flash-attn"]

[env]                           # overrides launcher defaults (last wins in -e)
KEY = "val"

[build.versions]                # override pins in docker-images/<dir>/versions.env
VLLM_COMMIT = "00907fc..."
```

`before_serve` / `after_stop` entries: bare string = raw shell command;
`{ command = "...", on_failure = "warn" }`; `{ script = "<alias>" }` references a
built-in script baked into the binary (see **Built-in scripts** in AGENTS.md).

## Config (`config.toml`, gitignored)

```toml
ssh_user, ssh_key_path
[[clusters]] name/master_ib_ip/nodes[{name,ssh_ip,ib_ip,rank,entity}]/extra_entities
[nccl."c"] ib_hca/socket_ifname/ib_gid_index/ib_disable
[home_assistant] url/token/shutdown_delay_seconds
[exec_aliases], [mount_aliases]
[before_serve."c"] commands
```

`[mount_aliases]` lets operators repoint host paths site-wide — the alias wins
over a recipe's `host`.

## Docker image builds

`sm121 serve` builds `docker_image` on demand from `docker-images/<dir>/`
(`<dir>` = `docker_image_dir` or recipe name): uploads the dir to a node, runs
`bash build.sh` in tmux, stamps a `sm121.checksum` label; later serves skip the
build when the label matches a recursive SHA-256 of the dir. Pinned builds use
`versions.env` (every build input as `KEY=VAL`, forwarded as `--build-arg`);
recipe `[build.versions]` overrides win and fold into the checksum. `locks/` =
`--require-hashes` lockfiles, `checksums/` = verified out-of-band downloads.

## Safety (critical)

> ⚠️ **Cluster may be serving live traffic.** A model (often the one serving this
> very agent) may run on the cluster at any time. Before `stop`, `power off`,
> or `power switch`, ALWAYS run `sm121 health --cluster <c>` first and confirm
> with the user that the cluster is not serving. Never assume idle — a stray
> stop/power has killed running agents mid-task before. `logs`, `health`,
> `doctor`, and `-d` dry-runs are always safe.

Known operational constraints (see AGENTS.md → *Serving notes*):
- 0.78 `--gpu-memory-utilization` is the safe ceiling on GB10 for DS4-DSpark;
  0.80 boots but dies under traffic.
- `--max-num-batched-tokens 2048` for interactive agent traffic, 8192 for
  exclusive batch jobs.

## Dev workflow for new work

1. Read the recipe+image pair; if touching engine code, follow the
   `vllm-mod-patches` skill.
2. `cargo build` then `cargo run -- -d serve --recipe <r> --cluster <c>` to
   validate command construction (no SSH).
3. `make check` before commit (fmt → clippy → test → ≤500 lines/file; no
   `#[allow]` — refactor instead; no `use foo::*;`).

## References

- Full command reference & recipe format: `AGENTS.md`
- Architecture / module layout / dev flow: `DEVELOPMENT.md`
- Patch conventions for engine images: `vllm-mod-patches` skill