name: sm121-launcher

description: Operate the sm121 Rust CLI launcher for this repo — serve/stop/logs/health/doctor/power/tmux on the DGX Spark cluster, cluster+recipe selection, image builds, and dev gates. Use whenever working with sm121 commands, recipes, config, or the DGX Spark cluster.

sm121 Launcher

sm121 is the Rust CLI (edition 2024) that launches vLLM/SGLang engine containers across the DGX Spark cluster over SSH (russh, Docker default, tmux alternative). A model may be serving live traffic at any time — read the Safety section before anything destructive.

Quick reference

Action	Command
Build	`cargo build`
All gates (before commit)	`make check`
Dry-run (safe, no SSH)	`cargo run -- -d serve --recipe <name>`
Tests	`cargo test` (65)
l10n	`locales/{en,ru,es,it,fr,de}/main.ftl`

Commands

serve / stop / logs / health / doctor / exec / power / tmux / init-host-keys.

sm121 health --cluster <c> — safe; ALWAYS run before stop/power (see Safety).
sm121 logs --recipe <r> [--cluster <c>] — safe.
sm121 doctor [--repair] — inspect/repair node+image state; safe.
sm121 exec --recipe <r> -- <cmd> — run a command in the serving container; honors exec_aliases from config.toml.
sm121 serve --recipe <r> [--cluster <c>] [--keep] [--follow] — main launch; --keep keeps the container after engine exit, --follow tails logs.
sm121 stop --recipe <r> [--cluster <c>] — teardown (Docker stop/rm, or tmux session kill; runs after_stop hooks). ⚠️ see Safety.
sm121 power off|on|switch --cluster <c> — node power via HA. ⚠️ see Safety.
sm121 tmux ... — tmux ops (serve uses tmux when deploy = "tmux").
sm121 build --recipe <r> — build/upload the recipe's docker image.
sm121 init-host-keys — seed SSH host keys on nodes.

Cluster selection

Exact: --cluster 4nodes-switch
Prefix: --cluster 4 → 4nodes-switch (must be unambiguous)
Auto: --cluster auto (or omit) → smallest cluster with NCCL config matching required_nodes

serve --solo overrides cluster selection: first free node across all clusters, single-node container (no --nnodes/--tp/--master-addr — engine defaults). --no-solo forces multi-node even when a recipe sets solo = true. --force skips the occupancy check (keeps occupied nodes in the available set; reachable still required) — dangerous for GPU recipes (OOM / port conflicts).

Recipe anatomy (`recipes/<name>/recipe.toml`)

engine = "vllm"                 # or "sglang"
docker_image = "img:tag"
docker_image_dir = "recipe-name" # build dir under docker-images/ (default: recipe name)
build_profile = "profile-name"  # optional: docker-images/<dir>/profiles/<name>.env
#   merged over the image versions.env (image < profile < [build.versions])
#   and folded into the build checksum — a rev's pin set in one file.
#   MUST be a top-level key: after [[extra_mounts]] it silently lands in the
#   last mount table (TOML context) and is ignored.
model = "org/model"
port = 30000               # the only port convention — when adapting a foreign
                             # recipe, drop/rewrite its port (8000/8888/…) to this default
required_nodes = 2
tensor_parallel_size = 2
pipeline_parallel_size = 1
distributed_backend = "mp"      # or "ray"
shm_size = "32gb"
deploy = "docker"               # or "tmux"
venv = "~/venvs/vllm025"        # tmux only
venv_setup = ["..." ]           # tmux only; runs if venv missing
entrypoint = ["vllm", "serve"]
before_serve = [{ command = "echo", on_failure = "abort" }]
after_stop = [{ script = "restore_dgx_services" }]
args = ["--kv-cache-dtype", "fp8"]

[[extra_mounts]]                # alias | host+container, or repo-based
alias = "jit_cache"             # resolved via [mount_aliases] in config.toml
container = "/root/.cache/jit"
env = "XDG_CACHE_HOME"          # injects -e NAME=<container path>

# repo-based mount (resolves HF cache path per-node at serve time):
[[extra_mounts]]
repo = "org/model"
subfolder = "mtp-draft"         # optional
container = "mtp_draft"

[mods]                          # docker only; runs inside container before engine
commands = ["pip install flash-attn"]

[env]                           # overrides launcher defaults (last wins in -e)
KEY = "val"

build_profile = "glm53-r28"     # named pin set: docker-images/<dir>/profiles/<name>.env

[build.versions]                # override pins in docker-images/<dir>/versions.env
VLLM_COMMIT = "00907fc..."      # precedence: versions.env < profile < recipe

before_serve / after_stop entries: bare string = raw shell command; { command = "...", on_failure = "warn" }; { script = "<alias>" } references a built-in script baked into the binary (see Built-in scripts in AGENTS.md).

Config (`config.toml`, gitignored)

ssh_user, ssh_key_path
[[clusters]] name/master_ib_ip/nodes[{name,ssh_ip,ib_ip,rank,entity}]/extra_entities
[nccl."c"] ib_hca/socket_ifname/ib_gid_index/ib_disable
[home_assistant] url/token/shutdown_delay_seconds
[exec_aliases], [mount_aliases]
[before_serve."c"] commands

[mount_aliases] lets operators repoint host paths site-wide — the alias wins over a recipe's host.

Docker image builds

sm121 serve builds docker_image on demand from docker-images/<dir>/ (<dir> = docker_image_dir or recipe name): uploads the dir to a node, runs bash build.sh in tmux, stamps a sm121.checksum label; later serves skip the build when the label matches a recursive SHA-256 of the dir. Pinned builds use versions.env (every build input as KEY=VAL, forwarded as --build-arg); recipe build_profile = "<name>" (a profiles/<name>.env pin set, e.g. an engine generation: vLLM + b12x + LMCache cut together) and [build.versions] overrides win over it, all folding into the checksum. Adopt revs as ONE profile across all forks — never advance vLLM alone (the 2026-09-08 half-cut lesson; see the jovian-images skill). locks/ = --require-hashes lockfiles, checksums/ = verified out-of-band downloads.

Safety (critical)

⚠️ Cluster may be serving live traffic. A model (often the one serving this very agent) may run on the cluster at any time. Before stop, power off, or power switch, ALWAYS run sm121 health --cluster <c> first and confirm with the user that the cluster is not serving. Never assume idle — a stray stop/power has killed running agents mid-task before. logs, health, doctor, and -d dry-runs are always safe.

Known operational constraints (see AGENTS.md → Serving notes):

0.78 --gpu-memory-utilization is the safe ceiling on GB10 for DS4-DSpark; 0.80 boots but dies under traffic.
--max-num-batched-tokens 2048 for interactive agent traffic, 8192 for exclusive batch jobs.

Dev workflow for new work

Read the recipe+image pair; if touching engine code, follow the vllm-mod-patches skill.
cargo build then cargo run -- -d serve --recipe <r> --cluster <c> to validate command construction (no SSH).
make check before commit (fmt → clippy → test → ≤500 lines/file; no #[allow] — refactor instead; no use foo::*;).

References

Full command reference & recipe format: AGENTS.md
Architecture / module layout / dev flow: DEVELOPMENT.md
Patch conventions for engine images: vllm-mod-patches skill

SKILL.md 7.3 KB Histórico Raw