# AGENTS.md — sm121

Rust CLI (`edition 2024`) for launching vLLM/SGLang inference containers across a DGX Spark cluster via SSH (`russh`). Docker default, tmux alternative.

## Dev quick-ref

| Action | Command |
|--------|---------|
| Build | `cargo build` |
| All gates | `make check` (fmt → clippy → test → ≤500 lines) |
| Dry-run | `cargo run -- -d serve --recipe deepseek-v4-flash` |
| Tests | `cargo test` (65, all passing) |
| l10n | `locales/{en,ru,es,it,fr,de}/main.ftl` — Fluent FTL files |

See [`DEVELOPMENT.md`](DEVELOPMENT.md) for architecture, module layout, and dev workflow.

## Commands

```
serve/stop/logs/health/doctor/exec/power/tmux/init-host-keys
```

Cluster: exact name, prefix (`4`→`4nodes-switch`), or `auto` (smallest with NCCL + matching `required_nodes`).

## Recipe format (`recipes/<name>/recipe.toml`)

```toml
engine = "vllm"                     # or "sglang"
docker_image = "img:tag"
model = "org/model"
port = 30000                        # default
required_nodes = 2                  # optional
tensor_parallel_size = 2
pipeline_parallel_size = 1          # vLLM only, 0 = auto
distributed_backend = "mp"          # or "ray"
shm_size = "64gb"                   # optional, only set when non-empty
entrypoint = ["vllm", "serve"]      # optional, after image
before_serve = [{ command = "echo", on_failure = "abort" }]
args = ["--kv-cache-dtype", "fp8"]

[[extra_mounts]]                    # extra bind-mounts for `docker run -v`
alias = "jit_cache"                # resolves via [mount_aliases] in config.toml; wins over `host`
host = "~/.cache/jit"              # fallback default if alias absent; `~` expanded to ssh_user home
container = "/root/.cache/jit"     # container-side target (required)
read_only = false                 # optional, default false (adds :ro)
env = "XDG_CACHE_HOME"            # optional: injects -e <name>=<container path>

[mods]
commands = [                         # run *inside* container before engine
    "pip install flash-attn",
    { command = "./patch.sh", on_failure = "warn" },
]

[env]
KEY = "val"                         # overrides launcher defaults
```

## Env vars (auto-set)

Cluster-driven: `NCCL_IB_DISABLE/HCA/SOCKET_IFNAME/GID_INDEX`, `GLOO_/TP_SOCKET_IFNAME`, `MN_IF_NAME`, `UCX_NET_DEVICES`, `OMPI_MCA_btl_tcp_if_include`
Defaults: `OMP_NUM_THREADS=4`, `NCCL_IGNORE_CPU_AFFINITY=1`, `NCCL_NET=IB`, `NCCL_DEBUG=WARN`
Engine IP: `VLLM_HOST_IP` / `SGLANG_WORKER_IP`
Head: `VLLM_LOGGING_COLOR=1`, `TERM=xterm-256color`, `-p <port>:<port>`
Ray: `RAY_ADDRESS/PORT/NODE_IP_ADDRESS/OVERRIDE_NODE_IP_ADDRESS`
Recipe `[env]` wins (last wins in `-e`).

## Mods

Mods run **inside** the container before the engine process via entrypoint
wrapping.  When `[mods]` is present, the recipe **must** also set `entrypoint`
(the known process to `exec` at the end).  Same `on_failure` semantics as
`before_serve` (default: `abort`).  On failure the container exits; configure
`docker run --restart` externally to restart on crash.

Implementation: `--entrypoint sh image -c 'mods && exec <original entrypoint>'`

## Flags auto-generated

- vLLM: `--tensor-parallel-size`, `--pipeline-parallel-size`, `--nnodes`, `--node-rank`, `--master-addr`, `--master-port`, `--host`, `--port`, `--headless` (workers)
- SGLang: `--tp`, `--nnodes`, `--node-rank`, `--dist-init-addr`, `--host`, `--port`
- Both: `--distributed-executor-backend` (ray only, mp is auto-detected)

## Config fields

```toml
ssh_user = "u"
ssh_key_path = "/path"              # PathBuf

[[clusters]]
name = "c"
master_ib_ip = "10.0.0.1"
nodes = [{ name = "n", ssh_ip = "...", ib_ip = "...", rank = 0, entity = "switch.n" }]
extra_entities = ["switch.network"] # cluster-level HA entities

[nccl."c"]
ib_hca = "..."
socket_ifname = "..."
ib_gid_index = 3
ib_disable = 0

[home_assistant]
url = "..."
token = "..."
shutdown_delay_seconds = 30

[exec_aliases]
name = "command"

[mount_aliases]                     # host paths referenced by name from recipe [[extra_mounts]]
jit_cache = "/mnt/nas/ai/jit"      # alias wins over recipe `host`; lets operators repoint mounts site-wide
hf_cache = "/mnt/nas/ai/huggingface"

[before_serve."c"]
commands = ["cmd", { command = "cmd", on_failure = "warn" }]
```

## Power

- `off`: SSH shutdown → wait → HA `turn_off` per-node entities + cluster extras
- `on`: HA `turn_on` (no SSH, requires HA config)
- `switch --from <src> --to <tgt>`: disable nodes/extras unique to `from`, enable all in `to`

## i18n

`fluent-i18n` crate, auto-detect via `set_locale(None)`. FTL in `locales/`.

## Constraints

- Files ≤800 lines unless impractical; extract submodules when a file exceeds 700 lines (`make check-lines`)
- `#![deny(clippy::wildcard_imports)]` — no `use foo::*;`
- **Clippy warnings**: solve by refactoring the code (extract structs, reduce arg count, restructure logic). Do **not** add `#[allow(...)]` attributes. If the only viable fix is an allow attribute, consult the user for guidance.