صفحهٔ اصلی گشت‌و‌گذار راهنما
ورود
sm121
/
launcher
2
0
انشعاب 0
پرونده‌ها مشکلات 0 درخواست واکشی 0 ویکی
درخت: 2d379f9752
شاخه‌ها تگ‌ها
launcher
/
DEVELOPMENT.md

DEVELOPMENT.md 5.3 KB
تاريخچه خام

Development — sm121-launcher

Rust CLI tool for launching vLLM/SGLang containers across a DGX Spark cluster. SSH-based provisioning over russh.

Project Layout

Cargo.toml                 # Binary crate, Edition 2024
config.toml                # User config (gitignored)
locales/                   # Fluent i18n files (en, ru, es, it, fr, de)
recipes/                   # Recipe directories (<name>/recipe.toml)
src/
├── main.rs                # CLI entrypoint, command dispatch, i18n init
├── cli.rs                 # clap derive CLI definitions
├── config.rs              # Config/Recipe deserialization, cluster resolution
├── ssh.rs                 # SSH client, key loading, host key verification
├── docker/
│   ├── mod.rs             # Docker command builder: serve, stop, health, logs
│   └── tests.rs           # Command builder tests
├── tmux/
│   ├── mod.rs             # Tmux module root
│   ├── serve.rs           # Tmux serve (window/panel layout)
│   └── ops.rs             # Tmux stop/status/logs/attach
├── power.rs               # Power on/off/switch flows
├── home_assistant.rs      # HA REST API client
├── host_keys.rs           # init-host-keys command
├── exec.rs                # Exec subcommand + alias resolution
├── doctor.rs              # Doctor mode + --repair
├── pre_serve.rs           # PreServeCommand types
├── before_serve.rs        # Pre-serve hook execution
└── health.rs              # Tmux health-check helpers

Quality Gates

make check            # All gates: fmt → clippy → test → line-count
make fmt              # cargo fmt
make check-fmt        # cargo fmt --check
make check-clippy     # cargo clippy --all-targets -- -D warnings
make check-test       # cargo test
make check-lines      # File-size ≤500 lines per src/**/*.rs

Architecture

Docker launch flow

  1. serve reads the recipe TOML
  2. Resolves cluster (exact/prefix match → smallest with NCCL + matching node count)
  3. Runs pre-serve hooks (cluster-level + recipe-level)
  4. Checks for repaired image (via doctor --repair)
  5. For each node: builds docker run command via build_docker_command(), runs via SSH
  6. Waits 10s, then shows tail logs

build_docker_command() in src/docker/mod.rs is engine-aware:

Aspect vLLM SGLang
Host IP env var VLLM_HOST_IP SGLANG_WORKER_IP
Model argument bare name --model-path
TP flag --tensor-parallel-size --tp
PP flag --pipeline-parallel-size —
Backend flag --distributed-executor-backend (ray only) —
Init addr --master-addr + --master-port --dist-init-addr
Workers --headless —

Auto-generated flags: --host 0.0.0.0, --port <recipe.port>, --nnodes, --node-rank.

Cluster selection

  • Exact match: --cluster 4nodes-switch
  • Prefix match: --cluster 4 → resolves to 4nodes-switch (unique prefix, fails on ambiguous)
  • Auto: --cluster auto (or omitted) → picks the smallest cluster with NCCL config matching required_nodes

Env var precedence

  1. Cluster-driven (from [nccl]): NCCL_IB_DISABLE, NCCL_IB_HCA, NCCL_SOCKET_IFNAME, etc.
  2. Network interface (from nccl.socket_ifname): MN_IF_NAME, UCX_NET_DEVICES, OMPI_MCA_btl_tcp_if_include
  3. Defaults: OMP_NUM_THREADS=4, NCCL_IGNORE_CPU_AFFINITY=1, NCCL_NET=IB, NCCL_DEBUG=WARN
  4. Engine IP: VLLM_HOST_IP / SGLANG_WORKER_IP
  5. Head node: VLLM_LOGGING_COLOR=1, TERM=xterm-256color, -p <port>:<port>
  6. Ray backend: RAY_ADDRESS, RAY_PORT, RAY_NODE_IP_ADDRESS, RAY_OVERRIDE_NODE_IP_ADDRESS
  7. Recipe [env]: appended last, overrides everything above

Power management

  • power off: SSH sudo shutdown now → wait shutdown_delay_seconds → HA turn_off per-node entities + cluster extras
  • power on: HA turn_on per-node entities + cluster extras (no SSH)
  • power switch: disable nodes/extras unique to from → enable all in to

i18n

Mozilla Fluent via fluent-i18n crate. Locale auto-detected from system. FTL files in locales/{lang}/main.ftl.

Module size limit

Every src/**/*.rs file must stay ≤500 lines. Enforced by make check-lines.

Dev Workflow

# Build
cargo build

# Run locally with dry-run (no SSH execution)
cargo run -- -d serve --recipe deepseek-v4-flash

# Run tests
cargo test

# All gates (must pass before commit)
make check

# Add a new recipe
mkdir -p recipes/my-model/
# create recipes/my-model/recipe.toml
cargo run -- -d serve --recipe my-model --cluster <name>

# Add a new locale
cp locales/en/main.ftl locales/xx/main.ftl
# translate strings, commit

Dependencies

Crate Version Purpose
clap 4.6 CLI parsing (derive)
russh 0.61 SSH client
russh-keys 0.50-beta SSH key loading
tokio 1 (full) Async runtime
toml 1 Config deserialization
serde 1 Serialization
tracing 0.1 Logging
anyhow 1.0 Error handling
futures 0.3 Concurrent SSH
sha2 0.11 Host key fingerprints
base64 0.22 Base64 encoding
reqwest 0.13 HTTP client (HA)
serde_json 1 JSON (HA)
dirs 6 OS config dir
fluent-i18n — Mozilla Fluent i18n