# SGLang Support Implementation Plan

## Background

`vllm-spark-launcher` is a Rust CLI for launching vLLM/SGLang containers across DGX Spark GPU clusters via SSH. The current SGLang support is incomplete — it only changes the engine name but uses identical docker commands as vLLM. SGLang has different defaults and CLI flags that need proper handling.

## Problem

Current gaps:
1. `VLLM_HOST_IP` is always set — SGLang uses `SGLANG_WORKER_IP` instead
2. No SGLang-specific CLI flags (`--tp`, `--dp`, `--pp` vs vLLM's `--tensor-parallel-size`)
3. No distributed backend selection (Ray vs MP) — currently hardcoded to MP
4. No way to distinguish SGLang-specific distributed config from vLLM's
5. Container naming doesn't reflect engine type (always uses recipe name)
6. No health check/repair infrastructure for container issues

## Implementation Plan

### Phase 1: Recipe & Config Changes

**r1. Add `distributed_backend` field to Recipe struct**
- Add `distributed_backend: String` with default `"mp"` (matches current behavior)
- Supported values: `"mp"` (default), `"ray"`
- This controls docker flag composition in `build_docker_command`

**r2. Add `distributed_backend` to recipe TOML schema**
- Document in AGENTS.md: `distributed_backend = "mp"` (or `"ray"`)
- Ray mode: `--tensor-parallel-size=<nnodes>` + `RAY_ADDRESS` env var on worker nodes
- MP mode: `--tensor-parallel-size=1` + `--pipeline-parallel-size` + `--nnodes` + `--node-rank` + `--master-addr` + `--distributed-executor-backend=mp`

### Phase 3: SGLang-Specific Docker Command Composition

**r6. Update `build_docker_command` for Ray/MP backend switch**
- When `distributed_backend == "ray"`:
  - Head node: normal vLLM/SGLang command with `--tensor-parallel-size=<nnodes>`
  - Worker nodes: `--tensor-parallel-size=<nnodes>` + `RAY_ADDRESS=<head_ip>` env var
  - No `--nnodes`, `--node-rank`, `--master-addr`, `--pipeline-parallel-size`, `--distributed-executor-backend` flags
- When `distributed_backend == "mp"` (current behavior):
  - All nodes: `--tensor-parallel-size=1` + `--pipeline-parallel-size=<computed>` + `--nnodes` + `--node-rank` + `--master-addr` + `--distributed-executor-backend=mp`

**Environment variable changes:**
- If engine is `"vllm"`: set `VLLM_HOST_IP`
- If engine is `"sglang"`: set `SGLANG_WORKER_IP` (instead of `VLLM_HOST_IP`)

### Phase 4: CLI Enhancements

**r4. Implement `doctor` subcommand** (checks node readiness + container state)
- Concurrent node checks: `docker --version`, `nvidia-smi`, NCCL env var validation
- Container checks: `docker ps --filter label=spark-launcher`, engine health endpoints, Ray cluster status
- Output: pass/fail per node, actionable suggestions

**r5. Implement `--repair` flag** (builds `.repaired` image on head node)
- Generates Dockerfile fixing common issues (missing deps, wrong entrypoint)
- Builds `<image>.repaired` on head node, tags it
- `serve` auto-detects and prefers `<image>.repaired` if it exists

### Phase 5: Wire Up & Testing

**r3. Wire up doctor handler in `main()`**

**r7. Update `serve()` to prefer repaired images**
- Before build_docker_command, check if `<image>.repaired` exists
- If yes, use it instead (log it)

**r8. Integration testing**
- `cargo build` succeeds
- Dry-run with `--recipe glm51 -d` shows correct flags for both engines
- Dry-run with `distributed_backend = "ray"` shows Ray-specific flags
- Health check on empty cluster

**r9. Update AGENTS.md**
- Document new recipe fields, commands, and behavior changes

## File Changes

| File | Changes |
|------|---------|
| `src/main.rs` | Recipe struct, `distributed_backend`, env var logic, `build_docker_command` rewrite, `doctor` handler, `repair` handler |
| `recipes/*/recipe.toml` | Add `distributed_backend` field (optional, defaults to `mp`) |
| `AGENTS.md` | Document new fields, commands, SGLang behavior |

## Risks & Pitfalls

- **NEVER use `write_file` on `src/main.rs`** — accidentally wiped it before. Always use `patch()` for incremental changes.
- Rust 2015 linter noise (`E0670 async fn not permitted`) — ignore, `Cargo.toml` uses edition 2024.
- SGLang command-line flags differ significantly from vLLM — `--tp`/`--dp`/`--pp` vs `--tensor-parallel-size`/`--pipeline-parallel-size`.
- Ray mode requires head node to start first (Ray GCS), then workers join. May need sequential startup for Ray.