# First-class engine drivers (`EngineDriver` trait)

## Goal

Make third-party inference engines (`gr8`, `DGPP`, future ones) **first-class
`engine` values** in a recipe — served/stopped/logged/health-probed/benched by
the same `sm121` commands as `vllm`/`sglang` — instead of hand-deployed outside
the launcher. The operator directive: engines like dgpp/gr8 must be
"равноправные члены наравне с vllm/sglang/tmux".

## Current state (the scattered branches this replaces)

`engine` is a free `String` (default `"vllm"`); only `vllm`/`sglang` have any
launcher logic, and it is duplicated across six sites, each branching on
`recipe.engine == "sglang"`:

| Site | Engine-specific behaviour |
|---|---|
| `engine_flags.rs::build_engine_args` | model arg (`--model-path` vs positional), distributed rendezvous flags (vLLM set vs SGLang set), `--headless` for vLLM workers |
| `docker/command.rs::collect_nccl_env` | worker-IP env: `SGLANG_WORKER_IP` vs `VLLM_HOST_IP` |
| `tmux/commands.rs` | entrypoint fallback (`python -m vllm…` / `python -m sglang…`), port on all nodes (SGLang) vs head-only (vLLM) |
| `tmux/mod.rs::build_host_ip_export` | host-IP env var per engine |
| `doctor.rs` | install hint (`pip install vllm` / `sglang`) |
| `publish_render.rs` | SGLang special-case in the rendered bundle |

## Design

Three orthogonal axes, each a first-class concept:

1. **`engine`** → an `EngineDriver` (the inference-engine semantics).
2. **`deploy`** → `Docker` | `Tmux` | **`Native`** (new) — how the process is
   started on a host (container / tmux session / bare SSH command).
3. **`launch_shape`** → `PerNode` (every launch node runs the engine; vLLM,
   SGLang, gr8) | `HeadCoordinated` (rank 0 runs a cluster manager that
   brings up the peers itself; DGPP).

### `EngineDriver` trait (in `src/engine.rs`)

```rust
pub enum LaunchShape { PerNode, HeadCoordinated }

pub trait EngineDriver: Send + Sync {
    fn name(&self) -> &'static str;
    fn launch_shape(&self) -> LaunchShape { LaunchShape::PerNode }
    fn default_deploy(&self) -> DeployTarget { DeployTarget::Docker }

    // Model argument (positional for vLLM, --model-path for SGLang,
    // --model for gr8, none for DGPP — the config file carries it).
    fn model_args(&self, recipe: &Recipe) -> Vec<String>;
    // Distributed rendezvous flags (vLLM/SGLang/gr8 sets; DGPP: none).
    fn distributed_args(&self, ctx: &EngineFlagsCtx, node: &Node) -> Vec<String>;
    // Non-head workers run headless (vLLM yes; SGLang/gr8/DGPP no).
    fn headless_workers(&self) -> bool { false }
    // Env var carrying the node's bind IP (VLLM_HOST_IP / SGLANG_WORKER_IP / none).
    fn engine_ip_env(&self) -> Option<&'static str> { None }
    // Port flag on every node (SGLang) vs head only (vLLM/gr8/DGPP).
    fn port_on_all_nodes(&self) -> bool { false }
    // tmux entrypoint fallback when the recipe sets none.
    fn entrypoint_default(&self) -> Vec<String>;
    // doctor install hint.
    fn install_hint(&self) -> &'static str;
}
```

A registry `driver_for(engine: &str) -> &dyn EngineDriver` maps the recipe's
`engine` string to a driver (`vllm`, `sglang`, `gr8`, `dgpp`; unknown → the
vLLM driver, matching today's default). `build_engine_args` and the five other
sites delegate to it. **Phase 1 is a pure refactor** — vLLM/SGLang drivers
reproduce the current behaviour byte-for-byte, all existing tests stay green.

### `EngineFlagsCtx` gains `peers`

gr8 takes `--peers <ip0>,<ip1>,…` (the ordered launch-node IPs; each rank
finds itself in the list). Add `peers: &[String]` to the ctx (vLLM/SGLang
ignore it).

### gr8 driver (`kindlingai/gr8`)

- `deploy = docker` (image `ghcr.io/kindlingai/gr8`), `launch_shape = PerNode`.
- entrypoint `["/opt/gr8/gr8", "glm53", "up"]`; `model_args` → `--model <repo>`;
  `distributed_args` → `--tp <N> --peers <list>`; no `--headless`, no engine-IP
  env, port on head only (API :8100 on the first peer).
- Recipe carries `--model-cache`, `--drafter`, `--shape` in `args`; the
  `vm.compaction_proactiveness=0` prerequisite is a `before_serve` host hook.
- Stop: label-based `docker stop` works (the container is the engine); the
  engine's own `gr8 glm53 down` is an optional `after_stop`.

### DGPP driver (`HawkBearPig/dgpp`)

- `deploy = native` (new), `launch_shape = HeadCoordinated`.
- The recipe names a DGPP deployment template (`cluster_<model>_<quant>_w<n>.json`)
  shipped next to the recipe; the launcher fills nodes/ssh-user/ports from the
  cluster + a per-recipe `.env`, uploads it to rank 0, and runs
  `scripts/dgpp-cluster up --config <path>` in a tmux session on rank 0 only.
- Readiness: `/v1/models` on :18080 (uniform). Stop:
  `scripts/dgpp-cluster down --config <path>` (an `after_stop` / native stop).
- No CLI model/dist flags — the config JSON carries model + world size + knobs.

## Phasing

1. **Trait + vLLM/SGLang drivers** — pure refactor, six call-sites delegate,
   all tests green. (foundation)
2. **gr8 driver** + `peers` ctx + a `glm53-full-tp4-gr8` recipe; dry-run + serve.
3. **`Native` deploy + HeadCoordinated** + DGPP driver + config-JSON handling +
   a `dgpp` recipe; serve.
4. New-model recipes (Ling-3.1-flash, Step-5-Preview) — separate, gated on the
   bleeding-edge-base decision (nightly/dedicated image, 5–8 Sparks).

## Constraints

- Files ≤500 lines — `engine.rs` holds the trait + registry + the four drivers;
  split drivers into `engine/gr8.rs` etc. if it grows.
- No `#[allow]` for clippy — refactor instead.
- i18n: any new user-facing strings get keys in all six locales.