launcher/specs/tech-architecture/engine-drivers.md

First-class engine drivers (EngineDriver trait)

Goal

Make third-party inference engines (gr8, DGPP, future ones) first-class engine values in a recipe — served/stopped/logged/health-probed/benched by the same sm121 commands as vllm/sglang — instead of hand-deployed outside the launcher. The operator directive: engines like dgpp/gr8 must be "равноправные члены наравне с vllm/sglang/tmux".

Current state (the scattered branches this replaces)

engine is a free String (default "vllm"); only vllm/sglang have any launcher logic, and it is duplicated across six sites, each branching on recipe.engine == "sglang":

Site	Engine-specific behaviour
engine_flags.rs::build_engine_args	model arg (--model-path vs positional), distributed rendezvous flags (vLLM set vs SGLang set), --headless for vLLM workers
docker/command.rs::collect_nccl_env	worker-IP env: SGLANG_WORKER_IP vs VLLM_HOST_IP
tmux/commands.rs	entrypoint fallback (python -m vllm… / python -m sglang…), port on all nodes (SGLang) vs head-only (vLLM)
tmux/mod.rs::build_host_ip_export	host-IP env var per engine
doctor.rs	install hint (pip install vllm / sglang)
publish_render.rs	SGLang special-case in the rendered bundle

Design

Three orthogonal axes, each a first-class concept:

1. engine → an EngineDriver (the inference-engine semantics).
2. deploy → Docker | Tmux | Native (new) — how the process is started on a host (container / tmux session / bare SSH command).
3. launch_shape → PerNode (every launch node runs the engine; vLLM, SGLang, gr8) | HeadCoordinated (rank 0 runs a cluster manager that brings up the peers itself; DGPP).

EngineDriver trait (in src/engine.rs)

pub enum LaunchShape { PerNode, HeadCoordinated }

pub trait EngineDriver: Send + Sync {
    fn name(&self) -> &'static str;
    fn launch_shape(&self) -> LaunchShape { LaunchShape::PerNode }
    fn default_deploy(&self) -> DeployTarget { DeployTarget::Docker }

    // Model argument (positional for vLLM, --model-path for SGLang,
    // --model for gr8, none for DGPP — the config file carries it).
    fn model_args(&self, recipe: &Recipe) -> Vec<String>;
    // Distributed rendezvous flags (vLLM/SGLang/gr8 sets; DGPP: none).
    fn distributed_args(&self, ctx: &EngineFlagsCtx, node: &Node) -> Vec<String>;
    // Non-head workers run headless (vLLM yes; SGLang/gr8/DGPP no).
    fn headless_workers(&self) -> bool { false }
    // Env var carrying the node's bind IP (VLLM_HOST_IP / SGLANG_WORKER_IP / none).
    fn engine_ip_env(&self) -> Option<&'static str> { None }
    // Port flag on every node (SGLang) vs head only (vLLM/gr8/DGPP).
    fn port_on_all_nodes(&self) -> bool { false }
    // tmux entrypoint fallback when the recipe sets none.
    fn entrypoint_default(&self) -> Vec<String>;
    // doctor install hint.
    fn install_hint(&self) -> &'static str;
}

A registry driver_for(engine: &str) -> &dyn EngineDriver maps the recipe's engine string to a driver (vllm, sglang, gr8, dgpp; unknown → the vLLM driver, matching today's default). build_engine_args and the five other sites delegate to it. Phase 1 is a pure refactor — vLLM/SGLang drivers reproduce the current behaviour byte-for-byte, all existing tests stay green.

EngineFlagsCtx gains peers

gr8 takes --peers <ip0>,<ip1>,… (the ordered launch-node IPs; each rank finds itself in the list). Add peers: &[String] to the ctx (vLLM/SGLang ignore it).

gr8 driver (kindlingai/gr8)

• deploy = docker (image ghcr.io/kindlingai/gr8), launch_shape = PerNode.
• entrypoint ["/opt/gr8/gr8", "glm53", "up"]; model_args → --model <repo>; distributed_args → --tp <N> --peers <list>; no --headless, no engine-IP env, port on head only (API :8100 on the first peer).
• Recipe carries --model-cache, --drafter, --shape in args; the vm.compaction_proactiveness=0 prerequisite is a before_serve host hook.
• Stop: label-based docker stop works (the container is the engine); the engine's own gr8 glm53 down is an optional after_stop.

DGPP driver (HawkBearPig/dgpp)

• deploy = native (new), launch_shape = HeadCoordinated.
• The recipe names a DGPP deployment template (cluster_<model>_<quant>_w<n>.json) shipped next to the recipe; the launcher fills nodes/ssh-user/ports from the cluster + a per-recipe .env, uploads it to rank 0, and runs scripts/dgpp-cluster up --config <path> in a tmux session on rank 0 only.
• Readiness: /v1/models on :18080 (uniform). Stop: scripts/dgpp-cluster down --config <path> (an after_stop / native stop).
• No CLI model/dist flags — the config JSON carries model + world size + knobs.

Phasing

1. Trait + vLLM/SGLang drivers — pure refactor, six call-sites delegate, all tests green. (foundation)
2. gr8 driver + peers ctx + a glm53-full-tp4-gr8 recipe; dry-run + serve.
3. Native deploy + HeadCoordinated + DGPP driver + config-JSON handling + a dgpp recipe; serve.
4. New-model recipes (Ling-3.1-flash, Step-5-Preview) — separate, gated on the bleeding-edge-base decision (nightly/dedicated image, 5–8 Sparks).

Constraints

• Files ≤500 lines — engine.rs holds the trait + registry + the four drivers; split drivers into engine/gr8.rs etc. if it grows.
• No #[allow] for clippy — refactor instead.
• i18n: any new user-facing strings get keys in all six locales.