Browse Source

docs(AGENTS): FLARE engine deploy, orbit/controller mapping and the two-store disk budget

Alexander Korolev 1 day ago
parent
commit
a1d58a7374
1 changed files with 24 additions and 0 deletions
  1. 24 0
      AGENTS.md

+ 24 - 0
AGENTS.md

@@ -416,6 +416,30 @@ The `deploy` field selects how the recipe is launched on each node:
   the same pin skips the build (the native analog of the docker image build +
   the same pin skips the build (the native analog of the docker image build +
   checksum label). No `repo`/`rev` → assume pre-installed.
   checksum label). No `repo`/`rev` → assume pre-installed.
 
 
+**`engine = "flare"` (native deploy)** serves a *prepared, signed package* through
+`flare orbit`: `serve` preflights the agents, runs `flare download` on the
+controller, then `flare orbit run <controller>:<package> --detach` and waits for
+`/health` + `/v1/models` on `[flare].http_port` (default 30000). Two node
+questions are config, not the recipe: `[flare.clusters.<our-cluster>]` maps our
+cluster to an orbit (`orbit_cluster`, `orbit_nodes`, `controller_node`,
+`local_dir`, `extra_flags`), and **`controller_node` pins the machine that runs
+the CLI** — it is the orbit's controller (its key is what the agents trust, its
+disk holds the package), which is *not* our rank 0 once the orbit is re-paired
+or reordered. `orbit_nodes` is listed in the orbit's rank order and is matched
+by node name (`gx10-dgx4` ↔ `dgx4.lan`), never by our rank index.
+
+Two stores, both budgeted: `flare download` fills the **CLI store**
+(`paths.package_cache`), and `orbit run` then stages into each node's **agent
+cache** (`~/.cache/flare/packages/<sha>-<size>/`, *not* movable by
+`paths.package_cache`). A `format 2` package is a blob store — `blobs/<sha>`
+holds the data once, `models/<version>/…` are hardlinks into it — so mirroring
+must use `rsync -aH` over the **store root** (never `--delete` it: it holds
+every other prepared package). Measured on the 488 GB gb10 package: rank 0
+(seed) holds 488 GB, ranks 1–3 hold 267 GB each (183 GB shared Engram + own
+slice), and a controller that is also rank 0 pays for both stores (976 GB) —
+so a 916 GB Spark is a fine rank, but the controller wants the big disk.
+Details and the re-pair procedure: `specs/tech-architecture/flare-bringup.md`.
+
 **First-class engines** (`engine` selects an `EngineDriver` in
 **First-class engines** (`engine` selects an `EngineDriver` in
 `src/engine/{mod,drivers}.rs`): `vllm` (default), `sglang`, `gr8`
 `src/engine/{mod,drivers}.rs`): `vllm` (default), `sglang`, `gr8`
 (`kindlingai/gr8`, docker deploy, `--tp/--peers` rendezvous, fixed API :8100),
 (`kindlingai/gr8`, docker deploy, `--tp/--peers` rendezvous, fixed API :8100),