|
|
@@ -241,7 +241,19 @@ serve is verification-only. `repos` defaults to the recipe's model + repo
|
|
|
path-refs + composed sources; `skip` leaves external/managed repos alone;
|
|
|
`min_free` gates the whole pass per node (`G`/`T`/`%`). `--stage=skip`
|
|
|
assumes staged (CI/known-good), `--stage=force` re-verifies everything.
|
|
|
-`kind = "sparse-rows"` (spec P2) packs each launch rank's slice of the
|
|
|
+`kind = "local-view"` keeps a **hot side table** local while the bulk of the
|
|
|
+weights stays on shared storage: files matching `local` (globs) become real
|
|
|
+node-local copies, everything else becomes a symlink into the shared HF cache's
|
|
|
+container path; `serve` mounts the view at `/root/.cache/hf-view` and rewrites
|
|
|
+`model` to `/root/.cache/hf-view/<repo>`. Use it when a runtime streams a big
|
|
|
+table at every token — the 1017 Engram tables are read at ~1 MB/token, and
|
|
|
+staging the two Engram shards locally took that lane's prefill from 750 to
|
|
|
+**3,957 t/s** (measured 2026-10-23; shared-mount reads 0 MB during a run that
|
|
|
+used to pull 258 MB per short completion). Verification is a content-hash
|
|
|
+ledger, so a second serve is verification-only and an operator's pre-rsync'd
|
|
|
+tree is adopted. `preflight` measures every launch node **before** anything is
|
|
|
+copied or started and refuses with node/repo/size when one is short — a 200 GB
|
|
|
+copy must not die halfway. `kind = "sparse-rows"` (spec P2) packs each launch rank's slice of the
|
|
|
Engram tables onto node-local NVMe (`dst`, ledger of sampled row hashes in
|
|
|
the dst dir); rank/tp derive from the cluster and recipe — row ranges are
|
|
|
never hardcoded. The runtime-visible dir must be bind-mounted by the
|
