# Changelog

Semantic versioning for the launcher CLI (`sm121`). The add-on image has its
own version and its own `sm121/CHANGELOG.md`; do not mix them.

## 0.2.5 — 2026-10-29

- **Image sync now waits for the unpack.** Every DGX Spark node uses the
  containerd snapshotter (`io.containerd.snapshotter.v1`), whose `docker load`
  returns before the rootfs snapshot is materialized — a `docker run` issued
  seconds after the sync can die with "unable to start container process …
  stat …: no such file" (dgx3 lost exactly that race on the first Mistral
  Large 4 serve: rank 2 died silently, `--rm` removed the evidence, and the
  rendezvous hung until its 30-min timeout). The sync now probes exec
  readiness on each peer (a trivial `/bin/sh` container start, retried up to
  2 min) before declaring the sync done; a serve refuses to launch against a
  half-materialized image.
- Also on dgx3-class races: the container's `docker run` no longer races the
  image sync's last hop — sync-image OK now means exec-ready, not just
  digest-present.

## 0.2.4 — 2026-10-29

- **Liveness checks are cached and less aggressive.** Entity states are kept
  process-wide (HA 120 s, RouterOS 300 s); the dashboard's 30 s poll no longer
  re-authenticates every switch or re-fetches every HA entity on each tick.
  A power action invalidates the cache so the next snapshot is fresh.
- **Switch liveness is a plain TCP connect, not a RouterOS login.** `health`
  and the dashboard only need to know whether the switch is reachable; the
  RouterOS login was unnecessary API load. `power off` still logs in and sends
  `/system/shutdown`.
- **Bench log belongs to the current job.** `GET /api/bench/log` follows the
  recipe's newest bench job and only accepts a log written at/after its start;
  a previous campaign is labelled `previous`, and a fresh launch shows
  `waiting` instead of a stale log.
- **Job ids survive a web restart.** The id counter resumes past persisted
  `j<hex>` ids, and `finish_job` updates the still-running job with that id,
  not an older finished one.
- **Bench append semantics.** `run_tool` always appends; `bench::run` truncates
  the log once and writes launcher progress notes, so the dashboard window sees
  readiness/notes and the depth suite shares one report log.
- **Remote bench forces a pty.** `ssh -tt` avoids the "Pseudo-terminal will not
  be allocated" path when the launcher has no terminal.
- **Upstream watch** uses 100 commits and follows redirects.

## 0.2.3 — 2026-10-28

- **FLARE topology/LED board: every rank node is busy.** `flare orbit status`
  rows carry the orbit node and rank; the probe/`health` fold one pseudo-container
  onto **each** rank node, not just the controller. A 4-rank FLARE serve now
  shows 4 busy nodes and `free 0`, not one busy node and three idle.
- **Shared-node dedup.** A group is visible in `all8`/`pair12` too because those
  paired clusters share nodes; the lane is owned by the cluster whose node count
  and the recipe's `required_nodes` match the group's rank rows. Duplicate
  pseudo-containers on a shared node are collapsed by `(node, group)`.
- **FLARE logs.** Web log windows detect `engine = "flare"` and stream
  `flare orbit attach <group> --cluster <orbit>` on the controller (not Docker
  logs). The CLI does the same and prints both SSH stdout/stderr; `run_ssh`'s
  5 s inactivity timeout had killed the attach stream.
- `GET /api/logs` history and attach keys work for FLARE; the recipe-row `▤ logs`
  opens the controller's orbit session.

## 0.2.2 — 2026-10-28

- **Dashboard bench window.** The recipes panel's `⚡ bench` button now opens a
  bench window instead of firing the standard run blindly: choose recipe,
  cluster, target node (`node`), run host (`ssh_host`, e.g. `pluto.lan`),
  label, seed, concurrency, parallel, timeout, tokenizer, perf-only, depth and
  no-think. The choices persist in `localStorage` (`sm121-bench-params`).
- **Live harness log.** `GET /api/bench/log?recipe=…` tail-reads the newest
  bench log for the recipe; the window polls it while the run starts so early
  failures (readiness, tokenizer, harness) are visible before a long wait.
- **Cancel.** `POST /api/bench/cancel` sets the job's cancellation flag;
  `run_tool` polls it and kills the `tool-eval-bench` child (also the SSH child
  for remote runs), so the job finishes failed and the log stops.
- **Reopen.** The bench window is a named window: closing it leaves a dock chip,
  the dock menu has `bench…`, and the last recipe/job is restored on reload.
- Bench API now accepts `node`, `seed`, `concurrency`, `parallel`, `timeout`,
  `tokenizer`, `no_think`; the report row's bench button posts to the parent
  dashboard and opens the window.

## 0.2.1 — 2026-10-28

- **FLARE cluster identification.** A cluster is only paired with an orbit if
  the private config names it (`[flare.clusters.<our-cluster>]`). The flat
  `orbit_cluster` is a fallback for `serve`, not for the dashboard/`health`: our
  clusters share nodes (`dgx1` is rank 0 of prod4, 6nodes-switch, 2nodes-dac,
  qwen38 and 1node-1), so probing an unpaired cluster with `sparks` made the
  sparks lane look like that cluster's lane. `serve` now refuses an unpaired
  FLARE cluster instead of silently launching onto the flat default.
- **Dashboard shows the orbit.** Each cluster head carries a `◆ prod4` badge
  (orbit + controller + live groups; dimmed when nothing runs), and the topology
  frame/chip labels include the orbit. `health` folds only the selected cluster's
  lane, not every lane that shares its controller.
- **Prepared-package identification.** After a successful mirror the controller
  writes `.sm121-flare-package.json` (repo, revision, TP, package version,
  local `model.safetensors`). A serve with a matching marker — checked against
  the CURRENT `flare models list --json` version, so a lab re-cut under the same
  revision is detected — skips both `flare download` verification and the
  500 GB copy.
- **Incremental mirror with `rsync`.** The local mirror uses
  `rsync -a --delete` (marker excluded), so a changed/stale package copies only
  the changed files; a partial mirror resumes instead of being wiped and
  re-copied. Download/mirror/orbit launch all use long-running SSH (the 5 s
  inactivity timeout used to kill a 500 GB copy or a detached launch).
- **Settings editor.** `POST /api/settings/extra/check` validates without
  writing; parse errors report line/column for the textarea; `Ctrl+Enter`
  saves. The private block stays in `/data/config-extra.toml`, edited in the
  dashboard's settings window.

## 0.2.0 — 2026-10-28

- FLARE: cached `flare download` now resolves the prepared package through
  `flare models list --json` instead of failing on the metadata-only
  "Resolved package" line.
- FLARE: `flare download` gets `--tensor-parallel` from the recipe's launch
  size; a TP4 package is no longer rejected as "TP size 1".
- FLARE: local mirror copies the package directory recursively, including
  `weights/` and `deployments/` companions; it clears the destination first
  and runs over a long-lived SSH session (read-only package files and a 500 GB
  copy must not be killed by the 5 s inactivity timeout).
- FLARE: `flare download` and `flare orbit run` also use long-running SSH, so
  a cached-package verification or a detached launch is not cut short.
- FLARE ledger: `serve` records the recipe **directory** name, not the display
  `name` field — the old `recipe.name` lookup failed with
  `recipe dir not found for 'DeepSeek-V4.1-Flash-1017 (…) '`.
- FLARE `gb10`: production recipe updated to the 2026-10-27 package
  (`c32`, 1 GiB row cache, resident scales, `--served-model-name`).
- Tooling: `sm121` installed with `cargo install --path . --force`; version
  now follows `Cargo.toml`, not a hard-coded 0.1.0.