# `sm121 web` — dashboard design

> Plan for the `web` subcommand: an axum + SSE dashboard that manages the
> same things the CLI manages (serve/stop/power/logs/health/raw config),
> with LED status, toggles, live log windows, light/dark themes, and a
> floating-window UI. Read this before implementing; update the phase list
> as it ships.

## Purpose

One command that replaces the *monitoring + action* loop of
`health`/`logs -f`/`serve`/`stop`/`power` with a single browser surface.
The CLI stays the source of truth and the fallback; the web layer is a thin
RPC/event layer over the **existing** command functions — no re-implementation
of serve/stop/power logic.

**Non-goals (v1):** multi-user auth beyond an optional bearer token,
metrics/history (no time-series storage), recipe *editing* (raw view is
read-only; edits happen in the files), mobile layout (desktop-first).

## Command

```
sm121 web [--port 3100] [--bind 127.0.0.1] [--token <secret>] [--open]
```

- `--port` / `--bind`: default `127.0.0.1:3100`. Binding non-localhost
  **requires** `--token` (checked as `Authorization: Bearer`).
- `--open`: open the browser (`open` on macOS, `xdg-open` otherwise).
- Config resolution identical to the CLI (`--config` > cwd > global).
- Runs until Ctrl-C. Exits non-zero if the port is taken.

## Architecture

```
sm121 web
└─ axum Router ── AppState { config, jobs, log_hub, bus }
    ├─ /            embedded UI (include_str! of src/web/ui.html)
    ├─ /api/*       REST (actions + snapshots + raw content)
    ├─ /events      single multiplexed SSE stream
    ├─ supervisor   tokio task: polls cluster_state::probe every 30 s →
    │               broadcasts state snapshot on the bus
    └─ jobs         serve/stop/power run as tracked tokio tasks that call
                    the existing docker::serve / stop_cluster / power fns
```

Key properties:

- **In-process reuse.** `POST /api/serve` calls `docker::serve(config, …)`
  with `ServeOpts` built from the request body — the same code path the CLI
  uses (before_serve hooks, mount clearing, NCCL, image sync all apply).
- **One event bus.** `tokio::sync::broadcast<Event>` (cap 1024). Event =
  `State | Job | Log`. The single `/events` SSE endpoint forwards every
  event; the UI routes by `type`. No per-topic server state.
- **Supervisor is read-only.** `cluster_state::probe` (what `health` does:
  `docker ps` per node over SSH) every 30 s. Same SSH load as a manual
  `health` loop (less); fine for ≤ a few nodes. Job-driven state changes
  (P2) push immediately, so the 30 s only bounds *external* changes.
- **Logs.** A log *attach* = one `docker logs -f` SSH session (reusing
  `ssh_session_follow`) whose output is newline-split and pushed to the bus
  as `Log` events, plus a per-attach ring buffer (2000 lines) so new
  windows can fetch history via `GET /api/logs?lines=N` before attaching.
  Cap 8 concurrent attaches (ponytail: per-attach SSH sessions; queue if
  this ever matters). `run_ssh_follow` gets a callback-based sibling
  (`run_ssh_stream(config, node, cmd, on_chunk)`) — the existing fn becomes
  a wrapper, CLI behavior unchanged.

## API

### REST

| Method | Path | Purpose |
|---|---|---|
| GET | `/api/state` | full snapshot (clusters, nodes, containers, jobs, log-attaches) |
| POST | `/api/serve` | `{cluster, recipe, solo?, force?, fit?, clear?, mirror?, dry_run?}` → `{job}` |
| POST | `/api/stop` | `{cluster, recipe, node?}` → `{job}` |
| POST | `/api/power` | `{action: off\|on\|switch, cluster, from?, to?}` → `{job}` — **confirm required** (see safety) |
| POST | `/api/logs` | attach `{node, recipe, rank?}` → `{attach_id}` (max 8) |
| DELETE | `/api/logs` | detach `{attach_id}` |
| GET | `/api/logs?node=&recipe=&lines=200` | ring-buffer history for a fresh window |
| GET | `/api/recipes` | recipe names (from all `recipe_sources`) + deploy target |
| GET | `/api/raw/config` | active config file, verbatim |
| GET | `/api/raw/recipe/<name>` | recipe.toml, verbatim |
| GET | `/api/raw/effective` | merged `Config` as JSON (post cluster-source merge) |

Errors: `{ "error": "…" }` with 4xx/5xx; a running job for the same
(cluster, recipe) returns 409 on a second serve/stop (single-flight).

### SSE (`/events`)

Newline-delimited JSON, one event per `data:`:

```json
{"type":"state","ts":1690000000,"snapshot":{…}}
{"type":"job","id":"a1","event":"started|progress|ok|failed","cluster":"qwen38","recipe":"…","message":"…"}
{"type":"log","attach":"l2","node":"dgx1.lan","recipe":"qwen38-27b-nvfp4-unsloth","rank":0,"line":"…"}
```

- State events: every supervisor tick (30 s), immediately after any job
  state change, and **once on connect** — a fresh client gets the cached
  snapshot first (`AppState.last`), so it paints and shows `live` at once
  instead of waiting up to a full poll interval.
- Reconnect: browser `EventSource` auto-reconnects; the connection itself
  flips the indicator to `live` (`onopen`), replay keeps the data fresh.
- Log lines are split server-side on `\n`; a chunk without a trailing
  newline is buffered (vLLM banner lines are long but single-line).

## Jobs

```rust
struct Job { id, kind: Serve|Stop|Power, cluster, recipe: Option<String>,
             status: Running|Ok|Failed, started: Instant, message: String }
```

- Registry: `Arc<Mutex<HashMap<String, Job>>>` in `AppState` (lazy: no
  external map crate).
- Each action spawns a tokio task that (a) inserts `Running`, (b) calls the
  existing fn, (c) sets `Ok`/`Failed` + emits a `Job` event.
- **Process-death semantics:** if `sm121 web` dies mid-job, the job state is
  lost but the containers/actions are unaffected (docker + SSH are the real
  side effects). `sm121 health` / `/api/state` is the source of truth.
- `dry_run: true` on serve is exposed (a "dry-run preview" toggle) — it
  returns the rendered `docker run` instead of launching; the raw view shows
  it.

## UI (single embedded file, no build step)

`src/web/ui.html` — one HTML file (HTML + CSS + vanilla JS), embedded via
`include_str!` and served at `/`. No npm, no bundler, no static assets:
CSS transitions/keyframes cover the animation budget; `EventSource` is
native. A build-toolchain frontend is deliberately **not** chosen — the UI
is a status board, not an app with a component graph. (Revisit only if the
UI outgrows ~2–3k lines.)

### Layout & windows

- **Window manager (vanilla JS):** absolutely-positioned window divs with a
  titlebar (drag), corner resize handle, close, minimize-to-dock. Z-order by
  click-to-front; new windows scale-in. A dock bar (bottom) holds minimize
  buttons + a "new window" menu (State / Logs / Jobs / Raw Config / Raw
  Recipe).
- **Window kinds** (parameterized views):
  - `state` — the LED grid (default, opens at start).
  - `logs {node, recipe, rank}` — live stream + history + pause + raw-copy.
  - `jobs` — in-flight jobs with progress lines. Running containers are
    grouped **one row per launcher job** (docker `sm121.cluster` +
    `sm121.recipe` label) with the node list — a container is reported by
    every cluster sharing its node, so a per-node×cluster listing made the
    whole board look busy.
  - `recipes` — the `sm121 report --serve` panel (table of tested recipes,
    bench history, stars, the `/r/...` viewer) embedded in an iframe; the
    dashboard **nests that page at `/recipes`** (`report::html_page_at`
    rewrites the page's own absolute paths, its SSE stream and viewer links
    move with it) so there is one implementation of the panel, not two.
  - `raw {kind: config|recipe:name|effective}` — verbatim content, monospace,
    copy button, JSON pretty for `effective`.
- **Window manager** (vanilla, `ui.html`): absolute windows, drag by
  titlebar, corner resize, `▢` **maximize** (fills the window layer minus
  the dock strip; a second click restores the exact previous geometry),
  minimize → dock chip → restore, close. Maximized = **frameless fill**: no
  border/radius/shadow, the window *is* the layer (flush to the dock strip,
  which is a sibling below it and stays usable) and its title moves into the
  header with a restore button (`#max-restore`); exactly one window can be
  maximized. Window geometry is px, so `fitToLayer()` clamps windows back
  into the layer whenever the dock appears/disappears or the page resizes
  (run at the *end* of `renderDock` — at boot the dock is `:empty`, i.e.
  hidden, and the layer is one dock-height taller while windows are
  created). Every window ever created is
  registered in `winRegistry`, and the dock lists minimized ones from
  there — a window kept only in a local variable (recipes/raw/logs used to
  be) minimizes into nowhere. Default layout: left column = Topology
  window **sized to the map's aspect ratio** (a fixed-aspect drawing
  letterboxes ~190px of empty bands otherwise) + Jobs below it; right
  column = Cluster state, full height — no dead space in the layer.
- **LED grid:** rows = clusters → nodes; each node row: power LED,
  occupancy LED, per-container LEDs (Up / Created / Exited), and action
  toggles (serve/stop). Click a node → spawn its logs window. Colors:
  green/red/amber (running/failed/starting), gray (off).
- **Toggles:** serve/stop (with a two-state switch showing the job
  transition), power (guarded — see safety), log follow per window, theme.

### Theming & animation

- `light` / `dark` via a `data-theme` attribute; default `prefers-color-scheme`,
  choice persisted in `localStorage`. One CSS custom-property set per theme
  (`--bg`, `--fg`, `--led-*`, `--accent`, …) — no second stylesheet.
- Animation budget (CSS-only): LED glow pulse while a job runs; switch
  spring (transform transition); window open/close scale+fade; log-line
  fade-in; amber "attention" pulse on failed container; skeleton shimmer
  while `/api/state` is pending.

## Safety rails (non-negotiable)

The AGENTS.md live-traffic warning applies to the UI with extra force — a
toggle is one click away:

- **`power off` / `power switch`:** two-step confirm — the window shows the
  current health of the target cluster first, and requires the user to type
  the cluster name to arm. No silent power-off.
- **`stop`:** confirm dialog naming the container(s) that will be killed
  (from the current snapshot).
- **`serve`:** occupancy warnings surface in the job window (the CLI
  already bails on occupied nodes unless `force`; the `force` checkbox is
  labeled as such and default-off).
- Bind-to-localhost default + optional bearer token (above) — the board can
  act on the cluster, so it must not be an open LAN endpoint.

## Module layout (≤500 lines each)

| File | Contents |
|---|---|
| `src/web/mod.rs` | `run(config, opts)` — router, AppState, supervisor spawn |
| `src/web/events.rs` | `Event` enum, bus, `/events` SSE handler, state snapshot builder |
| `src/web/jobs.rs` | Job registry, single-flight, action spawning (serve/stop/power) |
| `src/web/logs.rs` | attach/detach, ring buffer, `run_ssh_stream` wiring |
| `src/web/api.rs` | REST handlers + raw-content endpoints |
| `src/web/ui.html` | the UI (embedded; exempt from the 500-line rule — not Rust) |
| `src/ssh.rs` | + `run_ssh_stream(config, node, cmd, on_chunk)` (~15 lines) |
| `src/cli.rs` / `src/main.rs` | `Web` subcommand + dispatch arm |

## New dependencies

- `axum = { version = "0.8", features = ["sse"] }` — the only addition
  (`futures`, `serde_json`, `tokio` already present; same-origin → no
  `tower-http`).

## Phases

1. **P1 — read-only board.** axum server, `/api/state`, supervisor poll,
   `/events` SSE (state only), UI shell: theme toggle + LED grid in a
   window. *Ship test: dashboard reflects a live serve/stop done from the
   CLI within ~4 s.* — **DONE 2026-08-22** (commit pending; smoke-tested
   against the real fleet: dgx1 up+occupied, dgx2-4 offline; SSE + token
   gate + non-loopback refusal verified).
2. **P2 — actions.** jobs registry, serve/stop/power POSTs, confirm flows,
   jobs window, job LEDs, dry-run preview.  ✅ done (`bc75836`) — jobs are
   tracked tokio tasks calling the CLI handlers (`handle_serve` /
   `handle_stop` / power fns), single-flight 409, `job` SSE events (started /
   ok / failed, with kind), state pushed on job finish. UI: per-cluster
   action bar (recipe select, dry-run), stop confirm lists running
   containers (new web probe fetches per-node sm121 containers), power
   off/switch type-the-name confirm + occupied-block. `GET /api/recipes`
   lists discoverable recipes (deploy + required_nodes). Deviations: job
   events are started/ok/failed (progress lines deferred to P3's log
   streams); serve/stop jobs run the *whole* CLI path (including
   occupancy / NCCL / venv checks) — a web serve is exactly a CLI serve.
3. **P3 — logs & windows.** log attaches + ring buffers + SSE log events,
   window manager (resize/minimize/dock), "new window" flows from the LED
   grid. ✅ done (`a99a642` + `3e0b679`, 2026-08-23) — `POST/GET/DELETE
   /api/logs` (8 attaches, 1000-line rings, `--tail=200` seed), `Event::Log`,
   `Snapshot.attaches` (window restore on reload); vanilla window manager
   (drag/resize/minimize/dock/z-order); node-click log windows with
   container select, pause/copy, 27 Playwright checks. Deviation: serve/stop
   job *progress lines* (the CLI-handler stdout) still not streamed — jobs
   stay started/ok/failed; would need per-job tracing spans + a Layer.
4. **P4 — raw & polish.** raw content windows (config/recipe/effective),
   copy buttons, animation pass. (`--open` shipped with P1.)
   ✅ done (`3ca1d59` + `887bcac`/`0950d9d`/`ec9b0d2`, 2026-08-23) —
   `GET /api/raw/{config,recipe/{name},effective}` (effective = merged
   Config as JSON, HA token redacted), one raw window with a kind/recipe
   selector + copy; skeleton shimmer until first state, attention pulses
   (Exited containers, busy buttons, running job dots), all honoring
   `prefers-reduced-motion`. 31 Playwright checks; `tests.rs` split into
   `api_tests.rs` at the 500-line cap.

**All phases shipped.** The web plan is complete; the only documented
deferred item is serve/stop job *progress lines* (P3 deviation, needs
per-job tracing spans).

**UI↔API parity audit (`4f1837d`, 2026-08-23):** every server capability
is reachable from the UI — serve flags (solo/force/fit/clear/mirror/node/
keep) via the `⚙ flags` expander (body built by the pure, node-probed
`serveBody`), per-node stop scope in the confirm dialog, visible per-node
logs buttons, and the dock `＋` new-window menu (state/jobs/logs…/raw)
completing the plan's window-kinds table. 36 Playwright checks.

P1 deviations from the plan: `Event` is a one-variant enum for now (job/log
variants added with P2/P3); per-container rows are deferred to P2/P3 (the
supervisor reuses `ClusterState::probe` as-is — node reachability +
occupancy only); `--open` already implemented (trivial).

## Testing

- Unit: event bus broadcast/late-joiner semantics; job single-flight
  (409); ring-buffer wrap + `?lines=` slice; `run_ssh_stream` newline
  chunking (fake stream).
- Integration (temp config, fake cluster, no SSH): boot server on an
  ephemeral port → `GET /api/state` 200 + JSON shape; `POST /api/serve
  {dry_run: true}` → job `ok` with rendered command; `GET
  /api/raw/effective` reflects a `cluster_sources` merge; token enforced
  when `--bind 0.0.0.0`.
- UI (Playwright under **deno** — no npm): `cargo test
  ui_playwright_checks` boots the real server (two offline fake clusters)
  and drives the embedded `ui.html` in real Chromium via
  `src/web/ui.playwright.mjs` (19 checks: load/render, theme toggle +
  persistence, chip switching, down-cluster action bar, type-the-name
  power gate, no-DOM-churn on identical state, serve/stop confirm dialogs,
  zero page-JS errors). One-time browser setup:
  `deno run -A src/web/pw-install.mjs`. Skips cleanly when deno or the
  browser is absent; negative-verified (a broken name gate →
  `FAIL wrong name keeps confirm hidden`). Gotchas learned: under deno
  `waitUntil: "load"` never fires (use `domcontentloaded`); the runner
  must use `tokio::process` — a blocking `std::process::output()`
  deadlocks the current-thread runtime that also serves the UI.
  `ui_js_syntax_and_snapshot_logic` (node) covers script syntax +
  `sameSnapshot` without a browser.

## Decisions (locked 2026-08-22)

1. **UI stack:** single embedded vanilla file (`src/web/ui.html`) — confirmed.
   A build-toolchain frontend is only worth it if the UI grows a real
   component graph (revisit at ~3k lines).
2. **Default port:** `3100` (next to the 30000 serving ports).