|
|
@@ -0,0 +1,230 @@
|
|
|
+# `sm121 web` — dashboard design
|
|
|
+
|
|
|
+> Plan for the `web` subcommand: an axum + SSE dashboard that manages the
|
|
|
+> same things the CLI manages (serve/stop/power/logs/health/raw config),
|
|
|
+> with LED status, toggles, live log windows, light/dark themes, and a
|
|
|
+> floating-window UI. Read this before implementing; update the phase list
|
|
|
+> as it ships.
|
|
|
+
|
|
|
+## Purpose
|
|
|
+
|
|
|
+One command that replaces the *monitoring + action* loop of
|
|
|
+`health`/`logs -f`/`serve`/`stop`/`power` with a single browser surface.
|
|
|
+The CLI stays the source of truth and the fallback; the web layer is a thin
|
|
|
+RPC/event layer over the **existing** command functions — no re-implementation
|
|
|
+of serve/stop/power logic.
|
|
|
+
|
|
|
+**Non-goals (v1):** multi-user auth beyond an optional bearer token,
|
|
|
+metrics/history (no time-series storage), recipe *editing* (raw view is
|
|
|
+read-only; edits happen in the files), mobile layout (desktop-first).
|
|
|
+
|
|
|
+## Command
|
|
|
+
|
|
|
+```
|
|
|
+sm121 web [--port 3100] [--bind 127.0.0.1] [--token <secret>] [--open]
|
|
|
+```
|
|
|
+
|
|
|
+- `--port` / `--bind`: default `127.0.0.1:3100`. Binding non-localhost
|
|
|
+ **requires** `--token` (checked as `Authorization: Bearer`).
|
|
|
+- `--open`: open the browser (`open` on macOS, `xdg-open` otherwise).
|
|
|
+- Config resolution identical to the CLI (`--config` > cwd > global).
|
|
|
+- Runs until Ctrl-C. Exits non-zero if the port is taken.
|
|
|
+
|
|
|
+## Architecture
|
|
|
+
|
|
|
+```
|
|
|
+sm121 web
|
|
|
+└─ axum Router ── AppState { config, jobs, log_hub, bus }
|
|
|
+ ├─ / embedded UI (include_str! of src/web/ui.html)
|
|
|
+ ├─ /api/* REST (actions + snapshots + raw content)
|
|
|
+ ├─ /events single multiplexed SSE stream
|
|
|
+ ├─ supervisor tokio task: polls cluster_state::probe every 4 s →
|
|
|
+ │ broadcasts state snapshot on the bus
|
|
|
+ └─ jobs serve/stop/power run as tracked tokio tasks that call
|
|
|
+ the existing docker::serve / stop_cluster / power fns
|
|
|
+```
|
|
|
+
|
|
|
+Key properties:
|
|
|
+
|
|
|
+- **In-process reuse.** `POST /api/serve` calls `docker::serve(config, …)`
|
|
|
+ with `ServeOpts` built from the request body — the same code path the CLI
|
|
|
+ uses (before_serve hooks, mount clearing, NCCL, image sync all apply).
|
|
|
+- **One event bus.** `tokio::sync::broadcast<Event>` (cap 1024). Event =
|
|
|
+ `State | Job | Log`. The single `/events` SSE endpoint forwards every
|
|
|
+ event; the UI routes by `type`. No per-topic server state.
|
|
|
+- **Supervisor is read-only.** `cluster_state::probe` (what `health` does:
|
|
|
+ `docker ps` per node over SSH) every 4 s. Same SSH load as a manual
|
|
|
+ `health` loop; fine for ≤ a few nodes.
|
|
|
+- **Logs.** A log *attach* = one `docker logs -f` SSH session (reusing
|
|
|
+ `ssh_session_follow`) whose output is newline-split and pushed to the bus
|
|
|
+ as `Log` events, plus a per-attach ring buffer (2000 lines) so new
|
|
|
+ windows can fetch history via `GET /api/logs?lines=N` before attaching.
|
|
|
+ Cap 8 concurrent attaches (ponytail: per-attach SSH sessions; queue if
|
|
|
+ this ever matters). `run_ssh_follow` gets a callback-based sibling
|
|
|
+ (`run_ssh_stream(config, node, cmd, on_chunk)`) — the existing fn becomes
|
|
|
+ a wrapper, CLI behavior unchanged.
|
|
|
+
|
|
|
+## API
|
|
|
+
|
|
|
+### REST
|
|
|
+
|
|
|
+| Method | Path | Purpose |
|
|
|
+|---|---|---|
|
|
|
+| GET | `/api/state` | full snapshot (clusters, nodes, containers, jobs, log-attaches) |
|
|
|
+| POST | `/api/serve` | `{cluster, recipe, solo?, force?, fit?, clear?, mirror?, dry_run?}` → `{job}` |
|
|
|
+| POST | `/api/stop` | `{cluster, recipe, node?}` → `{job}` |
|
|
|
+| POST | `/api/power` | `{action: off\|on\|switch, cluster, from?, to?}` → `{job}` — **confirm required** (see safety) |
|
|
|
+| POST | `/api/logs` | attach `{node, recipe, rank?}` → `{attach_id}` (max 8) |
|
|
|
+| DELETE | `/api/logs` | detach `{attach_id}` |
|
|
|
+| GET | `/api/logs?node=&recipe=&lines=200` | ring-buffer history for a fresh window |
|
|
|
+| GET | `/api/recipes` | recipe names (from all `recipe_sources`) + deploy target |
|
|
|
+| GET | `/api/raw/config` | active config file, verbatim |
|
|
|
+| GET | `/api/raw/recipe/<name>` | recipe.toml, verbatim |
|
|
|
+| GET | `/api/raw/effective` | merged `Config` as JSON (post cluster-source merge) |
|
|
|
+
|
|
|
+Errors: `{ "error": "…" }` with 4xx/5xx; a running job for the same
|
|
|
+(cluster, recipe) returns 409 on a second serve/stop (single-flight).
|
|
|
+
|
|
|
+### SSE (`/events`)
|
|
|
+
|
|
|
+Newline-delimited JSON, one event per `data:`:
|
|
|
+
|
|
|
+```json
|
|
|
+{"type":"state","ts":1690000000,"snapshot":{…}}
|
|
|
+{"type":"job","id":"a1","event":"started|progress|ok|failed","cluster":"qwen38","recipe":"…","message":"…"}
|
|
|
+{"type":"log","attach":"l2","node":"dgx1.lan","recipe":"qwen38-27b-nvfp4-unsloth","rank":0,"line":"…"}
|
|
|
+```
|
|
|
+
|
|
|
+- State events: every supervisor tick (4 s) and immediately after any job
|
|
|
+ state change.
|
|
|
+- Reconnect: browser `EventSource` auto-reconnects; on open the client
|
|
|
+ re-fetches `GET /api/state` (SSE is fire-and-forget, not a log).
|
|
|
+- Log lines are split server-side on `\n`; a chunk without a trailing
|
|
|
+ newline is buffered (vLLM banner lines are long but single-line).
|
|
|
+
|
|
|
+## Jobs
|
|
|
+
|
|
|
+```rust
|
|
|
+struct Job { id, kind: Serve|Stop|Power, cluster, recipe: Option<String>,
|
|
|
+ status: Running|Ok|Failed, started: Instant, message: String }
|
|
|
+```
|
|
|
+
|
|
|
+- Registry: `Arc<Mutex<HashMap<String, Job>>>` in `AppState` (lazy: no
|
|
|
+ external map crate).
|
|
|
+- Each action spawns a tokio task that (a) inserts `Running`, (b) calls the
|
|
|
+ existing fn, (c) sets `Ok`/`Failed` + emits a `Job` event.
|
|
|
+- **Process-death semantics:** if `sm121 web` dies mid-job, the job state is
|
|
|
+ lost but the containers/actions are unaffected (docker + SSH are the real
|
|
|
+ side effects). `sm121 health` / `/api/state` is the source of truth.
|
|
|
+- `dry_run: true` on serve is exposed (a "dry-run preview" toggle) — it
|
|
|
+ returns the rendered `docker run` instead of launching; the raw view shows
|
|
|
+ it.
|
|
|
+
|
|
|
+## UI (single embedded file, no build step)
|
|
|
+
|
|
|
+`src/web/ui.html` — one HTML file (HTML + CSS + vanilla JS), embedded via
|
|
|
+`include_str!` and served at `/`. No npm, no bundler, no static assets:
|
|
|
+CSS transitions/keyframes cover the animation budget; `EventSource` is
|
|
|
+native. A build-toolchain frontend is deliberately **not** chosen — the UI
|
|
|
+is a status board, not an app with a component graph. (Revisit only if the
|
|
|
+UI outgrows ~2–3k lines.)
|
|
|
+
|
|
|
+### Layout & windows
|
|
|
+
|
|
|
+- **Window manager (vanilla JS):** absolutely-positioned window divs with a
|
|
|
+ titlebar (drag), corner resize handle, close, minimize-to-dock. Z-order by
|
|
|
+ click-to-front; new windows scale-in. A dock bar (bottom) holds minimize
|
|
|
+ buttons + a "new window" menu (State / Logs / Jobs / Raw Config / Raw
|
|
|
+ Recipe).
|
|
|
+- **Window kinds** (parameterized views):
|
|
|
+ - `state` — the LED grid (default, opens at start).
|
|
|
+ - `logs {node, recipe, rank}` — live stream + history + pause + raw-copy.
|
|
|
+ - `jobs` — in-flight jobs with progress lines.
|
|
|
+ - `raw {kind: config|recipe:name|effective}` — verbatim content, monospace,
|
|
|
+ copy button, JSON pretty for `effective`.
|
|
|
+- **LED grid:** rows = clusters → nodes; each node row: power LED,
|
|
|
+ occupancy LED, per-container LEDs (Up / Created / Exited), and action
|
|
|
+ toggles (serve/stop). Click a node → spawn its logs window. Colors:
|
|
|
+ green/red/amber (running/failed/starting), gray (off).
|
|
|
+- **Toggles:** serve/stop (with a two-state switch showing the job
|
|
|
+ transition), power (guarded — see safety), log follow per window, theme.
|
|
|
+
|
|
|
+### Theming & animation
|
|
|
+
|
|
|
+- `light` / `dark` via a `data-theme` attribute; default `prefers-color-scheme`,
|
|
|
+ choice persisted in `localStorage`. One CSS custom-property set per theme
|
|
|
+ (`--bg`, `--fg`, `--led-*`, `--accent`, …) — no second stylesheet.
|
|
|
+- Animation budget (CSS-only): LED glow pulse while a job runs; switch
|
|
|
+ spring (transform transition); window open/close scale+fade; log-line
|
|
|
+ fade-in; amber "attention" pulse on failed container; skeleton shimmer
|
|
|
+ while `/api/state` is pending.
|
|
|
+
|
|
|
+## Safety rails (non-negotiable)
|
|
|
+
|
|
|
+The AGENTS.md live-traffic warning applies to the UI with extra force — a
|
|
|
+toggle is one click away:
|
|
|
+
|
|
|
+- **`power off` / `power switch`:** two-step confirm — the window shows the
|
|
|
+ current health of the target cluster first, and requires the user to type
|
|
|
+ the cluster name to arm. No silent power-off.
|
|
|
+- **`stop`:** confirm dialog naming the container(s) that will be killed
|
|
|
+ (from the current snapshot).
|
|
|
+- **`serve`:** occupancy warnings surface in the job window (the CLI
|
|
|
+ already bails on occupied nodes unless `force`; the `force` checkbox is
|
|
|
+ labeled as such and default-off).
|
|
|
+- Bind-to-localhost default + optional bearer token (above) — the board can
|
|
|
+ act on the cluster, so it must not be an open LAN endpoint.
|
|
|
+
|
|
|
+## Module layout (≤500 lines each)
|
|
|
+
|
|
|
+| File | Contents |
|
|
|
+|---|---|
|
|
|
+| `src/web/mod.rs` | `run(config, opts)` — router, AppState, supervisor spawn |
|
|
|
+| `src/web/events.rs` | `Event` enum, bus, `/events` SSE handler, state snapshot builder |
|
|
|
+| `src/web/jobs.rs` | Job registry, single-flight, action spawning (serve/stop/power) |
|
|
|
+| `src/web/logs.rs` | attach/detach, ring buffer, `run_ssh_stream` wiring |
|
|
|
+| `src/web/api.rs` | REST handlers + raw-content endpoints |
|
|
|
+| `src/web/ui.html` | the UI (embedded; exempt from the 500-line rule — not Rust) |
|
|
|
+| `src/ssh.rs` | + `run_ssh_stream(config, node, cmd, on_chunk)` (~15 lines) |
|
|
|
+| `src/cli.rs` / `src/main.rs` | `Web` subcommand + dispatch arm |
|
|
|
+
|
|
|
+## New dependencies
|
|
|
+
|
|
|
+- `axum = { version = "0.8", features = ["sse"] }` — the only addition
|
|
|
+ (`futures`, `serde_json`, `tokio` already present; same-origin → no
|
|
|
+ `tower-http`).
|
|
|
+
|
|
|
+## Phases
|
|
|
+
|
|
|
+1. **P1 — read-only board.** axum server, `/api/state`, supervisor poll,
|
|
|
+ `/events` SSE (state only), UI shell: theme toggle + LED grid in a
|
|
|
+ window. *Ship test: dashboard reflects a live serve/stop done from the
|
|
|
+ CLI within ~4 s.*
|
|
|
+2. **P2 — actions.** jobs registry, serve/stop/power POSTs, confirm flows,
|
|
|
+ jobs window, job LEDs, dry-run preview.
|
|
|
+3. **P3 — logs & windows.** log attaches + ring buffers + SSE log events,
|
|
|
+ window manager polish (resize/minimize/dock), "new window" flows from
|
|
|
+ the LED grid.
|
|
|
+4. **P4 — raw & polish.** raw content windows (config/recipe/effective),
|
|
|
+ copy buttons, animation pass, `--open`.
|
|
|
+
|
|
|
+## Testing
|
|
|
+
|
|
|
+- Unit: event bus broadcast/late-joiner semantics; job single-flight
|
|
|
+ (409); ring-buffer wrap + `?lines=` slice; `run_ssh_stream` newline
|
|
|
+ chunking (fake stream).
|
|
|
+- Integration (temp config, fake cluster, no SSH): boot server on an
|
|
|
+ ephemeral port → `GET /api/state` 200 + JSON shape; `POST /api/serve
|
|
|
+ {dry_run: true}` → job `ok` with rendered command; `GET
|
|
|
+ /api/raw/effective` reflects a `cluster_sources` merge; token enforced
|
|
|
+ when `--bind 0.0.0.0`.
|
|
|
+- UI: no automation in v1 — the phase-ship tests above + manual checklist
|
|
|
+ (theme persists, window drag/resize, SSE reconnect after network blip,
|
|
|
+ power confirm can't be armed without typing the name).
|
|
|
+
|
|
|
+## Decisions (locked 2026-08-22)
|
|
|
+
|
|
|
+1. **UI stack:** single embedded vanilla file (`src/web/ui.html`) — confirmed.
|
|
|
+ A build-toolchain frontend is only worth it if the UI grows a real
|
|
|
+ component graph (revisit at ~3k lines).
|
|
|
+2. **Default port:** `3100` (next to the 30000 serving ports).
|
