Home Explore Help
Sign In
sm121
/
launcher
2
0
Fork 0
Files Issues 0 Pull Requests 0 Wiki
Tree: 305f9066a6
Branches Tags
launcher
/
specs
/
tech-architecture
/
web-dashboard.md

web-dashboard.md 16 KB
History Raw

sm121 web — dashboard design

Plan for the web subcommand: an axum + SSE dashboard that manages the same things the CLI manages (serve/stop/power/logs/health/raw config), with LED status, toggles, live log windows, light/dark themes, and a floating-window UI. Read this before implementing; update the phase list as it ships.

Purpose

One command that replaces the monitoring + action loop of health/logs -f/serve/stop/power with a single browser surface. The CLI stays the source of truth and the fallback; the web layer is a thin RPC/event layer over the existing command functions — no re-implementation of serve/stop/power logic.

Non-goals (v1): multi-user auth beyond an optional bearer token, metrics/history (no time-series storage), recipe editing (raw view is read-only; edits happen in the files), mobile layout (desktop-first).

Command

sm121 web [--port 3100] [--bind 127.0.0.1] [--token <secret>] [--open]
  • --port / --bind: default 127.0.0.1:3100. Binding non-localhost requires --token (checked as Authorization: Bearer).
  • --open: open the browser (open on macOS, xdg-open otherwise).
  • Config resolution identical to the CLI (--config > cwd > global).
  • Runs until Ctrl-C. Exits non-zero if the port is taken.

Architecture

sm121 web
└─ axum Router ── AppState { config, jobs, log_hub, bus }
    ├─ /            embedded UI (include_str! of src/web/ui.html)
    ├─ /api/*       REST (actions + snapshots + raw content)
    ├─ /events      single multiplexed SSE stream
    ├─ supervisor   tokio task: polls cluster_state::probe every 30 s →
    │               broadcasts state snapshot on the bus
    └─ jobs         serve/stop/power run as tracked tokio tasks that call
                    the existing docker::serve / stop_cluster / power fns

Key properties:

  • In-process reuse. POST /api/serve calls docker::serve(config, …) with ServeOpts built from the request body — the same code path the CLI uses (before_serve hooks, mount clearing, NCCL, image sync all apply).
  • One event bus. tokio::sync::broadcast<Event> (cap 1024). Event = State | Job | Log. The single /events SSE endpoint forwards every event; the UI routes by type. No per-topic server state.
  • Supervisor is read-only. cluster_state::probe (what health does: docker ps per node over SSH) every 30 s. Same SSH load as a manual health loop (less); fine for ≤ a few nodes. Job-driven state changes (P2) push immediately, so the 30 s only bounds external changes.
  • Logs. A log attach = one docker logs -f SSH session (reusing ssh_session_follow) whose output is newline-split and pushed to the bus as Log events, plus a per-attach ring buffer (2000 lines) so new windows can fetch history via GET /api/logs?lines=N before attaching. Cap 8 concurrent attaches (ponytail: per-attach SSH sessions; queue if this ever matters). run_ssh_follow gets a callback-based sibling (run_ssh_stream(config, node, cmd, on_chunk)) — the existing fn becomes a wrapper, CLI behavior unchanged.

API

REST

Method Path Purpose
GET /api/state full snapshot (clusters, nodes, containers, jobs, log-attaches)
POST /api/serve {cluster, recipe, solo?, force?, fit?, clear?, mirror?, dry_run?} → {job}
POST /api/stop {cluster, recipe, node?} → {job}
POST /api/power {action: off\|on\|switch, cluster, from?, to?} → {job} — confirm required (see safety)
POST /api/logs attach {node, recipe, rank?} → {attach_id} (max 8)
DELETE /api/logs detach {attach_id}
GET /api/logs?node=&recipe=&lines=200 ring-buffer history for a fresh window
GET /api/recipes recipe names (from all recipe_sources) + deploy target
GET /api/raw/config active config file, verbatim
GET /api/raw/recipe/<name> recipe.toml, verbatim
GET /api/raw/effective merged Config as JSON (post cluster-source merge)

Errors: { "error": "…" } with 4xx/5xx; a running job for the same (cluster, recipe) returns 409 on a second serve/stop (single-flight).

SSE (/events)

Newline-delimited JSON, one event per data::

{"type":"state","ts":1690000000,"snapshot":{…}}
{"type":"job","id":"a1","event":"started|progress|ok|failed","cluster":"qwen38","recipe":"…","message":"…"}
{"type":"log","attach":"l2","node":"dgx1.lan","recipe":"qwen38-27b-nvfp4-unsloth","rank":0,"line":"…"}
  • State events: every supervisor tick (30 s), immediately after any job state change, and once on connect — a fresh client gets the cached snapshot first (AppState.last), so it paints and shows live at once instead of waiting up to a full poll interval.
  • Reconnect: browser EventSource auto-reconnects; the connection itself flips the indicator to live (onopen), replay keeps the data fresh.
  • Log lines are split server-side on \n; a chunk without a trailing newline is buffered (vLLM banner lines are long but single-line).

Jobs

struct Job { id, kind: Serve|Stop|Power, cluster, recipe: Option<String>,
             status: Running|Ok|Failed, started: Instant, message: String }
  • Registry: Arc<Mutex<HashMap<String, Job>>> in AppState (lazy: no external map crate).
  • Each action spawns a tokio task that (a) inserts Running, (b) calls the existing fn, (c) sets Ok/Failed + emits a Job event.
  • Process-death semantics: if sm121 web dies mid-job, the job state is lost but the containers/actions are unaffected (docker + SSH are the real side effects). sm121 health / /api/state is the source of truth.
  • dry_run: true on serve is exposed (a "dry-run preview" toggle) — it returns the rendered docker run instead of launching; the raw view shows it.

UI (single embedded file, no build step)

src/web/ui.html — one HTML file (HTML + CSS + vanilla JS), embedded via include_str! and served at /. No npm, no bundler, no static assets: CSS transitions/keyframes cover the animation budget; EventSource is native. A build-toolchain frontend is deliberately not chosen — the UI is a status board, not an app with a component graph. (Revisit only if the UI outgrows ~2–3k lines.)

Layout & windows

  • Window manager (vanilla JS): absolutely-positioned window divs with a titlebar (drag), corner resize handle, close, minimize-to-dock. Z-order by click-to-front; new windows scale-in. A dock bar (bottom) holds minimize buttons + a "new window" menu (State / Logs / Jobs / Raw Config / Raw Recipe).
  • Window kinds (parameterized views):
    • state — the LED grid (default, opens at start).
    • logs {node, recipe, rank} — live stream + history + pause + raw-copy.
    • jobs — in-flight jobs with progress lines. Running containers are grouped one row per launcher job (docker sm121.cluster + sm121.recipe label) with the node list — a container is reported by every cluster sharing its node, so a per-node×cluster listing made the whole board look busy.
    • recipes — the sm121 report --serve panel (table of tested recipes, bench history, stars, the /r/... viewer) embedded in an iframe; the dashboard nests that page at /recipes (report::html_page_at rewrites the page's own absolute paths, its SSE stream and viewer links move with it) so there is one implementation of the panel, not two.
    • raw {kind: config|recipe:name|effective} — verbatim content, monospace, copy button, JSON pretty for effective.
  • Window manager (vanilla, ui.html): absolute windows, drag by titlebar, corner resize, ▢ maximize (fills the window layer minus the dock strip; a second click restores the exact previous geometry), minimize → dock chip → restore, close. Maximized = frameless fill: no border/radius/shadow, the window is the layer (flush to the dock strip, which is a sibling below it and stays usable) and its title moves into the header with a restore button (#max-restore); exactly one window can be maximized. Window geometry is px, so fitToLayer() clamps windows back into the layer whenever the dock appears/disappears or the page resizes (run at the end of renderDock — at boot the dock is :empty, i.e. hidden, and the layer is one dock-height taller while windows are created). Every window ever created is registered in winRegistry, and the dock lists minimized ones from there — a window kept only in a local variable (recipes/raw/logs used to be) minimizes into nowhere. Default layout: left column = Topology window sized to the map's aspect ratio (a fixed-aspect drawing letterboxes ~190px of empty bands otherwise) + Jobs below it; right column = Cluster state, full height — no dead space in the layer.
  • LED grid: rows = clusters → nodes; each node row: power LED, occupancy LED, per-container LEDs (Up / Created / Exited), and action toggles (serve/stop). Click a node → spawn its logs window. Colors: green/red/amber (running/failed/starting), gray (off).
  • Toggles: serve/stop (with a two-state switch showing the job transition), power (guarded — see safety), log follow per window, theme.

Theming & animation

  • light / dark via a data-theme attribute; default prefers-color-scheme, choice persisted in localStorage. One CSS custom-property set per theme (--bg, --fg, --led-*, --accent, …) — no second stylesheet.
  • Animation budget (CSS-only): LED glow pulse while a job runs; switch spring (transform transition); window open/close scale+fade; log-line fade-in; amber "attention" pulse on failed container; skeleton shimmer while /api/state is pending.

Safety rails (non-negotiable)

The AGENTS.md live-traffic warning applies to the UI with extra force — a toggle is one click away:

  • power off / power switch: two-step confirm — the window shows the current health of the target cluster first, and requires the user to type the cluster name to arm. No silent power-off.
  • stop: confirm dialog naming the container(s) that will be killed (from the current snapshot).
  • serve: occupancy warnings surface in the job window (the CLI already bails on occupied nodes unless force; the force checkbox is labeled as such and default-off).
  • Bind-to-localhost default + optional bearer token (above) — the board can act on the cluster, so it must not be an open LAN endpoint.

Module layout (≤500 lines each)

File Contents
src/web/mod.rs run(config, opts) — router, AppState, supervisor spawn
src/web/events.rs Event enum, bus, /events SSE handler, state snapshot builder
src/web/jobs.rs Job registry, single-flight, action spawning (serve/stop/power)
src/web/logs.rs attach/detach, ring buffer, run_ssh_stream wiring
src/web/api.rs REST handlers + raw-content endpoints
src/web/ui.html the UI (embedded; exempt from the 500-line rule — not Rust)
src/ssh.rs + run_ssh_stream(config, node, cmd, on_chunk) (~15 lines)
src/cli.rs / src/main.rs Web subcommand + dispatch arm

New dependencies

  • axum = { version = "0.8", features = ["sse"] } — the only addition (futures, serde_json, tokio already present; same-origin → no tower-http).

Phases

  1. P1 — read-only board. axum server, /api/state, supervisor poll, /events SSE (state only), UI shell: theme toggle + LED grid in a window. Ship test: dashboard reflects a live serve/stop done from the CLI within ~4 s. — DONE 2026-08-22 (commit pending; smoke-tested against the real fleet: dgx1 up+occupied, dgx2-4 offline; SSE + token gate + non-loopback refusal verified).
  2. P2 — actions. jobs registry, serve/stop/power POSTs, confirm flows, jobs window, job LEDs, dry-run preview. ✅ done (bc75836) — jobs are tracked tokio tasks calling the CLI handlers (handle_serve / handle_stop / power fns), single-flight 409, job SSE events (started / ok / failed, with kind), state pushed on job finish. UI: per-cluster action bar (recipe select, dry-run), stop confirm lists running containers (new web probe fetches per-node sm121 containers), power off/switch type-the-name confirm + occupied-block. GET /api/recipes lists discoverable recipes (deploy + required_nodes). Deviations: job events are started/ok/failed (progress lines deferred to P3's log streams); serve/stop jobs run the whole CLI path (including occupancy / NCCL / venv checks) — a web serve is exactly a CLI serve.
  3. P3 — logs & windows. log attaches + ring buffers + SSE log events, window manager (resize/minimize/dock), "new window" flows from the LED grid. ✅ done (a99a642 + 3e0b679, 2026-08-23) — POST/GET/DELETE /api/logs (8 attaches, 1000-line rings, --tail=200 seed), Event::Log, Snapshot.attaches (window restore on reload); vanilla window manager (drag/resize/minimize/dock/z-order); node-click log windows with container select, pause/copy, 27 Playwright checks. Deviation: serve/stop job progress lines (the CLI-handler stdout) still not streamed — jobs stay started/ok/failed; would need per-job tracing spans + a Layer.
  4. P4 — raw & polish. raw content windows (config/recipe/effective), copy buttons, animation pass. (--open shipped with P1.) ✅ done (3ca1d59 + 887bcac/0950d9d/ec9b0d2, 2026-08-23) — GET /api/raw/{config,recipe/{name},effective} (effective = merged Config as JSON, HA token redacted), one raw window with a kind/recipe selector + copy; skeleton shimmer until first state, attention pulses (Exited containers, busy buttons, running job dots), all honoring prefers-reduced-motion. 31 Playwright checks; tests.rs split into api_tests.rs at the 500-line cap.

All phases shipped. The web plan is complete; the only documented deferred item is serve/stop job progress lines (P3 deviation, needs per-job tracing spans).

UI↔API parity audit (4f1837d, 2026-08-23): every server capability is reachable from the UI — serve flags (solo/force/fit/clear/mirror/node/ keep) via the ⚙ flags expander (body built by the pure, node-probed serveBody), per-node stop scope in the confirm dialog, visible per-node logs buttons, and the dock ＋ new-window menu (state/jobs/logs…/raw) completing the plan's window-kinds table. 36 Playwright checks.

P1 deviations from the plan: Event is a one-variant enum for now (job/log variants added with P2/P3); per-container rows are deferred to P2/P3 (the supervisor reuses ClusterState::probe as-is — node reachability + occupancy only); --open already implemented (trivial).

Testing

  • Unit: event bus broadcast/late-joiner semantics; job single-flight (409); ring-buffer wrap + ?lines= slice; run_ssh_stream newline chunking (fake stream).
  • Integration (temp config, fake cluster, no SSH): boot server on an ephemeral port → GET /api/state 200 + JSON shape; POST /api/serve {dry_run: true} → job ok with rendered command; GET /api/raw/effective reflects a cluster_sources merge; token enforced when --bind 0.0.0.0.
  • UI (Playwright under deno — no npm): cargo test ui_playwright_checks boots the real server (two offline fake clusters) and drives the embedded ui.html in real Chromium via src/web/ui.playwright.mjs (19 checks: load/render, theme toggle + persistence, chip switching, down-cluster action bar, type-the-name power gate, no-DOM-churn on identical state, serve/stop confirm dialogs, zero page-JS errors). One-time browser setup: deno run -A src/web/pw-install.mjs. Skips cleanly when deno or the browser is absent; negative-verified (a broken name gate → FAIL wrong name keeps confirm hidden). Gotchas learned: under deno waitUntil: "load" never fires (use domcontentloaded); the runner must use tokio::process — a blocking std::process::output() deadlocks the current-thread runtime that also serves the UI. ui_js_syntax_and_snapshot_logic (node) covers script syntax + sameSnapshot without a browser.

Decisions (locked 2026-08-22)

  1. UI stack: single embedded vanilla file (src/web/ui.html) — confirmed. A build-toolchain frontend is only worth it if the UI grows a real component graph (revisit at ~3k lines).
  2. Default port: 3100 (next to the 30000 serving ports).