launcher/specs/tech-architecture/ha-addon-recipe-repos.md

sm121 HA Add-on + Recipe-Repo Architecture — Implementation Plan

Status: design frozen 2026-10-10, implementation pending. Supersedes: the sm121-ha-integration custom-integration prototype (commit 1ccd032 in sm121-ha-integration/ — keep as reference, the direction changed to an add-on).

Goal

Run the sm121 launcher inside Home Assistant as a Docker add-on (24/7, independent of the Mac), with launcher/recipe separation: recipes + config live in git repos (public or private), the launcher clones/pulls them, and recipes are promoted across repos and repos are archived. The HA UI drives serve/stop/power via the sm121 web dashboard (port 3100) exposed by the add-on.

Frozen design decisions (operator-confirmed)

Decision	Value
Form factor	HA add-on (Docker container running the sm121 binary), not a custom integration
SSH keys	part of the add-on config (mounted into the container)
Recipes + config	from git repos (public or private), launcher clones/pulls
Repo priority	explicit ordered list in config — first entry wins on a recipe-name collision
Promotion	launcher does it: sm121 promote <recipe> --from <repo> --to <repo> (git mv + commit + push, --dry-run/--apply)
Archive state	separate JSON registry (~/.config/sm121/repos.json), not a config flag
Docker images	hybrid: docker-images/ build contexts in the recipe repo (source of truth) + optional registry pull (--image-source registry) to skip the build
HA UI	the sm121 web dashboard (port 3100); entities (integration) are a later phase, not blocking

Components

1. Launcher changes (Rust, sm121-launcher)

New config block (ordered list; first = highest priority):

[[recipe_repos]]
url = "https://git.kilork.org/sm121/recipes-production.git"
token = "…"            # optional; private-repo credential (deploy key / PAT)

[[recipe_repos]]
url = "https://git.kilork.org/sm121/recipes-playground.git"
token = "…"

New module src/recipe_repos.rs:

• sync_recipe_repos(config) — for each non-archived [[recipe_repos]], clone (first time) / pull (subsequent) into a cache dir (~/.cache/sm121/recipe-repos/<sha1(url)>), in list order. A unreachable repo warns and is skipped (the launcher still starts with the reachable ones — a dead playground repo must not take down production serving).
• Returns the list of cloned repo roots → appended to recipe_sources (after the cwd recipes/), in the same order → a recipe name present in an earlier (higher-priority) repo shadows the later one (the existing recipe_sources search-order semantics already do this: earlier source wins).
• Reads the archive registry (repos.json); archived URLs are skipped.

New CLI subcommands:

• sm121 promote <recipe> --from <repo-url> --to <repo-url> [--dry-run|--apply] — git mv recipes/<recipe>/ from the source repo checkout to the target repo checkout, commit, push. --dry-run prints the exact git commands; --apply runs them. Refuses if the recipe exists in the target already (no silent overwrite).
• sm121 archive <repo-url> / sm121 unarchive <repo-url> — flip the entry in repos.json (creates the file if missing). The next sync_recipe_repos skips archived repos.
• sm121 repos — list the configured repos with their state (cloned? archived? last-pull ts? recipe count?).

Docker-image hybrid:

• serve/build gain --image-source {repo,registry} (default repo = current on-demand build from docker-images/). registry pulls the prebuilt docker_image ref from the [registry] base and skips the build (the image must already be pushed; the checksum label is verified).

Wiring:

• serve/stop/report/validate call sync_recipe_repos first (so the recipe set is current). The sync is best-effort (a failed repo degrades to a warning, never blocks a serve of a recipe from a reachable repo).
• The sm121 web dashboard's recipe panel reads the merged set (it already does, via recipe_sources).

2. Docker image (the add-on container)

docker-images/sm121-addon/ (in the launcher repo, or a build dir):

• Base: debian:bookworm-slim (or gcr.io/distroless/static for a smaller final image; debian for git + debuggability).
• Installs git + copies the sm121 binary (the release build).
• Entrypoint (entrypoint.sh):
  1. Write the effective config.toml from the add-on options (the recipe_repos, the ssh_key_path, the [home_assistant], the [registry]).
  2. Drop the SSH key at ssh_key_path (from a mounted secret).
  3. sm121 repos sync (the sync_recipe_repos).
  4. exec sm121 web --port 3100 --bind 0.0.0.1 (expose the dashboard).
  5. A supervisor loop re-syncs the repos every N minutes (a cron or a background loop) so new recipes land without a container restart.

The image is built from the launcher repo (a Makefile target or a build.sh), tagged + pushed to the [registry] base (the existing registry-publishing machinery).

3. HA add-on (sm121-ha-integration/ repo, restructured)

• config.yaml — the add-on manifest: name, description, the port (3100, host access), the options schema:
  • recipe_repos (a YAML list of {url, token}),
  • ssh_key (a secret — the private key to the cluster nodes),
  • ha_url / ha_token (for the power entities the launcher already uses),
  • registry_base (for the --image-source registry pulls),
  • repo_resync_minutes (the background re-sync interval).
• Dockerfile — builds the add-on image (or references the prebuilt registry tag).
• The add-on mounts the options → the entrypoint writes config.toml + the SSH key → runs the launcher.
• HA reachability: the add-on container must route to the cluster node SSH IPs (10G mgmt) + the HA REST API + the Mikrotik devices (8728). The operator ensures the HA host is on a network that reaches them (the same constraint the Mac has today).

4. Recipe repos (the git repos)

Two starting repos (the operator creates them on git.kilork.org):

• sm121/recipes-production — the qualified, serving recipes (higher priority, listed first).
• sm121/recipes-playground — the experimental recipes (lower priority).

Each repo layout (a subset of the launcher repo):

config.toml            # clusters, ssh_user, [home_assistant], [registry]
recipes/<name>/recipe.toml
docker-images/<dir>/   # build contexts (source of truth; optional if
                       # the image is prebuilt in the registry)

The config.toml in the repo is the cluster/config source of truth (the add-on's effective config = the repo's config.toml + the add-on options for the secrets). A repo without a config.toml inherits the add-on's base config.

Consequences + mitigations (the designed-in answers)

Consequence	Mitigation in the design
A recipe repo is unreachable at startup	sync_recipe_repos is best-effort: unreachable repo → warn + skip; the launcher still serves from the reachable repos. A production repo down is logged loudly (the repos subcommand shows the state).
A private repo's credential leaks / rotates	The token is an add-on option (a HA secret), not baked into the image; rotation = edit the option, no rebuild. The clone URL embeds the token only in the in-container cache, never committed.
promote pushes to a repo the key can't write	--dry-run first (prints the exact git commands); --apply fails loud on a rejected push; the recipe stays in the source repo (no partial state — the mv is a single commit).
promote name collision in the target	Refused up front (the recipe must not exist in the target); no silent overwrite.
repos.json corrupt / missing	Missing → created fresh (no repos archived). Corrupt → backed up to repos.json.bak + a fresh one created + a warning (the launcher never crashes on the registry).
Hybrid image: registry pull of a missing tag	--image-source registry verifies the checksum label after pull; a missing tag → clear error naming the tag + a hint to build/push first (falls back to --image-source repo).
Add-on container crashes / the launcher OOMs	The HA add-on config.yaml sets the Docker restart policy (always); the entrypoint supervisor re-syncs repos on restart. A crash loses no state (the ledger + the repos are on disk / in git).
Two launchers (Mac + HA add-on) serving the same cluster	The run-ledger (.lock) is per-recipe in the recipe repo (git) — a second launcher that serves the same recipe sees the open run and refuses (the existing ledger semantics). The operator runs one launcher as the source of truth (the add-on); the Mac launcher is for dev. Documented in the add-on README.
A stale cloned repo shadows a newer recipe	The background re-sync (every N minutes) + the repos subcommand show the last-pull ts; a sm121 repos sync --force re-pulls.

Testing strategy

1. Unit (Rust, cargo test):
   • recipe_repos.rs: the merge order (first repo shadows later on a name collision), the archive-registry skip, the best-effort skip of an unreachable repo (mock the git via a trait, or a temp-dir "repo" with a file:// URL — git supports file:// clones, no network).
   • repos.json: missing → fresh, corrupt → backup + fresh, the archive/unarchive round-trip.
   • promote: --dry-run prints the right git commands (assert on the printed string); --apply on two temp file:// repos (git mv + commit visible in the target's log); the collision refusal.
   • Use file:// git URLs for all repo tests (no network, hermetic).
2. Integration (the validate gate): the sm121 validate command runs over the merged recipe set (the existing recipe-contract gate) — a promoted recipe must still pass the contract in the target repo.
3. Docker image: a make addon-image target builds it; a smoke test runs the entrypoint with a file:// recipe repo + a stub config and asserts the sm121 web dashboard answers on 3100 (a curl /api/state).
4. HA add-on: manual (the operator installs it in HA): the config flow, the entities, a serve/stop/power round-trip against a test cluster. No automated HA test (HA is the operator's env).
5. End-to-end playground flow (manual, the operator): new recipe in playground → add-on re-syncs → serve → bench → sm121 promote → production → the add-on serves it from production → sm121 archive the playground repo → the recipe is still served (from production).

Implementation order (the sequence)

1. Launcher: recipe_repos config + src/recipe_repos.rs (clone/pull
   • merge + best-effort) + repos subcommand + the repos.json registry + archive/unarchive. Unit tests with file:// repos.
2. Launcher: promote subcommand (--dry-run/--apply, the git mv, the collision refusal). Unit tests with file:// repos.
3. Launcher: --image-source {repo,registry} (the hybrid pull). A unit test for the tag + checksum verification.
4. Docker image (docker-images/sm121-addon/ + the entrypoint + the supervisor re-sync) + a make addon-image target + the smoke test.
5. HA add-on (sm121-ha-integration/ restructured: config.yaml + Dockerfile + the options schema) + the README (the install, the network reachability, the single-launcher rule).
6. Recipe repos (the operator creates recipes-production + recipes-playground on git.kilork.org; a seed script copies the current recipes/ + config.toml + docker-images/ into them).
7. Docs (the AGENTS.md add-on section, the DEVELOPMENT.md architecture note, the add-on README).

Out of scope (later phases)

• HA entities (a thin custom integration over the add-on's /api/… for power/serve/stop/health) — the dashboard (port 3100) is enough for phase 1; the integration is a phase-2 nicety.
• Auto-branching of the playground repo (a PR per recipe) — the operator drives branches today; automation is later.
• A web UI for the repo registry / promotion (the CLI is enough for phase 1).

Addendum (2026-10-11): the HAOS build variant — the Pi is the build host

The HA on the target is HAOS on a Raspberry Pi — no Docker on the host, so the image cannot be built there classically. The add-on image is therefore thin (debian-slim + git/jq + the pinned Rust toolchain, no sm121 binary): the entrypoint compiles sm121 from the launcher repo on first boot (sm121_launcher_url @ sm121_launcher_rev, default main; a commit hash pins the binary), caching the binary + the cargo target dir in the add-on's data folder ($HOME). Properties: the arch always matches (x86_64 HA hosts work with a multi-arch image push), the binary never goes stale, and only the thin image needs a registry push. First boot is a full release build (minutes on a Pi 5, longer on a Pi 4; sm121_build_jobs defaults to 2 so a Pi 4 doesn't OOM under HA). Supersedes the original multi-stage (binary baked in) image for the add-on flow; the smoke test compiles hermetically from a file:// launcher checkout.

Addendum (2026-10-11, 2nd): the cluster config moves to a PRIVATE repo

The recipe repo is public — so it must not carry the cluster config ([[clusters]] with private node IPs / host keys / HA entity names). It lives in a private sm121/cluster-config git repo, fetched by the entrypoint at boot (sm121_base_config URL + sm121_base_config_token Gitea PAT, token embedded in the clone URL and never logged) and used as the base config. Being private, that repo may carry the cluster-side secrets ([home_assistant] token, [mikrotik] password). The public recipe repo carries only recipes/ + docker-images/.

Addendum (2026-10-11, 3rd): cluster config as an HA option (simplification)

The private cluster-config repo was dropped as over-complicated. The cluster config ([[clusters]]/nccl with the private IPs, [home_assistant], [mikrotik], [registry]) is supplied as the sm121_config_extra add-on option — raw TOML appended verbatim to the generated config. HA option storage is private and lives on the Pi, so the public recipe repo stays config-free without a second repo or a PAT. Caveat: if that TOML carries [home_assistant]/[registry], the matching flat options (sm121_ha_url/token, sm121_registry_base) must stay empty — duplicate TOML tables fail the parse.