| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408 |
- //! Recipe repos: git repos holding `recipes/` (+ `docker-images/` build
- //! contexts), cloned/pulled by the launcher into a local cache and merged
- //! into the recipe search path (first repo wins on a name collision — see
- //! `Config.recipe_repos`).
- //!
- //! Archive state lives in a separate JSON registry (`repos.json` next to
- //! the global config): a `[[recipe_repos]]` entry can be archived without
- //! editing the config (`sm121 archive <url>` / `unarchive`), and the next
- //! sync skips archived repos. A repo that is unreachable at sync time
- //! degrades to a warning — a dead playground repo must not take down
- //! production serving.
- use crate::cli::ReposCommands;
- use crate::config::{RecipeRepo, default_config_dir};
- use anyhow::{Context, Result};
- use serde::{Deserialize, Serialize};
- use sha2::{Digest, Sha256};
- use std::path::{Path, PathBuf};
- use tracing::{info, warn};
- /// Stamp file written into a synced checkout after each successful
- /// clone/pull (unix-epoch seconds; shown by `sm121 repos`).
- pub const LAST_SYNC_FILE: &str = ".sm121-last-sync";
- /// Per-repo cache dir root: `~/.cache/sm121/recipe-repos`.
- pub fn cache_base() -> PathBuf {
- dirs::home_dir()
- .unwrap_or_else(|| PathBuf::from("."))
- .join(".cache/sm121/recipe-repos")
- }
- /// `sha256(url)` hex (full; callers truncate for dir names). Stable per
- /// URL — the cache dir is keyed on the repo URL, not the hostname.
- pub(crate) fn url_hash(url: &str) -> String {
- let digest = Sha256::digest(url.as_bytes());
- digest.iter().map(|b| format!("{:02x}", b)).collect()
- }
- /// Cache checkout dir for one repo: `<base>/<sha256(url)16>`.
- pub fn repo_cache_dir(base: &Path, url: &str) -> PathBuf {
- let h = url_hash(url);
- base.join(&h[..16])
- }
- /// Run `git <args>` locally, capturing output. Stdout on success; an
- /// anyhow error carrying the stderr tail on failure.
- pub(crate) fn git(args: &[&str]) -> Result<String> {
- let out = std::process::Command::new("git")
- .args(args)
- .output()
- .with_context(|| {
- format!(
- "failed to run git (first arg: '{}')",
- args.first().unwrap_or(&"")
- )
- })?;
- if !out.status.success() {
- let err = String::from_utf8_lossy(&out.stderr).trim().to_string();
- anyhow::bail!(fluent_i18n::t!(
- "git-failed",
- { "cmd" => args.first().copied().unwrap_or("").to_string(), "err" => err }
- ));
- }
- Ok(String::from_utf8_lossy(&out.stdout).trim().to_string())
- }
- /// Clone URL with the optional token embedded (HTTPS only — for ssh URLs
- /// the token is a no-op and the URL is used as-is). The token lives only
- /// in the cached checkout's `.git/config`, never in a recipe repo.
- fn auth_url(repo: &RecipeRepo) -> String {
- if let Some(t) = &repo.token
- && (repo.url.starts_with("https://") || repo.url.starts_with("http://"))
- {
- return repo.url.replace("//", &format!("//{t}@"));
- }
- repo.url.clone()
- }
- /// The push URL for a repo (the token embedded for HTTPS pushes) — used
- /// by `promote` so the push authenticates without a credential store.
- pub fn push_url(repo: &RecipeRepo) -> String {
- auth_url(repo)
- }
- /// Clone (first time) or `pull --ff-only` (subsequent) one repo into its
- /// cache dir; stamp `LAST_SYNC_FILE` on success. `force` re-clones from
- /// scratch (drops the cached checkout first).
- pub fn sync_one(url: &str, dir: &Path, force: bool) -> Result<()> {
- if force && dir.exists() {
- std::fs::remove_dir_all(dir)
- .with_context(|| format!("removing cached checkout {}", dir.display()))?;
- }
- if dir.join(".git").is_dir() {
- git(&["-C", &dir.display().to_string(), "pull", "--ff-only"])?;
- } else {
- if dir.exists() {
- // partial clone from an interrupted run — start clean
- std::fs::remove_dir_all(dir)?;
- }
- if let Some(parent) = dir.parent() {
- std::fs::create_dir_all(parent)?;
- }
- git(&["clone", "--", url, &dir.display().to_string()])?;
- }
- let ts = crate::stamp::now_secs().to_string();
- let stamp_path = dir.join(LAST_SYNC_FILE);
- std::fs::write(&stamp_path, ts)
- .with_context(|| format!("stamping {}", stamp_path.display()))?;
- Ok(())
- }
- /// The archive registry: which `[[recipe_repos]]` URLs are currently
- /// archived (skipped by sync). Missing file → a fresh empty registry; a
- /// corrupt file is moved to `repos.json.bak` and replaced (the launcher
- /// never crashes on the registry).
- #[derive(Clone, Debug, Default, Deserialize, Serialize)]
- pub struct ReposRegistry {
- #[serde(default)]
- pub archived: Vec<ArchiveEntry>,
- }
- #[derive(Clone, Debug, Deserialize, Serialize)]
- pub struct ArchiveEntry {
- pub url: String,
- /// Unix-epoch seconds the entry was archived.
- #[serde(default)]
- pub at: i64,
- }
- impl ReposRegistry {
- pub fn is_archived(&self, url: &str) -> bool {
- self.archived.iter().any(|e| e.url == url)
- }
- }
- /// `repos.json` next to the global config (`~/.config/sm121/` on Linux,
- /// `~/Library/Application Support/sm121/` on macOS).
- pub fn registry_path() -> Option<PathBuf> {
- default_config_dir().map(|d| d.join("repos.json"))
- }
- /// Load the registry at `path` (missing → fresh; corrupt → `.bak` + fresh
- /// + a warning).
- pub fn load_registry(path: &Path) -> ReposRegistry {
- match std::fs::read_to_string(path) {
- Ok(s) => match serde_json::from_str(&s) {
- Ok(r) => r,
- Err(e) => {
- warn!(
- "{}",
- fluent_i18n::t!(
- "repos-registry-corrupt",
- { "path" => path.display().to_string(), "err" => e.to_string(), "bak" => path.with_extension("json.bak").display().to_string() }
- )
- );
- let bak = path.with_extension("json.bak");
- if std::fs::copy(path, &bak).is_err() {
- warn!(
- "{}",
- fluent_i18n::t!("repos-registry-backup-failed", { "path" => path.display().to_string() })
- );
- }
- ReposRegistry::default()
- }
- },
- Err(_) => ReposRegistry::default(),
- }
- }
- pub fn save_registry(path: &Path, reg: &ReposRegistry) -> Result<()> {
- if let Some(parent) = path.parent() {
- std::fs::create_dir_all(parent)?;
- }
- std::fs::write(path, serde_json::to_string_pretty(reg)?)?;
- Ok(())
- }
- /// `sm121 archive <url>`: flip the entry in `repos.json` (idempotent).
- pub fn archive_at(path: &Path, url: &str) -> Result<()> {
- let mut reg = load_registry(path);
- if !reg.is_archived(url) {
- reg.archived.push(ArchiveEntry {
- url: url.to_string(),
- at: crate::stamp::now_secs(),
- });
- save_registry(path, ®)?;
- }
- Ok(())
- }
- /// `sm121 unarchive <url>`: drop the entry (a no-op when absent).
- pub fn unarchive_at(path: &Path, url: &str) -> Result<()> {
- let mut reg = load_registry(path);
- let before = reg.archived.len();
- reg.archived.retain(|e| e.url != url);
- if reg.archived.len() != before {
- save_registry(path, ®)?;
- }
- Ok(())
- }
- /// Clone/pull every non-archived `repos` entry (list order) into `base`,
- /// best-effort: an unreachable repo warns and is skipped, the reachable
- /// ones still land. Returns the checkout roots in list order.
- pub fn sync_repos(
- repos: &[RecipeRepo],
- base: &Path,
- registry: &ReposRegistry,
- force: bool,
- ) -> Vec<PathBuf> {
- let mut roots = Vec::new();
- for repo in repos {
- if registry.is_archived(&repo.url) {
- info!(
- "{}",
- fluent_i18n::t!("recipe-repo-archived-skip", { "url" => &repo.url })
- );
- continue;
- }
- let dir = repo_cache_dir(base, &repo.url);
- match sync_one(&auth_url(repo), &dir, force) {
- Ok(()) => {
- info!(
- "{}",
- fluent_i18n::t!("recipe-repo-synced", { "url" => &repo.url, "dir" => dir.display().to_string() })
- );
- roots.push(dir);
- }
- Err(e) => {
- warn!(
- "{}",
- fluent_i18n::t!("recipe-repo-unreachable", { "url" => &repo.url, "err" => e.to_string() })
- );
- }
- }
- }
- roots
- }
- /// Clone/pull the `Config.recipe_repos` entries into the default cache
- /// base (blocking git work off the async runtime). Best-effort: per-repo
- /// failures are warnings, never errors — returns the reachable roots.
- pub async fn sync_recipe_repos(config: &crate::config::Config) -> Vec<PathBuf> {
- let repos = config.recipe_repos.clone();
- let reg_path = registry_path();
- let base = cache_base();
- let registry = reg_path.as_deref().map(load_registry).unwrap_or_default();
- tokio::task::spawn_blocking(move || {
- crate::locale::apply_global_locale();
- let _ = std::fs::create_dir_all(&base);
- sync_repos(&repos, &base, ®istry, false)
- })
- .await
- .unwrap_or_default()
- }
- /// Clone/pull ONE configured repo into its cache dir (hard failure —
- /// unlike the best-effort batch sync). Used by `promote` to make sure
- /// both checkouts exist before planning the move.
- pub fn sync_repo(repo: &RecipeRepo) -> Result<()> {
- let base = cache_base();
- let _ = std::fs::create_dir_all(&base);
- let dir = repo_cache_dir(&base, &repo.url);
- sync_one(&auth_url(repo), &dir, false)
- }
- /// Count the recipe dirs (`<root>/recipes/<name>/recipe.toml`) in a synced
- /// checkout — shown by `sm121 repos`.
- fn recipe_count(root: &Path) -> Option<usize> {
- let dir = root.join("recipes");
- let Ok(entries) = std::fs::read_dir(&dir) else {
- return None;
- };
- Some(
- entries
- .flatten()
- .filter(|e| e.path().join("recipe.toml").is_file())
- .count(),
- )
- }
- /// `sm121 repos` / `sm121 repos sync` / `sm121 archive` / `sm121 unarchive`
- /// dispatch (offline except `sync`, which shells out to git locally).
- pub async fn run(config: &crate::config::Config, cmd: &ReposCommands, dry_run: bool) -> Result<()> {
- match cmd {
- ReposCommands::List => {
- if config.recipe_repos.is_empty() {
- println!("{}", fluent_i18n::t!("repos-none-configured"));
- return Ok(());
- }
- let reg_path = registry_path().unwrap_or_else(|| PathBuf::from("repos.json"));
- let reg = load_registry(®_path);
- let base = cache_base();
- println!(
- "{:<52} {:<22} {:<18} {n}",
- fluent_i18n::t!("repos-col-repo"),
- fluent_i18n::t!("repos-col-state"),
- fluent_i18n::t!("repos-col-last"),
- n = fluent_i18n::t!("repos-col-count")
- );
- for repo in &config.recipe_repos {
- let dir = repo_cache_dir(&base, &repo.url);
- let state = if reg.is_archived(&repo.url) {
- fluent_i18n::t!("repos-state-archived")
- } else if dir.join(".git").is_dir() {
- fluent_i18n::t!("repos-state-synced")
- } else {
- fluent_i18n::t!("repos-state-not-cloned")
- };
- let last = std::fs::read_to_string(dir.join(LAST_SYNC_FILE))
- .ok()
- .map(|s| crate::stamp::fmt_ts(s.trim().parse().unwrap_or(0)))
- .unwrap_or_else(|| "—".to_string());
- let n = recipe_count(&dir)
- .map(|n| n.to_string())
- .unwrap_or_else(|| "—".into());
- println!("{:<52} {:<22} {:<18} {n}", repo.url, state, last);
- }
- }
- ReposCommands::Sync {
- force,
- watch,
- interval,
- } => {
- let force = *force;
- if dry_run {
- for repo in &config.recipe_repos {
- let dir = repo_cache_dir(&cache_base(), &repo.url);
- if force {
- println!(
- "{}",
- fluent_i18n::t!("repos-sync-dry-reclone", { "url" => &repo.url, "dir" => dir.display().to_string() })
- );
- } else {
- println!(
- "{}",
- fluent_i18n::t!("repos-sync-dry-sync", { "url" => &repo.url, "dir" => dir.display().to_string() })
- );
- }
- }
- return Ok(());
- }
- let interval_min = *interval;
- loop {
- let reg_path = registry_path().unwrap_or_else(|| PathBuf::from("repos.json"));
- let reg = load_registry(®_path);
- let base = cache_base();
- let repos = config.recipe_repos.clone();
- let n = repos.len();
- let roots =
- tokio::task::spawn_blocking(move || sync_repos(&repos, &base, ®, force))
- .await
- .unwrap_or_default();
- if roots.is_empty() {
- warn!("{}", fluent_i18n::t!("repos-none-synced"));
- }
- if !watch {
- break;
- }
- info!(
- "{}",
- fluent_i18n::t!("recipe-repo-watch", { "n" => n, "interval" => interval_min })
- );
- tokio::time::sleep(std::time::Duration::from_secs(interval_min as u64 * 60)).await;
- }
- }
- }
- Ok(())
- }
- /// `sm121 archive <url>` / `sm121 unarchive <url>` — flip the entry in the
- /// `repos.json` registry (creates the file if missing).
- pub fn archive(url: &str, unarchive: bool, dry_run: bool) -> Result<()> {
- let path = registry_path()
- .ok_or_else(|| anyhow::anyhow!(fluent_i18n::t!("archive-config-dir-fail")))?;
- if dry_run {
- if unarchive {
- println!(
- "{}",
- fluent_i18n::t!("archive-dry-unarchive", { "url" => url.to_string(), "path" => path.display().to_string() })
- );
- } else {
- println!(
- "{}",
- fluent_i18n::t!("archive-dry-archive", { "url" => url.to_string(), "path" => path.display().to_string() })
- );
- }
- return Ok(());
- }
- if unarchive {
- unarchive_at(&path, url)?;
- println!(
- "{}",
- fluent_i18n::t!("unarchive-done", { "url" => url.to_string(), "path" => path.display().to_string() })
- );
- } else {
- archive_at(&path, url)?;
- println!(
- "{}",
- fluent_i18n::t!("archive-done", { "url" => url.to_string(), "path" => path.display().to_string() })
- );
- }
- Ok(())
- }
- #[cfg(test)]
- #[path = "recipe_repos_tests.rs"]
- mod tests;
|