Explorar o código

docker: image sync inner hop over the fabric (ib_ip) when available

Alexander Korolev hai 1 mes
pai
achega
1b605f259b
Modificáronse 2 ficheiros con 13 adicións e 2 borrados
  1. 6 0
      AGENTS.md
  2. 7 2
      src/docker/sync.rs

+ 6 - 0
AGENTS.md

@@ -353,6 +353,12 @@ management, and the stop-wait verification. Raw SSH/docker is allowed only
 for one-off diagnostics the launcher cannot express (hung-worker `docker
 rm -f` by name, log forensics) — and those get noted in the shift history.
 
+Image distribution (`sync_image_to_nodes`) is node-to-node, never through
+the Mac: the source node (newest build, or `--build-node`/`--sync-image-source`)
+runs `docker save | ssh <peer> docker load` per peer, with the inner hop
+host-key verified against the pinned fingerprint and routed over the peer's
+`ib_ip` (200G fabric) when it has one, else `ssh_ip` (10G mgmt).
+
 A recipe's `docker_image` is built on demand from `docker-images/<docker_image_dir>/`
 (default `<dir>` = recipe name). `sm121 serve --build-node <n>` (or the
 standalone `sm121 build`) uploads the dir to a node, runs `bash build.sh` in a

+ 7 - 2
src/docker/sync.rs

@@ -117,6 +117,11 @@ pub async fn sync_image_to_nodes(
             "{}",
             fluent_i18n::t!("sync-syncing", { "name" => &target.name, "ip" => &target.ssh_ip })
         );
+        // Data path: node-to-node, never through the Mac (mac→node is ~32
+        // MB/s vs ~547 MB/s per stream on the 200G fabric). Prefer the
+        // target's fabric IP (ib_ip) for the inner hop; nodes without a
+        // 200G uplink fall back to the 10G mgmt IP.
+        let target_ip = target.ib_ip.clone().unwrap_or_else(|| target.ssh_ip.clone());
         // Fetch the target's full public key over a verified russh connection
         // (which enforces the pinned fingerprint), then write a per-target
         // known_hosts on the SOURCE node and run the inner ssh with it,
@@ -124,7 +129,7 @@ pub async fn sync_image_to_nodes(
         // is rejected before any auth — no key leak, no forgery.
         let pubkey = crate::ssh::fetch_host_pubkey(config, target).await?;
         let known_hosts = format!("/tmp/sm121_known_hosts_{}", target.name);
-        let entry = format!("{} {}\n", target.ssh_ip, pubkey);
+        let entry = format!("{} {}\n", target_ip, pubkey);
         let write_known = format!(
             "printf '%s' {} > {} && chmod 600 {}",
             shq(&entry),
@@ -139,7 +144,7 @@ pub async fn sync_image_to_nodes(
             shq(&known_hosts),
             shq(&config.ssh_key_path.display().to_string()),
             shq(&config.ssh_user),
-            shq(&target.ssh_ip),
+            shq(&target_ip),
             shq(&known_hosts),
         );
         run_ssh_long_running(config, src_node, pipe_cmd, false).await?;