|
|
@@ -2,6 +2,23 @@
|
|
|
|
|
|
Rust CLI (`edition 2024`) for launching vLLM/SGLang inference containers across a DGX Spark cluster via SSH (`russh`). Docker default, tmux alternative.
|
|
|
|
|
|
+## Memory discipline (the web process runs 24/7 on a 4 GB Pi)
|
|
|
+
|
|
|
+The add-on image builds the launcher **with `--features mimalloc`** (the CI
|
|
|
+Dockerfile does this; a plain `cargo build` stays on the default allocator).
|
|
|
+On musl the default malloc never returns memory to the OS, so the web
|
|
|
+process's transient per-tick spikes (report collects, SSE serialization, log
|
|
|
+strings) ratcheted RSS to ~2.7 GiB and the Pi OOM-killed it daily
|
|
|
+(2026-10-13, dmesg `oom-kill … task=sm121`). mimalloc trims; the web
|
|
|
+process now holds ~40 MiB. Keep the add-on build on mimalloc.
|
|
|
+
|
|
|
+Related levers already in place: the report refresh loop is lazy (full
|
|
|
+re-collect only on debounced file events; the 30 s tick only re-runs the
|
|
|
+stale cross-check, riding the dashboard's fleet bus — no extra SSH), and the
|
|
|
+SSH private key is parsed once and cached (`ssh.rs`). Heap hunting:
|
|
|
+`cargo build --release --features dhat` + graceful SIGTERM exit dumps a
|
|
|
+per-callsite retained-heap `dhat.json` (summarize with `/tmp/dhat-sum.py`).
|
|
|
+
|
|
|
## Dev quick-ref
|
|
|
|
|
|
| Action | Command |
|
