|
|
@@ -141,3 +141,52 @@ OK, 155K-token needle OK.
|
|
|
- A new llama-benchy perf table that is 10×+ below the rest of the fleet's
|
|
|
numbers: suspect a per-request cold path on the server (check
|
|
|
prefix-cache behavior + server-side stats), not the harness.
|
|
|
+
|
|
|
+## Second half of the shift (post power-off, ~06:00+)
|
|
|
+
|
|
|
+User re-engaged: finish the two recipes (Aiden + Qwen) with complete results,
|
|
|
+then the new **EXL3 2x** task (temporary 2 GB headless-RAM unlock via
|
|
|
+recipe scripts).
|
|
|
+
|
|
|
+- **Aiden recipe TOML fixed** — the header-comment edit had dropped the `#`
|
|
|
+ on line 29 → tomllib `Expected '=' after a key`. Repaired, validates
|
|
|
+ (38 args / 40 env).
|
|
|
+- **Perf table missing from .md reports root-caused**: tool-eval-bench v2.7.0
|
|
|
+ `--json-file` implies `--json` → `_run_json` dispatch path drops
|
|
|
+ `throughput_samples` → the "## Throughput Metrics" section is absent from
|
|
|
+ the md (and the perf data from the JSON); the table survives only in the
|
|
|
+ `.log`. Fixed launcher-side: `src/bench_report.rs`
|
|
|
+ (`ensure_throughput_section`) re-attaches the llama-benchy table parsed
|
|
|
+ from the bench `.log` into the copied `.md` (harness pin untouched).
|
|
|
+ Retroactively applied to the Aiden `…_015216` and Qwen `…_022931` reports.
|
|
|
+ Commit 4546237.
|
|
|
+- **EXL3 2x recipe** (`dsv41-flash-exl3-2x`, coolbho3k rc2 image
|
|
|
+ `ghcr.io/coolbho3k/deepseek-v4.1-flash-exl3-3bpw-2x-dgx-spark:20260917-rc2`):
|
|
|
+ - User started the model download on dgx5 **local** — 636 GB does not fit
|
|
|
+ (251 G free) → killed, redirected to the NAS
|
|
|
+ (`/mnt/nas/ai/huggingface/hub/…`, hub layout). The 2x pair serves from
|
|
|
+ the NAS; drafter overlay `coolbho3k/DeepSeek-V4.1-Flash-DSpark-EXL3-3bpw`
|
|
|
+ downloaded too (small, done first).
|
|
|
+ - Runtime requirements extracted from the pinned repo (commit 878e0ee):
|
|
|
+ model must be mounted at exactly `/model` (drafter in `/model/draft`,
|
|
|
+ engrams in `/model/engrams`), drafter overlay at
|
|
|
+ `DS41_DRAFT_EXL3_PATH=/draft-exl3`, engine call
|
|
|
+ `serve.py serve /model --config conservative.yaml --worker-cls
|
|
|
+ combined_worker.CombinedWorker --disable-custom-all-reduce …`, hard
|
|
|
+ serving cgroup guard (9 GiB / no-swap / ≤6 CPU), 2 GB display reserve
|
|
|
+ (nvidia_drm modeset=1 fbdev=0) for the 1.75 GiB display-pool KV.
|
|
|
+ - Launcher extensions (all committed): per-recipe `devices = [...]`
|
|
|
+ (`--device d:d`), `memory_limit`/`cpus` (→ `--memory/--memory-swap/--cpus`),
|
|
|
+ built-in scripts `unlock_display_ram` / `restore_display_ram` (temporary
|
|
|
+ per-boot remod, no UEFI/modprobe.d/initramfs changes; restore on stop),
|
|
|
+ `--cluster auto` now honors `recipe.cluster` on the docker path (tmux
|
|
|
+ path already did — parity fix).
|
|
|
+ - Recipe: NAS hub cache + per-node before_serve symlinks
|
|
|
+ (`/home/kilork/exl3-model` → current refs/main snapshot) bound ro at
|
|
|
+ `/model` / `/draft-exl3`; 41 DS41/NCCL envs; 200G pair dgx1+dgx2
|
|
|
+ (powered on for this), 1M ctx, GMU 0.92, C2.
|
|
|
+ - Unlock/restore scripts validated live on dgx2 (default state was
|
|
|
+ modeset=N fbdev=Y → remod → modeset=Y fbdev=N + /dev/dri/card0 →
|
|
|
+ restore back).
|
|
|
+ - In flight at this point: model download ~55/636 GB on the NAS, image
|
|
|
+ pull on dgx1.
|
