Kaynağa Gözat

refactor(recipe): use relative symlinks instead of rsync for NF4 expert overlay

The Lloyd L*.safetensors (385 GB) were being copied into the MXFP4 snapshot dir via rsync. Replace with relative symlinks (ln -snrf): ~75 links, no data copy — mirrors how HF's own cache works (snapshot files are already relative symlinks to ../../blobs/<hash>).

Relative (not absolute) is mandatory: the HF cache is mounted at /root/.cache/huggingface inside the container vs /home/<user>/.cache/huggingface on the host. Absolute symlinks would break under the path rewrite; relative paths resolve the same under both roots (verified on dgx1.lan: ln -snrf produces ../../../models--aidendle94--GLM-5.2-NF4-Lloyd-Experts-GPTQ/blobs/<hash>).

The -n flag is critical: without it, ln dereferences the existing MXFP4 symlink and replaces the BLOB it points to (corrupting the 390 GB MXFP4 cache). -n treats the symlink as a normal file and replaces the link itself. Idempotent — safe to re-run after huggingface-cli download refreshes the snapshot.
Alexander Korolev 2 ay önce
ebeveyn
işleme
51ed390d64
1 değiştirilmiş dosya ile 14 ekleme ve 4 silme
  1. 14 4
      recipes/glm52-nf4-lloyd-experts-gptq/recipe.toml

+ 14 - 4
recipes/glm52-nf4-lloyd-experts-gptq/recipe.toml

@@ -33,14 +33,24 @@
 #        huggingface-cli download aidendle94/GLM-5.2-MXFP4-Experts-GPTQ
 #        huggingface-cli download aidendle94/GLM-5.2-NF4-Lloyd-Experts-GPTQ
 #      (MXFP4 base ~390 GB, Lloyd experts ~385 GB)
-#   2. Overlay the Lloyd NF4 experts onto the MXFP4 snapshot directory:
+#   2. Overlay the Lloyd NF4 experts onto the MXFP4 snapshot directory (same
+#      filenames, same tensor names — they replace the E2M1 experts with
+#      NF4-packed codes). Use RELATIVE symlinks (ln -snrf), not copies: the
+#      HF cache is mounted at /root/.cache/huggingface inside the container
+#      (vs /home/<user>/.cache/huggingface on the host), so absolute symlinks
+#      would break — relative paths resolve the same under both roots, like
+#      HF's own snapshot→blob links. -n is critical: it stops ln from
+#      dereferencing the existing MXFP4 symlink (without -n, ln would replace
+#      the BLOB the link points to, corrupting the 390 GB MXFP4 cache).
+#      Lightweight: ~75 symlinks, no 385 GB copy.
 #        MXFP4=$(readlink -f ~/.cache/huggingface/hub/models--aidendle94--GLM-5.2-MXFP4-Experts-GPTQ/snapshots/main)
 #        LLOYD=$(readlink -f ~/.cache/huggingface/hub/models--aidendle94--GLM-5.2-NF4-Lloyd-Experts-GPTQ/snapshots/main)
-#        cp "$LLOYD"/L*.safetensors "$MXFP4"/
+#        for f in "$LLOYD"/L*.safetensors; do ln -snrf "$f" "$MXFP4/$(basename "$f")"; done
 #      Without this overlay the kernel decodes E2M1 weights with the NF4
 #      codebook → garbage output (the model card warns about this).
-#   3. Create the NF4 layer-list JSON in the Lloyd snapshot dir:
-#        echo '{"layers":[3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77]}' > "$LLOYD"/nf4_layers.json
+#   3. Create the NF4 layer-list JSON in the Lloyd snapshot dir (the kernel
+#      reads this via VLLM_NF4_LAYERS — {{lloyd_experts}}/nf4_layers.json):
+#        echo "{\"layers\":[$(seq -s, 3 77)]}" > "$LLOYD"/nf4_layers.json
 #
 # REFERENCE:
 #   https://huggingface.co/aidendle94/GLM-5.2-NF4-Lloyd-Experts-GPTQ