|
@@ -115,15 +115,15 @@ failure mode this kills.
|
|
|
109.5) survives the ~12 GiB of starting vLLM processes on top of
|
|
109.5) survives the ~12 GiB of starting vLLM processes on top of
|
|
|
`drop_caches` (which already ran post-sync, pre-container, default on).
|
|
`drop_caches` (which already ran post-sync, pre-container, default on).
|
|
|
|
|
|
|
|
-Pending: (1) b12x O_DIRECT loader test (`--load-format b12x`, recipe
|
|
|
|
|
-`glm53-flash-r28-4x-dcp4-b12xload` ready) — the r28 b12x added
|
|
|
|
|
-`B12xModelLoader` (O_DIRECT reads into CUDA managed memory; weight loads stop
|
|
|
|
|
-polluting the page cache — relevant to the memory gate above); (2) Discord
|
|
|
|
|
-env vars `VLLM_GLM53_SPLIT_TARGET_BLOCK_SIZE=2048` /
|
|
|
|
|
-`VLLM_GLM53_SPLIT_MAMBA_BLOCK_SIZE=256` — split GLM-5.3 KV into independent
|
|
|
|
|
-target-MLA (2048) + recurrent-Mamba (256) page pools, requires
|
|
|
|
|
-`mamba_cache_mode='align'`, fine-grained prefix reuse; research done in the
|
|
|
|
|
-fork source (`vllm/platforms/interface.py:811`), test pending.
|
|
|
|
|
|
|
+Pending: (1) b12x O_DIRECT loader test — DONE, blocked upstream: `--load-format b12x`
|
|
|
|
|
+boots the loader (O_DIRECT, 44 shards, 92.6 GB physical, 404.1 s execution,
|
|
|
|
|
+40.07/44.59 GiB into write-combined shared storage) but dies at
|
|
|
|
|
+`b12x/loader/_checkpoint.py:370`: `ValueError: checkpoint destination was not
|
|
|
|
|
+allocated as a weight: model.language_model.layers.45.eh_proj.weight` — the
|
|
|
|
|
+GLM/Step-3.5-MTP `eh_proj` linear is missing from the pool allocation plan.
|
|
|
|
|
+Report to the fork owner; keep `--load-format instanttensor` for GLM-5.3
|
|
|
|
|
+NVFP4 meanwhile. Test recipe kept: `glm53-flash-r28-4x-dcp4-b12xload`
|
|
|
|
|
+(re-test once the fork fix lands, then delete). (2) Discord env vars
|
|
|
|
|
|
|
|
## Ops notes / lessons
|
|
## Ops notes / lessons
|
|
|
|
|
|
