|
|
@@ -195,3 +195,28 @@ about it.
|
|
|
A worker at `ROLE=head` crash-loops on mentat's "group already has an active
|
|
|
driver session" while the real head waits for GPUs. The entrypoint refuses to
|
|
|
start that way now, but it's the first thing to check when a boot hangs.
|
|
|
+
|
|
|
+## Changing the vLLM base (the ARG BASE in image/Dockerfile)
|
|
|
+
|
|
|
+The `experimental/` overlays are **whole-file copies** of vLLM's files (baked
|
|
|
+into the image by build.sh's generated `Dockerfile.bake`, which runs AFTER
|
|
|
+verify-base.py — a stale overlay silently overwrites the new base's files).
|
|
|
+When you move BASE to a new vLLM ref:
|
|
|
+
|
|
|
+1. Generate the bake list (`build.sh`'s python over `experimental/compose/*.yaml`)
|
|
|
+ and diff each overlay's base file: `git diff <old-ref> <new-ref> -- vllm/<rel>`.
|
|
|
+2. Rebase every drifted overlay: `new = <new-ref> file + (overlay − <old-ref>)`.
|
|
|
+ `patch` does the mechanical ones; hand-merge the collisions (they are
|
|
|
+ always in the hot files: model_runner, rejection_sampler, the MLA backends).
|
|
|
+3. Hunt the version guards: `rg "__version__|VLLM_COMMIT" experimental/` —
|
|
|
+ `draft_trunc.py` pins the engine's version string and stops the engine on
|
|
|
+ a mismatch. Nightly bases carry a `+g<sha>` scm suffix; release images
|
|
|
+ carry the bare version. Update the pin with the fail-loud semantics kept.
|
|
|
+4. Rebuild via sm121, then a scratch-container import smoke
|
|
|
+ (`python3 -c "import vllm.entrypoints.openai.api_server"`) BEFORE serving —
|
|
|
+ verify-base.py runs before the bake stage and cannot see overlay breakage.
|
|
|
+
|
|
|
+The 2026-10-15 ddd6fbca-nightly → v0.31.0 move (dead Hub nightly tags) did
|
|
|
+all three: 23/38 overlays drifted (21 auto, 2 hand-merged), the draft-trunc
|
|
|
+guard pinned `+gddd6fbca`, and an unrebaked overlay's import block killed the
|
|
|
+engine at first import.
|
