소스 검색

feat(glm52): build on production-hybrid-1.3 base with pinned deps

Switch the glm52 overlay image to the
aidendle94/sparkrun-vllm-ds4-gb10:production-hybrid-1.3 base and pin
nvidia-nccl-cu13==2.30.4 and instanttensor==0.1.9 (no-deps,
force-reinstall) on top of it.

The glm52-hybrid-fp8-mxfp4 recipe now builds/uses the local
sparkrun-vllm-ds4-gb10:glm52 tag (docker_image_dir = glm52-int4-int8),
renames the served model to 'glm52', and points the MTP speculative
draft at the pre-staged local path /root/.cache/huggingface/links/
mtp-draft.
Alexander Korolev 3 달 전
부모
커밋
75a7055af5
2개의 변경된 파일과 9개의 추가작업 그리고 5개의 파일을 삭제
  1. 5 2
      docker-images/glm52-int4-int8/Dockerfile
  2. 4 3
      recipes/glm52-hybrid-fp8-mxfp4/recipe.toml

+ 5 - 2
docker-images/glm52-int4-int8/Dockerfile

@@ -1,4 +1,4 @@
-# GLM-5.2 Int4-Int8Mix overlay on sparkrun-vllm-ds4-gb10:fixed
+# GLM-5.2 Int4-Int8Mix overlay on sparkrun-vllm-ds4-gb10:glm52
 #
 # Unsets VLLM_PREFIX_CACHE_RETENTION_INTERVAL (baked at 4096 in the base
 # deepseek image, breaks GLM-5.2) via a wrapper that sources the original
@@ -8,7 +8,10 @@
 #   cd docker-images/glm52-int4-int8 && bash build.sh
 #   Tags: sparkrun-vllm-ds4-gb10:glm52
 
-FROM sparkrun-vllm-ds4-gb10:fixed
+FROM aidendle94/sparkrun-vllm-ds4-gb10:production-hybrid-1.3
+
+RUN pip install --no-cache-dir --no-deps --force-reinstall nvidia-nccl-cu13==2.30.4 \
+ && pip install --no-cache-dir --no-deps --force-reinstall instanttensor==0.1.9
 
 # Wrapper: unset the baked var, then hand off to the original entrypoint.
 # The original nvidia_entrypoint.sh does `exec "$@"` at the end, so this

+ 4 - 3
recipes/glm52-hybrid-fp8-mxfp4/recipe.toml

@@ -26,19 +26,20 @@
 
 engine = "vllm"
 name = "GLM-5.2 Hybrid FP8-MXFP4 (vLLM)"
-docker_image = "aidendle94/sparkrun-vllm-ds4-gb10:production-hybrid-1.2"
+docker_image = "sparkrun-vllm-ds4-gb10:glm52"
 model = "aidendle94/GLM-5.2-Hybrid-FP8-MXFP4"
 required_nodes = 4
 tensor_parallel_size = 4
 pipeline_parallel_size = 1
 distributed_backend = "mp"
 shm_size = "10gb"
+docker_image_dir = "glm52-int4-int8"
 
 entrypoint = ["vllm", "serve"]
 
 args = [
     # Model name
-    "--served-model-name", "glm-5.2",
+    "--served-model-name", "glm52",
     # General
     "--trust-remote-code",
     "--reasoning-parser", "glm45",
@@ -63,7 +64,7 @@ args = [
     "--gpu-memory-utilization", "0.89",
     # Speculative decode — MTP k=4 with adaptive depths
     "--speculative-config", """
-'{"model":"aidendle94/GLM-5.2-Hybrid-FP8-MXFP4/mtp-draft","method":"mtp","num_speculative_tokens":4,"draft_attention_backend":"B12X_MLA_SPARSE","adaptive_speculative_tokens_window":32}'""",
+'{"model":"/root/.cache/huggingface/links/mtp-draft","method":"mtp","num_speculative_tokens":4,"draft_attention_backend":"B12X_MLA_SPARSE","adaptive_speculative_tokens_window":32}'""",
     # Compilation — FULL_AND_PIECEWISE with all custom ops, 8 capture sizes
     "--compilation-config", """
 '{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"],"cudagraph_capture_sizes":[1,2,3,4,5,6,8]}'""",