Pārlūkot izejas kodu

fix(deepseek-v4): add NCCL ABI fix + instanttensor overlay image

Base image aidendle94/sparkrun-vllm-ds4-gb10:production-3.75 ships
nvidia-nccl-cu13 2.29.7 but the stack expects 2.30.4 ABI, breaking
NCCL collectives. Re-align pip wheel and add instanttensor for
faster model loading (load-format instanttensor).

- New Dockerfile in docker-images/deepseek-v4-flash-dspark/
- Tag built locally as sparkrun-vllm-ds4-gb10:fixed
- Recipe updated: docker_image + docker_image_dir
Alexander Korolev 3 mēneši atpakaļ
vecāks
revīzija
57d686b17b

+ 19 - 0
docker-images/deepseek-v4-flash-dspark/Dockerfile

@@ -0,0 +1,19 @@
+# syntax=docker/dockerfile:1.7
+# DeepSeek-V4-Flash-DSpark — NCCL ABI fix + instanttensor on top of
+# aidendle94/sparkrun-vllm-ds4-gb10:production-3.75
+#
+# The base image ships nvidia-nccl-cu13 2.29.7 but the rest of the
+# CUDA 12.1 stack expects the 2.30.4 ABI.  This mismatch breaks NCCL
+# collectives (AllGather etc.).  Re-align and add instanttensor for
+# faster model loading (load-format instanttensor).
+#
+# Build: cd docker-images/deepseek-v4-flash-dspark && docker build -t sparkrun-vllm-ds4-gb10:fixed .
+# Tag:   docker tag sparkrun-vllm-ds4-gb10:fixed aidendle94/sparkrun-vllm-ds4-gb10:production-3.75
+
+FROM aidendle94/sparkrun-vllm-ds4-gb10:production-3.75
+
+RUN pip install --no-cache-dir --no-deps --force-reinstall nvidia-nccl-cu13==2.30.4 \
+ && pip install --no-cache-dir --no-deps --force-reinstall instanttensor==0.1.9
+
+ENTRYPOINT []
+CMD ["bash"]

+ 2 - 1
recipes/deepseek-v4-flash-dspark/recipe.toml

@@ -11,13 +11,14 @@
 
 engine = "vllm"
 name = "DeepSeek V4 Flash DSpark (vLLM)"
-docker_image = "aidendle94/sparkrun-vllm-ds4-gb10:production-3.75"
+docker_image = "sparkrun-vllm-ds4-gb10:fixed"
 model = "deepseek-ai/DeepSeek-V4-Flash-DSpark"
 required_nodes = 2
 tensor_parallel_size = 2
 pipeline_parallel_size = 1
 distributed_backend = "mp"
 shm_size = "32gb"
+docker_image_dir = "deepseek-v4-flash-dspark"
 
 entrypoint = ["vllm", "serve"]