Home Explore Help
Sign In
sm121
/
launcher
2
0
Fork 0
Files Issues 0 Pull Requests 0 Wiki
Tree: 9d72c8b804
Branches Tags
launcher
/
docker-images
/
glm53-flash-nvfp4-lab
/
Dockerfile

Dockerfile 2.6 KB
History Raw

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748 
  1. # syntax=docker/dockerfile:1.7
    2. 

  2. # GLM-5.3-Flash NVFP4 (local-inference-lab quant) on DGX Spark (SM121/GB10), TP=2.
    3. 

  3. #
    4. 

  4. # Same day-0 base + NoPE-MLA mod as docker-images/glm53-flash-nvfp4, PLUS the
    5. 

  5. # four lab-checkpoint patches from FujitsuPolycom/glm53-flash-tp2-spark
    6. 

  6. # (Apache-2.0, attribution in patches/UPSTREAM.md):
    7. 

  7. #   * model.patch               — checkpoint naming shim (attn_hc.* submodules,
    8. 

  8. #                                 forget_gate flatten, fused [3C,1,4] conv1d split)
    9. 

  9. #   * modelopt.patch            — MTP MIXED_PRECISION fix: the draft model
    10. 

  10. #                                 resolves quantization across the checkpoint /
    11. 

  11. #                                 target-mapper namespaces (without it:
    12. 

  12. #                                 KeyError model.layers.45.mtp_block...w2_weight_scale)
    13. 

  13. #   * sparse_attn_indexer*.patch — disable persistent_topk / cooperative
    14. 

  14. #                                 workspace on CC 12.x (requests 62 blocks vs 48,
    15. 

  15. #                                 128 KB smem vs GB10's 99 KB)
    16. 

  16. #
    17. 

  17. # NOT vendored (base-image or fabric specific, see recipe header): the upstream
    18. 

  18. # image-layer patches (flashkda prefill, mm-renderer), their bundled NCCL 2.30.7
    19. 

  19. # (direct-cabled ring), and the instanttensor loader (optional speed-up).
    20. 

  20. #
    21. 

  21. # Verification markers: `[quantprobe]` in modelopt.py (runtime log must show
    22. 

  22. # algo=MXFP8 for the MTP layer), `hc_attn_base` in model.py.
    23. 

  23. 

  24. ARG BASE_IMAGE
    25. 

  25. FROM ${BASE_IMAGE}
    26. 

  26. 

  27. # 1) NoPE-MLA rope-pad + sm120 sparse-MLA topk mod (anchors self-assert).
    28. 

  28. COPY patches/patch_mla.py /tmp/patch_mla.py
    29. 

  29. 

  30. # 2) FujitsuPolycom lab-checkpoint patches (unmodified upstream .patch files).
    31. 

  31. COPY patches/model.patch patches/modelopt.patch \
    32. 

  32.      patches/sparse_attn_indexer.patch patches/sparse_attn_indexer_kpool.patch /tmp/
    33. 

  33. 

  34. RUN set -eux; \
    35. 

  35.     B=/usr/local/lib/python3.12/dist-packages/vllm; \
    36. 

  36.     python3 /tmp/patch_mla.py; \
    37. 

  37.     patch "$B/models/glm5next/nvidia/model.py" /tmp/model.patch; \
    38. 

  38.     patch "$B/model_executor/layers/quantization/modelopt.py" /tmp/modelopt.patch; \
    39. 

  39.     patch "$B/model_executor/layers/sparse_attn_indexer.py" /tmp/sparse_attn_indexer.patch; \
    40. 

  40.     patch "$B/model_executor/layers/sparse_attn_indexer_kpool.py" /tmp/sparse_attn_indexer_kpool.patch; \
    41. 

  41.     grep -q quantprobe "$B/model_executor/layers/quantization/modelopt.py"; \
    42. 

  42.     grep -q hc_attn_base "$B/models/glm5next/nvidia/model.py"; \
    43. 

  43.     python3 -m py_compile \
    44. 

  44.         "$B/models/glm5next/nvidia/model.py" \
    45. 

  45.         "$B/model_executor/layers/quantization/modelopt.py" \
    46. 

  46.         "$B/model_executor/layers/sparse_attn_indexer.py" \
    47. 

  47.         "$B/model_executor/layers/sparse_attn_indexer_kpool.py"; \
    48. 

  48.     rm -f /tmp/patch_mla.py /tmp/*.patch
    49.