Strona główna Odkrywaj Pomoc
Zaloguj się
sm121
/
recipes
2
0
Forkuj 0
Pliki Problemy 0 Oczekujące zmiany 0 Wiki
Drzewo: f1c95cde2d
Gałęzie Tagi
recipes
/
docker-images
/
glm53-kindling
/
experimental
/
compose
/
fixes.yaml

fixes.yaml 3.0 KB
Historia Czysty

1234567891011121314151617181920212223242526272829303132333435 
  1. # Fixes found by profiling (2026-09-27): FlashInfer's per-step rollback clone,
    2. 

  2. # the indexer's fp32 head-gate GEMM, a switch for the fused mHC kernel's token
    3. 

  3. # limit, and a Triton sparse MLA kernel in place of FlashInfer's FA2 path
    4. 

  4. # (VLLM_TRITON_SPARSE_MLA=1, about 3.3x faster on GB10). kda.py and
    5. 

  5. # causal_conv1d.py: the KDA short conv writes q, k and v as separate dense
    6. 

  6. # tensors, so FlashKDA no longer copies them (~51 ms per 16k-token step).
    7. 

  7. # kv_cache_*.py, worker_utils.py, warmup.py: the drafter's KV gets its own
    8. 

  8. # small pool, paged 256 tokens at a time, instead of a page in every KV block
    9. 

  9. # (VLLM_GLM5NEXT_DRAFT_POOL=1, VLLM_GLM5NEXT_DRAFT_BLOCK_SIZE).
    10. 

  10. # tilelang_kernels.py and mhc_tilelang.py: the fused mHC decode kernel reads
    11. 

  11. # its projection weights (hc_*_fn, stored bf16 in the checkpoint) as a cached
    12. 

  12. # bf16 copy instead of the model's fp32 one, which halves its weight reads and
    13. 

  13. # leaves every product unchanged.
    14. 

  14. services:
    15. 

  15.   glm53:
    16. 

  16.     volumes:
    17. 

  17.       - ../experimental/fixes/_fa_common.py:/usr/local/lib/python3.12/dist-packages/flashinfer/mla/_batch_mla/_backends/_fa_common.py:ro
    18. 

  18.       - ../experimental/fixes/attention.py:/usr/local/lib/python3.12/dist-packages/vllm/models/glm5next/common/attention.py:ro
    19. 

  19.       - ../experimental/fixes/tilelang_kernels.py:/usr/local/lib/python3.12/dist-packages/vllm/model_executor/kernels/mhc/tilelang_kernels.py:ro
    20. 

  20.       - ../experimental/fixes/mhc_tilelang.py:/usr/local/lib/python3.12/dist-packages/vllm/model_executor/kernels/mhc/tilelang.py:ro
    21. 

  21.       - ../experimental/fixes/flashinfer_mla_sparse_sm90.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/mla/flashinfer_mla_sparse_sm90.py:ro
    22. 

  22.       - ../experimental/fixes/gb10_sparse_mla.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/mla/gb10_sparse_mla.py:ro
    23. 

  23.       - ../experimental/fixes/gdn_attn.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/gdn_attn.py:ro
    24. 

  24.       - ../experimental/fixes/kda.py:/usr/local/lib/python3.12/dist-packages/vllm/models/glm5next/common/kda.py:ro
    25. 

  25.       - ../experimental/fixes/causal_conv1d.py:/usr/local/lib/python3.12/dist-packages/vllm/model_executor/layers/mamba/ops/causal_conv1d.py:ro
    26. 

  26.       - ../experimental/fixes/kv_cache_utils.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/core/kv_cache_utils.py:ro
    27. 

  27.       - ../experimental/fixes/kv_cache_coordinator.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/core/kv_cache_coordinator.py:ro
    28. 

  28.       - ../experimental/fixes/kv_cache_interface.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/kv_cache_interface.py:ro
    29. 

  29.       - ../experimental/fixes/worker_utils.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/utils.py:ro
    30. 

  30.       - ../experimental/fixes/warmup.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu/warmup.py:ro
    31. 

  31.     environment:
    32. 

  32.       - VLLM_MHC_FUSED_MAX_TOKENS=${VLLM_MHC_FUSED_MAX_TOKENS:-32}
    33. 

  33.       - VLLM_TRITON_SPARSE_MLA=${VLLM_TRITON_SPARSE_MLA:-1}
    34. 

  34.       - VLLM_GLM5NEXT_DRAFT_POOL=${VLLM_GLM5NEXT_DRAFT_POOL:-1}
    35. 

  35.       - VLLM_GLM5NEXT_DRAFT_BLOCK_SIZE=${VLLM_GLM5NEXT_DRAFT_BLOCK_SIZE:-}
    36.