Acasă Explorează Ajutor
Autentificare
sm121
/
launcher
2
0
Bifurcare 0
Fisiere Probleme 0 Trageți solicitările 0 Wiki
Arbore: 65c1cfcdc8
Ramuri Etichete
launcher
/
docker-images
/
glm53-kindling
/
image
/
patches
Alexander Korolev 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
..
LICENSE.MiaAI-Lab 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
README.md 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
gb10_plugin_backend.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
glm47_failclosed.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
glm53-flash_SM121.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
glm53_deterministic_topk.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
glm53_dflash2_kv_groups.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
glm53_eagle3_aux.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
glm53_mtp_bf16.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
glm53_reasoning_always_parsed.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
link_cuda_headers.sh 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
mia_retarget.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
sm90_fp8_kv_dtype.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
spark_mem_trace.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
spin_wait.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
vllm-58720-routed-experts.patch 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
worker_memory_cap.py 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă

README.md

Vendored SM121 kernel patcher

glm53-flash_SM121.py is taken verbatim from MiaAI-Lab/GLM-5.3-Flash-NVFP4-Dual-DGX-Spark, MIT licensed — the licence is kept beside it as LICENSE.MiaAI-Lab.

It is what makes GLM-5.3-Flash run on GB10 at all. The stock glm53-flash-arm64-cu130 image offers only FLASHINFER_MLA_SPARSE_SM120 on compute capability 12, and that backend mandates the packed fp8_ds_mla layout with pe_dim == 64. This checkpoint is NoPE (qk_rope_head_dim: 0), so the kernel refuses and the engine dies in concat_and_cache_mla — after a full ~10 minute weight load.

The seven patches gate paths that already exist onto GB10 rather than writing new kernels. Two are the crux:

  • bake_sm90_on_capability_12 lists FLASHINFER_MLA_SPARSE_SM90 for capability 12, ahead of the SM120 entry.
  • bake_sm90_wrapper_for_gb10 relaxes that backend's capability.major == 9 to in (9, 12) and swaps FA3 for FA2 off Hopper.

That second half is the part worth internalising: the SM90 sparse-MLA backend is not inherently Hopper-only. Its FA3 kernel is, but FA2 is portable to sm_12x. An earlier reading of this tree concluded the SM90 path was unreachable on GB10 because of wgmma; that was wrong, and it cost a working configuration.

The rest handle FlashInfer's FA2 fp8 gate and CTA tile for GB10's ~101 KB smem, turn PDL off (KDA race on this hardware), and fix indexer/kpool details.

Two of its targets have moved since it was written: the indexer allocation now lives in models/glm5next/nvidia/sparse_indexer.py, and FlashInfer 0.7.0 moved the FA2 fp8 gate into mla/_batch_mla/_backends/_fa_common.py. mia_retarget.py rewrites those paths and anchors in a copy before it runs, so this file stays verbatim.

The script asserts every replacement matches exactly once and aborts otherwise, so a base-image change fails the build rather than silently producing a half-patched tree. Drop it once these land upstream.