Acasă Explorează Ajutor
Autentificare
sm121
/
launcher
2
0
Bifurcare 0
Fisiere Probleme 0 Trageți solicitările 0 Wiki
Arbore: e9a4ff28ef
Ramuri Etichete
launcher
/
docker-images
/
glm53-kindling
Alexander Korolev 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
..
.submodules 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
compose 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
dev 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
experimental 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
gate 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
image 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
smoketest 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
.dockerignore 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
.env.example 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
.gitignore 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
.gitmodules 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
AGENTS.md 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
KNOBS.md 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
NOTES.md 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
README.md 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
README.upstream.md 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
build.sh 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
glm53 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
model.yaml 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă
vllm 71163f27e6 kindlingai glm-5.3-flash-gx10: vendor their stack (mode B — image+patches under our rendezvous) + 3 transcribed recipes (TP4/TP2/TP6): entrypoint exec-args + all 9 experimental overlay envs 1:1, per-TP mimics of their case block (pins 26/8/26 GiB, MML 524288/163840/524288, batched 16384/8192/16384), DFlash2 spec + AdaptiveK + tp6 padded checkpoint; build.sh bakes the 39 overlay files into the image (no per-node mounts) 3 săptămâni în urmă

README.md

glm53-kindling — vendored kindlingai/glm-5.3-flash-gx10 (B-mode)

Upstream: https://github.com/kindlingai/glm-5.3-flash-gx10 @ a3c3a1d7b215 (our fork name for their tree; original mmastrac/glm-5.3-flash-4x-gx10). GLM-5.3-Flash NVFP4 on 2/4/6 GB10 boxes: stock vLLM nightly + 15 anchored patches + the experimental/ overlay stack (arx RDMA collectives, weight snapshots, adaptive-k scheduling, fp8 dense, megamoe, profiling fixes, sequence parallelism, RecoverSSM), DFlash2 drafter, mentat instead of Ray.

Status: integration mode B (2026-10-01)

Operator decision: image + patches under sm121 rendezvous first, mode A (their full entrypoint + resident mentatd daemons) only if B proves out.

What B keeps vs skips:

kept (baked/transcribed) skipped (mode A territory)
their image build (all 15 patches, FlashKDA rebuild, chat template) /entrypoint.sh orchestration + discover.sh
all nine overlay file sets — baked by build.sh, not mounted resident mentatd host daemons (ray shim, 6379-6382)
every measured knob: args + env transcribed 1:1 per TP preflight / fabric-check / status server / self-test
DFlash2 spec, AdaptiveK scheduler, per-TP KV pin + limits MODEL_HOST_DIR local-disk rule (NAS, operator call)

Recipes: recipes/glm53-kindling-{2x,4x,6x}/recipe.toml. The overlay environment: blocks from experimental/compose/*.yaml live in recipe [env], the entrypoint.sh exec vllm serve … construction in [args]. glm53-kindling == entrypoint defaults == "the order the numbers at the top [of the upstream README] were measured".

Deviations (each in the recipe headers too)

  • Executor: mp + sm121 rank flags instead of mentat's Ray surface.
  • Weights on NFS: upstream requires each box's local disk (boot-time NFS races — upstream README). We serve from the NAS HF cache (operator call; download nvidia/GLM-5.3-Flash-NVFP4 + incoai/GLM-5.3-Flash-DFlash2 land there). Freeing ~181 GiB local per node ×4 was not an option.
  • Snapshot tags are TP-scoped (fp8w4c-tp4-…): upstream snapshots are per TP rank; its single shared tag is safe only on a fixed-topology box set. TP2/TP4 recipes must never share the dir.
  • Port 30000 (API_PORT upstream default 8002), served name glm53 (upstream default, already simple).
  • Chat template stays upstream's (thinking:false → low reasoning effort; GLM-5.3-Flash has no true non-thinking mode).

Full upstream documentation (their README for mode A reference) is kept verbatim as README.upstream.md. KNOBS.md / NOTES.md / AGENTS.md are theirs and unmodified.

Build

sm121 build                        # build.sh, tag spark-glm53:v9 (default)
TAG=spark-glm53:v10 ./build.sh      # or directly on a node

Context = this directory (repo root upstream; .dockerignore from upstream applies). Dockerfile.bake/Dockerfile.sm121 are generated at build time (node-side, not committed).

Vendoring an upstream update

git clone --recursive --depth 1 https://github.com/kindlingai/glm-5.3-flash-gx10 /tmp/new
rsync -a --delete --exclude build.sh --exclude README.md … /tmp/new/ ./

then re-check: image/entrypoint.sh (knob construction — recipes mirror it), experimental/compose/*.yaml (overlays/env), KNOBS.md.

Model contract

  • nvidia/GLM-5.3-Flash-NVFP4 rev da920bb0b9f… (181 GiB, ModelOpt NVFP4 — NOT the local-inference-lab/-Spark cuts; their MoE kernels read the checkpoint's own per-projection input scales).
  • incoai/GLM-5.3-Flash-DFlash2 rev bf582e4eacc… (2.2 GiB, CC-BY-NC-ND 4.0 non-commercial — upstream flags this; same family we already serve).
  • TP3/TP6 only: a zero-padded checkpoint from experimental/tp3/make_padded.py (66/66/2304 heads don't split by 3/6).