launcher/docker-images/glm53-kindling

glm53-kindling — vendored kindlingai/glm-5.3-flash-gx10 (B-mode)

Upstream: https://github.com/kindlingai/glm-5.3-flash-gx10 @ c748079d45e6 (re-vendored from 27cf839c2133; carries the TP3/TP6 DFlash block-drop fix

• the optional dispram display-RAM overlay). (our fork name for their tree; original mmastrac/glm-5.3-flash-4x-gx10). GLM-5.3-Flash NVFP4 on 2/4/6 GB10 boxes: stock vLLM nightly + 15 anchored patches + the experimental/ overlay stack (arx RDMA collectives, weight snapshots, adaptive-k scheduling, fp8 dense, megamoe, profiling fixes, sequence parallelism, RecoverSSM), DFlash2 drafter, mentat instead of Ray.

Status: integration mode B (2026-10-01)

Operator decision: image + patches under sm121 rendezvous first, mode A (their full entrypoint + resident mentatd daemons) only if B proves out.

What B keeps vs skips:

kept (baked/transcribed)	skipped (mode A territory)
their image build (all 15 patches, FlashKDA rebuild, chat template)	/entrypoint.sh orchestration + discover.sh
all nine overlay file sets — baked by build.sh, not mounted	resident mentatd host daemons (ray shim, 6379-6382)
every measured knob: args + env transcribed 1:1 per TP	preflight / fabric-check / status server / self-test
DFlash2 spec, AdaptiveK scheduler, per-TP KV pin + limits	MODEL_HOST_DIR local-disk rule (NAS, operator call)

Recipes: recipes/glm53-kindling-{2x,4x,6x}/recipe.toml. The overlay environment: blocks from experimental/compose/*.yaml live in recipe [env], the entrypoint.sh exec vllm serve … construction in [args]. glm53-kindling == entrypoint defaults == "the order the numbers at the top [of the upstream README] were measured".

Deviations (each in the recipe headers too)

• Executor: mp + sm121 rank flags instead of mentat's Ray surface.
• Weights on NFS: upstream requires each box's local disk (boot-time NFS races — upstream README). We serve from the NAS HF cache (operator call; download nvidia/GLM-5.3-Flash-NVFP4 + incoai/GLM-5.3-Flash-DFlash2 land there). Freeing ~181 GiB local per node ×4 was not an option.
• Snapshot tags are TP-scoped (fp8w4c-tp4-…): upstream snapshots are per TP rank; its single shared tag is safe only on a fixed-topology box set. TP2/TP4 recipes must never share the dir.
• Port 30000 (API_PORT upstream default 8002), served name glm53 (upstream default, already simple).
• Chat template stays upstream's (thinking:false → low reasoning effort; GLM-5.3-Flash has no true non-thinking mode).

Full upstream documentation (their README for mode A reference) is kept verbatim as README.upstream.md. KNOBS.md / NOTES.md / AGENTS.md are theirs and unmodified.

Note on 27cf839c2133 (#64): upstream moved spec-config assembly into the entrypoint (overlays now pass SPEC_EXTRA, a second --speculative-config is refused — adaptive-k's EXTRA_ARGS used to silently win the same way our first transcription predated the fix). B-mode recipes keep the effective merged config inline in [args], matching the pre/post-fix TP4 result upstream live-checked — no change needed, but re-transcribe from the new EXTRA/SPEC_EXTRA shape if a recipe ever grows a mode-A switch.

Build

sm121 build                        # build.sh, tag spark-glm53:v9 (default)
TAG=spark-glm53:v10 ./build.sh      # or directly on a node

Context = this directory (repo root upstream; .dockerignore from upstream applies). Dockerfile.bake/Dockerfile.sm121 are generated at build time (node-side, not committed).

Vendoring an upstream update

git clone --recursive --depth 1 https://github.com/kindlingai/glm-5.3-flash-gx10 /tmp/new
rsync -a --delete --exclude build.sh --exclude README.md … /tmp/new/ ./

then re-check: image/entrypoint.sh (knob construction — recipes mirror it), experimental/compose/*.yaml (overlays/env), KNOBS.md.

Model contract

• nvidia/GLM-5.3-Flash-NVFP4 rev da920bb0b9f… (181 GiB, ModelOpt NVFP4 — NOT the local-inference-lab/-Spark cuts; their MoE kernels read the checkpoint's own per-projection input scales).
• incoai/GLM-5.3-Flash-DFlash2 rev bf582e4eacc… (2.2 GiB, CC-BY-NC-ND 4.0 non-commercial — upstream flags this; same family we already serve).
• TP3/TP6 only: a zero-padded checkpoint from experimental/tp3/make_padded.py (66/66/2304 heads don't split by 3/6).