kindling: re-vendor to upstream 27cf839c2133 (spec-config moved to entrypoint/SPEC_EXTRA #64 — same bug we hit; megamoe routed-expert-0 fix #67; kindling.json results #68); README notes B-mode equivalence

docker-images/glm53-kindling/AGENTS.md

@@ -111,6 +111,40 @@ The last full gate (2026-09-28):
 
 When your PR lands with better numbers, update this table in the same PR.
 
+## kindling.json
+
+`kindling.json` at the repo root is the README's results table in a form a
+program can read. The Kindling AI site reads it from `main`, so whatever is
+merged there is what people see.
+
+Keep it in step with the README:
+
+- When a PR changes the README results table, change `kindling.json` in the
+  same PR. Same numbers, same rounding. A PR that changes one and not the
+  other isn't done.
+- It mirrors the README table, not the last-full-gate table above. The gate
+  table moves with every PR; the README is what we publish.
+- One entry in `configs` per TP column. Leave out a number that wasn't
+  measured. Don't write 0, and don't fill a gap with an estimate.
+- Check it parses before you push: `python3 -m json.tool kindling.json`.
+
+The fields:
+
+| field | what |
+|---|---|
+| `title`, `variant`, `summary` | the card heading, the quant tag, one or two sentences |
+| `model`, `hardware`, `stack` | the checkpoint, the boxes, a few short tags |
+| `default_config` | the `id` of the config shown first |
+| `configs[].id`, `label`, `nodes` | e.g. `tp4`, `TP=4`, `4` |
+| `configs[].decode` | single-stream decode tok/s by workload: `code`, `prose`, `structured` |
+| `configs[].prefill` | cold prefill, `[{ "context": tokens, "tps": tok/s }]` |
+| `configs[].concurrency` | aggregate tok/s, `[{ "streams": n, "aggregate_tps": tok/s }]` |
+| `configs[].max_context`, `kv_pool_tokens`, `max_concurrent` | integers: tokens, tokens, requests |
+| `configs[].quality` | `[{ "name": ..., "value": ... }]`, shown as short text |
+| `configs[].notes` | one line of caveats |
+
+Context lengths are in tokens: 32k is `32768`, 128k is `131072`.
+
 ## Extra checks for some changes
 
 The gate covers the common cases. Some changes need more.

docker-images/glm53-kindling/KNOBS.md

@@ -68,6 +68,7 @@ Meaning and rationale stay in the entrypoint and compose comments.
 | `SERVICE_NAME` | `glm53` |
 | `SHARED_TAG` | `glm53-${_arch}-${_kv}` _(derived)_ |
 | `SKIP_MM_PROFILING` | `0` |
+| `SPEC_EXTRA` | _(empty)_ |
 | `SPEC_METHOD` | `dflash` |
 | `SPEC_TOKENS` | `7` |
 | `STAGE_FILE` | `/tmp/glm53-stage` |

docker-images/glm53-kindling/README.md

@@ -1,6 +1,6 @@
 # glm53-kindling — vendored kindlingai/glm-5.3-flash-gx10 (B-mode)
 
-Upstream: <https://github.com/kindlingai/glm-5.3-flash-gx10> @ `a3c3a1d7b215`
+Upstream: <https://github.com/kindlingai/glm-5.3-flash-gx10> @ `27cf839c2133`
 (our fork name for their tree; original mmastrac/glm-5.3-flash-4x-gx10).
 GLM-5.3-Flash NVFP4 on 2/4/6 GB10 boxes: stock vLLM nightly + 15 anchored
 patches + the `experimental/` overlay stack (arx RDMA collectives, weight
@@ -46,6 +46,15 @@ Full upstream documentation (their README for mode A reference) is kept
 verbatim as `README.upstream.md`. KNOBS.md / NOTES.md / AGENTS.md are
 theirs and unmodified.
 
+Note on `27cf839c2133` (#64): upstream moved spec-config assembly into
+the entrypoint (overlays now pass `SPEC_EXTRA`, a second
+`--speculative-config` is refused — adaptive-k's EXTRA_ARGS used to
+silently win the same way our first transcription predated the fix).
+B-mode recipes keep the effective merged config inline in `[args]`,
+matching the pre/post-fix TP4 result upstream live-checked — no change
+needed, but re-transcribe from the new EXTRA/SPEC_EXTRA shape if a
+recipe ever grows a mode-A switch.
+
 ## Build
 
 ```

docker-images/glm53-kindling/README.upstream.md

@@ -71,6 +71,10 @@ structured, level on prose, and about 5% below our RigMark prefill.
 [NOTES.md](NOTES.md) has the measurements and diagnosis behind the choices
 here.
 
+[kindling.json](kindling.json) has the results table above in a form a program
+can read. The Kindling AI site shows this recipe's numbers from it. A PR that
+changes the table changes kindling.json too.
+
 ## What you need
 
 - **Four ASUS GX10 or other GB10 boxes** (sm_121a, 128 GB unified memory), or
@@ -99,7 +103,8 @@ here.
     (`DFLASH_HOST_DIR`). It is licensed CC BY-NC-ND
     4.0, non-commercial; check that before you serve it. `SPEC_METHOD=mtp`
     uses the checkpoint's own MTP head instead, slower but with no second
-    download.
+    download. Start that without `experimental/compose/adaptive-k.yaml`, which
+    is tuned for DFlash2; the entrypoint refuses the combination.
 
   `hf download` is resumable:
 
@@ -132,6 +137,7 @@ that holds cluster membership.
 | `image/` | Dockerfile, entrypoint, patches, `verify-base.py`, `self-test.py`, chat template, `build.sh` |
 | `compose/glm53.yaml` | the model, the same file on every box |
 | `.env.example` | optional overrides for `compose/.env` |
+| `kindling.json` | the results table, machine-readable, for the Kindling AI site |
 | `smoketest/` | `run.sh <base> [served-name]` |
 | `.submodules/spark-agent` | the status server, reached through the `vllm` symlink |
 | `dev/` | not in the image: the corruption diagnosis and repros, kernel and patch tests, the step tap |
@@ -549,7 +555,9 @@ lock, [#53](https://github.com/kindlingai/glm-5.3-flash-gx10/issues/53)) ·
 [#57](https://github.com/kindlingai/glm-5.3-flash-gx10/pull/57), and the stock tool
 parser default, [#58](https://github.com/kindlingai/glm-5.3-flash-gx10/pull/58)) ·
 [tfolkman](https://github.com/tfolkman) (how a killed worker leaves the JIT lock,
-[#53](https://github.com/kindlingai/glm-5.3-flash-gx10/issues/53))
+[#53](https://github.com/kindlingai/glm-5.3-flash-gx10/issues/53)) ·
+[strusty](https://github.com/strusty) (megamoe and rows with no routed experts,
+[#63](https://github.com/kindlingai/glm-5.3-flash-gx10/issues/63))
 
 The files in `experimental/` that replace vLLM and FlashInfer files keep their
 Apache-2.0 headers, and [experimental/README.md](experimental/README.md#sources)

docker-images/glm53-kindling/compose/glm53.yaml

@@ -152,6 +152,7 @@ services:
       - MTP=${MTP:-}
       - SPEC_METHOD=${SPEC_METHOD:-}
       - SPEC_TOKENS=${SPEC_TOKENS:-}
+      - SPEC_EXTRA=${SPEC_EXTRA:-}
       - DFLASH_MODEL=${DFLASH_MODEL:-}
       - MODEL_DIR=${MODEL_DIR:-}
       # Default and KV arithmetic in the entrypoint. Raise it against the

docker-images/glm53-kindling/dev/README.md

@@ -1 +1 @@
-Dev-only material that is not in the image: the token-corruption diagnosis and its repros, kernel tests, patch tests and the step tap.
+Dev-only material that is not in the image: the token-corruption diagnosis and its repros, kernel tests, patch tests, the step tap, and `entrypoint-spec-test.sh` (the entrypoint's speculative config under each setting, no GPU needed).

docker-images/glm53-kindling/dev/entrypoint-spec-test.sh

@@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+# Tests the entrypoint's speculative config (image/entrypoint.sh, from `SPEC=()` to its "speculative:" line) without a GPU:
+# runs those exact lines under each setting and checks the one --speculative-config vLLM gets. The reference for the default
+# stack is what vLLM used before SPEC_EXTRA: the second --speculative-config adaptive-k.yaml put in EXTRA_ARGS (vLLM keeps
+# the last one). Usage: dev/entrypoint-spec-test.sh   (prints PASS or the failing case; exit status = failures)
+cd "$(dirname "$0")/.."
+block=$(sed -n '/^SPEC=()$/,/^\[\[ \${#SPEC\[@\]} -gt 0 \]\] && echo "speculative:/p' image/entrypoint.sh)
+extra=$(sed -n 's/^ *- SPEC_EXTRA=//p' experimental/compose/adaptive-k.yaml)
+d1=$(mktemp -d); d2=$(mktemp -d); touch "$d1/config.json" "$d2/config.json"; trap 'rm -rf "$d1" "$d2"' EXIT
+fails=0
+run() {  # run <env assignments...>: prints the final SPEC[1] (or nothing), exit status of the block
+  env -i PATH="$PATH" BLOCK="$BLOCK" "$@" bash -c 'eval "$BLOCK" >/dev/null; printf %s "${SPEC[1]:-}"' 2>/dev/null
+}
+export BLOCK="$block"
+check() {  # check <name> <python condition on c (the parsed config, or None) and rc>
+  local name=$1 cond=$2; shift 2
+  out=$(run "$@"); rc=$?
+  python3 -c 'import json, sys; out, rc = sys.argv[1], int(sys.argv[2]); c = json.loads(out) if out else None
+d1, d2 = sys.argv[4], sys.argv[5]; sys.exit(0 if eval(sys.argv[3]) else 1)' "$out" "$rc" "$cond" "$d1" "$d2" \
+    || { echo "FAIL $name: rc=$rc out=$out"; fails=$((fails + 1)); }
+}
+ref='{"method": "dflash", "disable_eagle_block_drop": True, "model": d1, "num_speculative_tokens": 7, "attention_backend": "TRITON_ATTN", "num_speculative_tokens_per_batch_size": [[1,1,7],[2,2,5],[3,3,4],[4,4,3],[5,5,2],[6,32,7]]}'
+check "default stack = the old overlay's config" "rc == 0 and c == $ref" SPEC_METHOD=dflash DFLASH_MODEL="$d1" SPEC_EXTRA="$extra"
+old='{"method": "dflash", "model": d1, "num_speculative_tokens": 7, "attention_backend": "TRITON_ATTN", "num_speculative_tokens_per_batch_size": [[1,1,7],[2,2,5],[3,3,4],[4,4,3],[5,5,2],[6,32,7]]}'
+for f in tp3 tp6; do  # these restate EXTRA_ARGS and SPEC_EXTRA after adaptive-k.yaml; same config as before, no block-drop key
+  check "$f.yaml = its old config" "rc == 0 and c == $old" SPEC_METHOD=dflash DFLASH_MODEL="$d1" \
+    SPEC_EXTRA="$(sed -n 's/^ *- SPEC_EXTRA=//p' experimental/compose/$f.yaml)" EXTRA_ARGS="$(sed -n 's/^ *- EXTRA_ARGS=//p' experimental/compose/$f.yaml)"
+done
+check "stock (no overlay) unchanged" "rc == 0 and out == json.dumps({'method': 'dflash', 'model': d1, 'num_speculative_tokens': 7}, separators=(',', ':'))" SPEC_METHOD=dflash DFLASH_MODEL="$d1"
+check "SPEC_TOKENS takes effect, widths capped" "rc == 0 and c['num_speculative_tokens'] == 5 and [r[2] for r in c['num_speculative_tokens_per_batch_size']] == [5,5,4,3,2,5]" SPEC_METHOD=dflash SPEC_TOKENS=5 DFLASH_MODEL="$d1" SPEC_EXTRA="$extra"
+check "DFLASH_MODEL takes effect" "rc == 0 and c['model'] == d2" SPEC_METHOD=dflash DFLASH_MODEL="$d2" SPEC_EXTRA="$extra"
+check "mtp with the DFlash overlay is refused" "rc != 0" SPEC_METHOD=mtp SPEC_EXTRA="$extra"
+check "none with the DFlash overlay is refused" "rc != 0" SPEC_METHOD=none SPEC_EXTRA="$extra"
+check "mtp without the overlay still works" "rc == 0 and c == {'method': 'mtp', 'num_speculative_tokens': 4}" SPEC_METHOD=mtp
+check "a second --speculative-config in EXTRA_ARGS is refused" "rc != 0" SPEC_METHOD=dflash DFLASH_MODEL="$d1" EXTRA_ARGS='--speculative-config {"method":"dflash"}'
+check "SPEC_EXTRA cannot set the method, model or width" "rc != 0" SPEC_METHOD=dflash DFLASH_MODEL="$d1" SPEC_EXTRA='{"model":"/x"}'
+check "SPEC_EXTRA must be a JSON object" "rc != 0" SPEC_METHOD=dflash DFLASH_MODEL="$d1" SPEC_EXTRA='[1]'
+grep -q '"${SPEC\[\*\]:-}" == \*num_speculative_tokens_per_batch_size\*' image/entrypoint.sh \
+  || { echo "FAIL CUDA graph ceiling still reads the draft table from EXTRA_ARGS"; fails=$((fails + 1)); }
+[ $fails = 0 ] && echo PASS
+exit $fails

docker-images/glm53-kindling/experimental/compose/adaptive-k.yaml

@@ -21,7 +21,10 @@ services:
       - ../experimental/draft-trunc/speculator.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu/spec_decode/speculator.py:ro
       - ../experimental/draft-trunc/dflash2_speculator.py:/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu/spec_decode/dflash2/speculator.py:ro
     environment:
-      - EXTRA_ARGS=--scheduler-cls vllm.v1.core.sched.adaptive_k.AdaptiveKScheduler --speculative-config {"method":"dflash","disable_eagle_block_drop":true,"model":"/models/glm-5.3-flash-dflash2","num_speculative_tokens":7,"attention_backend":"TRITON_ATTN","num_speculative_tokens_per_batch_size":[[1,1,7],[2,2,5],[3,3,4],[4,4,3],[5,5,2],[6,32,7]]}
+      - EXTRA_ARGS=--scheduler-cls vllm.v1.core.sched.adaptive_k.AdaptiveKScheduler
+      # Merged into the entrypoint's --speculative-config (method, drafter and
+      # width stay SPEC_METHOD, DFLASH_MODEL and SPEC_TOKENS).
+      - SPEC_EXTRA={"disable_eagle_block_drop":true,"attention_backend":"TRITON_ATTN","num_speculative_tokens_per_batch_size":[[1,1,7],[2,2,5],[3,3,4],[4,4,3],[5,5,2],[6,32,7]]}
       - VLLM_ADAPTIVE_K_CONTROL=/root/.cache/adaptive-k
       - VLLM_DRAFT_TRUNC_TAU=${VLLM_DRAFT_TRUNC_TAU:-0.3}
       - VLLM_DRAFT_TRUNC_EXPLORE=${VLLM_DRAFT_TRUNC_EXPLORE:-4}

docker-images/glm53-kindling/experimental/compose/tp3.yaml

@@ -5,7 +5,7 @@
 # load. Zero heads and zero expert columns contribute nothing. Set TP=3 and
 # point MODEL_HOST_DIR / DFLASH_HOST_DIR at the padded dirs.
 #
-# Stack LAST, after adaptive-k.yaml (it restates that file's EXTRA_ARGS) and
+# Stack LAST, after adaptive-k.yaml (it restates that file's EXTRA_ARGS and SPEC_EXTRA) and
 # after fixes.yaml / sp.yaml / recoverssm.yaml (model.py there carries the
 # load hook, a no-op without tp_pad_orig in the config).
 services:
@@ -15,8 +15,11 @@ services:
       - ../experimental/tp3/vocab_parallel_embedding.py:/usr/local/lib/python3.12/dist-packages/vllm/model_executor/layers/vocab_parallel_embedding.py:ro
     environment:
       - TP=3
-      # adaptive-k.yaml's EXTRA_ARGS plus a data-parallel vision tower: its 16
+      # adaptive-k.yaml's scheduler plus a data-parallel vision tower: its 16
       # ViT heads and 10240-wide merger do not split by 3.
-      - EXTRA_ARGS=--scheduler-cls vllm.v1.core.sched.adaptive_k.AdaptiveKScheduler --speculative-config {"method":"dflash","model":"/models/glm-5.3-flash-dflash2","num_speculative_tokens":7,"attention_backend":"TRITON_ATTN","num_speculative_tokens_per_batch_size":[[1,1,7],[2,2,5],[3,3,4],[4,4,3],[5,5,2],[6,32,7]]} --mm-encoder-tp-mode data
+      - EXTRA_ARGS=--scheduler-cls vllm.v1.core.sched.adaptive_k.AdaptiveKScheduler --mm-encoder-tp-mode data
+      # The speculative config as it was here before SPEC_EXTRA (no
+      # disable_eagle_block_drop: adaptive-k.yaml's copy never reached this layout).
+      - SPEC_EXTRA={"attention_backend":"TRITON_ATTN","num_speculative_tokens_per_batch_size":[[1,1,7],[2,2,5],[3,3,4],[4,4,3],[5,5,2],[6,32,7]]}
       # Padded weights process differently: never restore a TP=4 snapshot.
       - VLLM_WEIGHT_SNAPSHOT_TAG=tp3pad-2026-09-28

docker-images/glm53-kindling/experimental/compose/tp6.yaml

@@ -10,5 +10,8 @@ services:
       - ../experimental/tp3/vocab_parallel_embedding.py:/usr/local/lib/python3.12/dist-packages/vllm/model_executor/layers/vocab_parallel_embedding.py:ro
     environment:
       - TP=6
-      - EXTRA_ARGS=--scheduler-cls vllm.v1.core.sched.adaptive_k.AdaptiveKScheduler --speculative-config {"method":"dflash","model":"/models/glm-5.3-flash-dflash2","num_speculative_tokens":7,"attention_backend":"TRITON_ATTN","num_speculative_tokens_per_batch_size":[[1,1,7],[2,2,5],[3,3,4],[4,4,3],[5,5,2],[6,32,7]]} --mm-encoder-tp-mode data
+      - EXTRA_ARGS=--scheduler-cls vllm.v1.core.sched.adaptive_k.AdaptiveKScheduler --mm-encoder-tp-mode data
+      # The speculative config as it was here before SPEC_EXTRA (no
+      # disable_eagle_block_drop: adaptive-k.yaml's copy never reached this layout).
+      - SPEC_EXTRA={"attention_backend":"TRITON_ATTN","num_speculative_tokens_per_batch_size":[[1,1,7],[2,2,5],[3,3,4],[4,4,3],[5,5,2],[6,32,7]]}
       - VLLM_WEIGHT_SNAPSHOT_TAG=tp6pad-2026-09-29

docker-images/glm53-kindling/experimental/megamoe/megamoe_vllm.py

@@ -91,6 +91,10 @@ def prefill_routed(m, x, topk_weights, topk_ids, y8: bool = False, x4=None):
     w13, w2 = m.w13_weight, m.w2_weight
     E, H, I = w13.shape[0], w2.shape[1], w13.shape[1] // 2
     M, topk = topk_ids.shape
+    # A row with no routed experts carries id -1 (weight 0). Every expert id is used as an
+    # index below (scatter_add_, argsort/searchsorted, the fc1 gather), so point it at expert 0;
+    # its weight is 0, so finalize adds nothing for it, the same as CUTLASS skipping it.
+    topk_ids = topk_ids.masked_fill(topk_ids < 0, 0)
     from vllm import _custom_ops as ops
 
     if x4 is None:
@@ -154,6 +158,13 @@ def _buffers(M: int, topk: int, H: int, I: int, device) -> dict:
 def _apply(self, layer, x, topk_weights, topk_ids, shared_experts, shared_experts_input):
     global _check_left
     M = x.shape[0]
+    # Rows with no routed experts arrive as id -1 / weight 0 (seen in batches that mix a long
+    # prefill chunk with concurrent decode rows). CUTLASS skips them; megamoe and moe_prefill
+    # index by id, so send them to expert 0 at weight 0, which adds exactly nothing.
+    # Elementwise, no host sync.
+    neg = topk_ids < 0
+    topk_ids = topk_ids.masked_fill(neg, 0)
+    topk_weights = topk_weights.masked_fill(neg, 0)
     if (_PREFILL and M >= _PREFILL_MIN and x.dtype == torch.bfloat16 and hasattr(layer, "_moe_prefill_experts")
             and not torch.cuda.is_current_stream_capturing()):
         out = _prefill(layer, x.contiguous(), topk_weights, topk_ids)

docker-images/glm53-kindling/image/entrypoint.sh

@@ -785,6 +785,35 @@ case "$SPEC_METHOD" in
   none) ;;
   *) echo "FATAL: SPEC_METHOD=$SPEC_METHOD; want mtp, dflash or none" >&2; exit 1 ;;
 esac
+# SPEC_EXTRA: more keys for the speculative config above, as one JSON object.
+# adaptive-k.yaml puts its per-batch-size draft widths, the drafter's attention
+# backend and disable_eagle_block_drop here. They used to arrive as a second
+# --speculative-config in EXTRA_ARGS, and vLLM keeps the last one, so whenever
+# that overlay was loaded SPEC_METHOD, SPEC_TOKENS and DFLASH_MODEL were
+# silently ignored (SPEC_METHOD=mtp still served DFlash2). Draft widths above
+# SPEC_TOKENS are capped to it.
+if [[ ${#SPEC[@]} -gt 0 && "${EXTRA_ARGS:-}" == *--speculative-config* ]]; then
+  echo "FATAL: EXTRA_ARGS sets --speculative-config too. vLLM keeps the last one, so SPEC_METHOD," \
+       "SPEC_TOKENS and DFLASH_MODEL would be ignored. Put extra keys in SPEC_EXTRA (a JSON object) instead." >&2
+  exit 1
+fi
+if [[ -n "${SPEC_EXTRA:-}" ]]; then
+  if [[ "$SPEC_METHOD" != dflash ]]; then
+    echo "FATAL: SPEC_EXTRA is set (adaptive-k.yaml sets it, tuned for DFlash2) but SPEC_METHOD=$SPEC_METHOD." \
+         "Start without experimental/compose/adaptive-k.yaml to run $SPEC_METHOD." >&2
+    exit 1
+  fi
+  SPEC[1]=$(python3 -c '
+import json, sys
+spec, extra = json.loads(sys.argv[1]), json.loads(sys.argv[2])
+if not isinstance(extra, dict): sys.exit("SPEC_EXTRA must be a JSON object")
+fixed = sorted(set(extra) & {"method", "model", "num_speculative_tokens"})
+if fixed: sys.exit("SPEC_EXTRA must not set " + ", ".join(fixed) + ": use SPEC_METHOD, DFLASH_MODEL and SPEC_TOKENS")
+n = spec["num_speculative_tokens"]
+for row in extra.get("num_speculative_tokens_per_batch_size", []): row[2] = min(row[2], n)
+spec.update(extra)
+print(json.dumps(spec, separators=(",", ":")))' "${SPEC[1]}" "$SPEC_EXTRA") || { echo "FATAL: SPEC_EXTRA: see above ($SPEC_EXTRA)" >&2; exit 1; }
+fi
 [[ ${#SPEC[@]} -gt 0 ]] && echo "speculative: ${SPEC[*]}"
 
 stage loading
@@ -893,12 +922,12 @@ else
   # new prompt, so stop at the largest decode step: 32 tokens at TP=2 instead
   # of 64, for example. Those mixed steps run without a graph.
   # A draft width that varies with batch size (adaptive-k.yaml's
-  # num_speculative_tokens_per_batch_size) decodes 1 + k tokens per request
+  # num_speculative_tokens_per_batch_size, through SPEC_EXTRA) decodes 1 + k tokens per request
   # for each k in the schedule: 3, 5, 6, 10, 12, 15. vLLM adds those sizes to
   # its capture list only when it picks the ceiling itself, so an explicit cap
   # leaves them to run without a full graph. Leave the ceiling to vLLM there,
   # unless CUDAGRAPH_MAX sets one.
-  if [[ -z "${CUDAGRAPH_MAX:-}" && "${EXTRA_ARGS:-}" == *num_speculative_tokens_per_batch_size* ]]; then
+  if [[ -z "${CUDAGRAPH_MAX:-}" && "${SPEC[*]:-}" == *num_speculative_tokens_per_batch_size* ]]; then
     echo "MoE backend: ${MOE_BACKEND}, vLLM's own CUDA graph sizes (draft width varies with batch size)"
   else
     _q=1; [[ -n "$SPEC_METHOD" && "$SPEC_METHOD" != none ]] && _q=$(( SPEC_TOKENS + 1 ))

docker-images/glm53-kindling/kindling.json

@@ -0,0 +1,59 @@
+{
+  "version": 1,
+  "title": "GLM-5.3-Flash",
+  "variant": "NVFP4",
+  "summary": "nvidia/GLM-5.3-Flash-NVFP4 on vLLM with mentat in place of Ray, over RoCE. DFlash2 speculative decoding, GB10 NVFP4 MoE kernels, RDMA collectives and processed-weight snapshots for ~3 minute restarts.",
+  "model": "nvidia/GLM-5.3-Flash-NVFP4",
+  "hardware": "ASUS GX10 (GB10)",
+  "stack": ["vLLM", "mentat", "DFlash2", "RoCE"],
+  "default_config": "tp4",
+  "configs": [
+    {
+      "id": "tp2",
+      "label": "TP=2",
+      "nodes": 2,
+      "decode": { "code": 66.2, "prose": 42.6, "structured": 94.7 },
+      "prefill": [ { "context": 32768, "tps": 2931 }, { "context": 131072, "tps": 2871 } ],
+      "concurrency": [ { "streams": 4, "aggregate_tps": 83 }, { "streams": 8, "aggregate_tps": 110 } ],
+      "max_context": 160000,
+      "kv_pool_tokens": 1100000,
+      "max_concurrent": 16,
+      "quality": [ { "name": "Needle recall", "value": "6/6 up to 128k" } ]
+    },
+    {
+      "id": "tp3",
+      "label": "TP=3",
+      "nodes": 3,
+      "decode": { "code": 88.0, "prose": 54.2, "structured": 121.4 },
+      "prefill": [ { "context": 32768, "tps": 3864 }, { "context": 131072, "tps": 3736 } ],
+      "concurrency": [ { "streams": 4, "aggregate_tps": 107 }, { "streams": 8, "aggregate_tps": 138 } ],
+      "max_context": 524288,
+      "kv_pool_tokens": 1910000,
+      "max_concurrent": 64,
+      "quality": [ { "name": "Needle recall", "value": "12/12 up to 507k" } ]
+    },
+    {
+      "id": "tp4",
+      "label": "TP=4",
+      "nodes": 4,
+      "decode": { "code": 117.3, "prose": 70.5, "structured": 164.3 },
+      "prefill": [ { "context": 32768, "tps": 4934 }, { "context": 131072, "tps": 4776 } ],
+      "concurrency": [ { "streams": 4, "aggregate_tps": 144 }, { "streams": 8, "aggregate_tps": 194 } ],
+      "max_context": 524288,
+      "kv_pool_tokens": 4400000,
+      "max_concurrent": 64,
+      "quality": [ { "name": "Needle recall", "value": "12/12 up to 507k" } ]
+    },
+    {
+      "id": "tp6",
+      "label": "TP=6",
+      "nodes": 6,
+      "prefill": [ { "context": 32768, "tps": 4907 }, { "context": 131072, "tps": 4731 } ],
+      "max_context": 524288,
+      "kv_pool_tokens": 4470000,
+      "max_concurrent": 64,
+      "quality": [ { "name": "Needle recall", "value": "12/12 up to 480k" } ],
+      "notes": "Decode not measured at TP=6. Ran on other boxes and builds, with GPU clocks locked at 1989 MHz."
+    }
+  ]
+}