|
|
@@ -0,0 +1,164 @@
|
|
|
+# Upstream watch — 2026-10-26
|
|
|
+
|
|
|
+Pass across the sources named in the todo list and the 2026-10-23/24 night
|
|
|
+reports. Only 2026-10-26 changes are treated as "today"; older changes are
|
|
|
+flagged as context.
|
|
|
+
|
|
|
+## 1. FLARE — HEAD moved, 12 commits today (the headline)
|
|
|
+
|
|
|
+`NVIDIA/flare-local-inference` main, commits on 2026-10-26:
|
|
|
+
|
|
|
+| sha | message |
|
|
|
+|---|---|
|
|
|
+| `de6f53e` | Count single-token reasoning delimiters only when generated as that token (#94) |
|
|
|
+| `aa84dd3` | Carry the checkpoint license across package promotion (#83) |
|
|
|
+| `3a6b46f` | Return undeclared MiniMax tool calls under automatic tool choice (#49) |
|
|
|
+| `9e35eeb` | Prepare an opt-in MXFP8 DeepSeek V4.1 draft vocabulary head (#98) |
|
|
|
+| `d592f1f` | Overlap DeepSeek V4.1 query expansion with cache preparation (#97) |
|
|
|
+| `63a4008` | Fuse DSpark draft scoring and sparse support restoration (#95) |
|
|
|
+| `230c5b4` | Move the host prefix-cache tiers into a CUDA-free selene crate (#80) |
|
|
|
+| `2f2875a` | Align preparation guidance and examples with CLI defaults |
|
|
|
+| `82cb15a` | Add optional GPU clock locking during Ignition autotuning |
|
|
|
+| `c370a85` | Default disk embedding scales to host RAM on discrete GPUs |
|
|
|
+| `9e19b3b` | Pin FlashInfer to the supported tuning-tier API (#85) |
|
|
|
+| `328ddce` | test: check sectioned unverified payload inspection output (#89) |
|
|
|
+
|
|
|
+Why it matters to our prod lane (`dsv41-flash-1017-4x-flare-gb10`):
|
|
|
+
|
|
|
+- `63a4008` + `9e35eeb` + `d592f1f` are exactly the DSpark/draft/prefill path
|
|
|
+ our recipe uses (`--mtp-draft-tokens 5 --mtp-adaptive-verification`).
|
|
|
+- `c370a85` overlaps our manual `--embedding-table-resident-scales` choice: the
|
|
|
+ new head makes disk embedding scales resident by default on discrete GPUs.
|
|
|
+- `82cb15a` overlaps our `before_serve` clock lock: Ignition autotune can lock
|
|
|
+ the GPU clock instead of relying on the hook.
|
|
|
+- `9e19b3b` pins FlashInfer to the tuning-tier API — a compatibility pin, so a
|
|
|
+ binary/package mismatch is a real risk.
|
|
|
+
|
|
|
+Our pins today: `specs/tech-architecture/flare-upstream.pin` = `3b9e86a0`
|
|
|
+(2026-10-22 15:48); the prod binary was built at `26646d85` (2026-10-23 19:18).
|
|
|
+HEAD is `de6f53e` — a bump needs a binary rebuild **and** a package prepared at
|
|
|
+the matching revision; the "proxy ABI" rule in `flare-bringup.md` is why a
|
|
|
+binary-only bump must not be applied to prod.
|
|
|
+
|
|
|
+## 2. tool-eval-bench — v3.0.0 released today
|
|
|
+
|
|
|
+`SeraphimSerapis/tool-eval-bench` released **v3.0.0** on 2026-10-26 09:22
|
|
|
+(commit `d85126e`). Highlights:
|
|
|
+
|
|
|
+- optional local web UI (`tool-eval-bench serve`, `[web]` extra);
|
|
|
+- `tool-eval-bench card` — one SVG per session, optional PNG;
|
|
|
+- `--perf-preset long` up to 128K tokens, `--context-fill-tokens`;
|
|
|
+- TC-21 and TC-89 grading changed; TC-45 structured-output retries before
|
|
|
+ exclusion; suite remains 69 standard / 97 Hard Mode.
|
|
|
+
|
|
|
+The release explicitly says TC-21/TC-89 and probe corrections are **not
|
|
|
+comparable** with earlier versions. Our pin is v2.8.0; reports such as
|
|
|
+`91/100` and `88/100` are v2.8.0. Do not upgrade casually — re-baseline the
|
|
|
+anchor lane first.
|
|
|
+
|
|
|
+Forum topic 366903 ("Introducing Tool Eval Bench CLI") has today's
|
|
|
+announcement (serapis, 09:12) and a user run with a `card --png` overwrite bug
|
|
|
+(Digital_David, 13:30/14:23), plus a request to delete old runs from the UI.
|
|
|
+
|
|
|
+## 3. vLLM — no new release today
|
|
|
+
|
|
|
+Latest stable tag is still **v0.32.0** (2026-10-17); **v0.32.1rc0** is
|
|
|
+2026-10-18. No new stable tag on 2026-10-26. Our `vLLM 0.32` lanes stay on the
|
|
|
+existing pin; the v0.32.0 stable re-pin remains a supervised task.
|
|
|
+
|
|
|
+## 4. GR8 — 7 commits today
|
|
|
+
|
|
|
+`kindlingai/gr8` main, 2026-10-26:
|
|
|
+
|
|
|
+- `33b1aa7` — require the model by name (`glm53`, `glm53f`) and refuse anything
|
|
|
+ else (#234/#245);
|
|
|
+- `29abc14` — k8s extraVolumes/extraVolumeMounts for a local release as MODEL (#246);
|
|
|
+- `dc8284d` — LeaderWorkerSet Helm chart that runs `./gr8 up` on every box (#235/#243);
|
|
|
+- `64816bb` — `./gr8 kv health` readiness check for KV node containers (#241/#242);
|
|
|
+- `eb14798` — ring shape with mentat: discover cable order instead of `--peers` (#185/#190);
|
|
|
+- `db8576d` — `./gr8 <model> health` one readiness check for compose/K8s probes (#233/#240);
|
|
|
+- `3264cb1` — container entrypoint: one `bin/` for every TP and its own ulimits (#238).
|
|
|
+
|
|
|
+This touches our GLM-5.3 / GR8 lanes: model-name enforcement, readiness, ring
|
|
|
+discovery and container packaging. A bump should be bench-gated.
|
|
|
+
|
|
|
+## 5. DGPP — 4 commits today (+ DFlash2 batch yesterday)
|
|
|
+
|
|
|
+`HawkBearPig/dgpp` master, 2026-10-26:
|
|
|
+
|
|
|
+- `1babfa6` — merge PR #144 template structural limits;
|
|
|
+- `bb5fdfe` — complete template limit boundary, concurrency and HTTP coverage;
|
|
|
+- `882c2ee` — align reasoning-control test with the always-on model contract;
|
|
|
+- `d9aae09` — merge origin/master into the template-limits branch.
|
|
|
+
|
|
|
+2026-10-25 batch (relevant to Qwen3.8 DFlash2):
|
|
|
+
|
|
|
+- `7f8f6de` — merge DFlash2 Flash-Next PR #145;
|
|
|
+- `314401a` — document DFlash2 performance choices and test acceptance lengths;
|
|
|
+- `afce06f` — honor draft-head screening and recurrent replay for Flash-Next DFlash2;
|
|
|
+- `9513ab3` — shared DFlash2 graph, cache and sampling integration;
|
|
|
+- `3dbeddc` — honor explicit DFlash disable regardless of flag order;
|
|
|
+- `90d602a` — paired speculative probe accounting and request attribution;
|
|
|
+- `b671677` — drop the 27B YaRN x4 template from the DFlash2 branch;
|
|
|
+- `d0fafab` — make the paired spec probe runnable;
|
|
|
+- `e8ed6da` — serve: make the block-draft graph gate earnable;
|
|
|
+- `fc7b2d3` — ship the DFlash2 drafter as a knob.
|
|
|
+
|
|
|
+The 2026-10-24 DGPP commit `606a514` (DeepSeek-V4.1-Flash-1017 EXL3, 16 slots,
|
|
|
+96-row decode batch, five-draft block) is the DSV41-EXL3 path.
|
|
|
+
|
|
|
+## 6. Blackwell LLM Docker — no commits today
|
|
|
+
|
|
|
+Latest commit 2026-10-20: `8bb0967` / `5f803f7` — "ds41-flash: serve top_p 1.0,
|
|
|
+the DeepSeek-V4.1-Flash-1017 recommendation". Not today, but a substantive
|
|
|
+upstream change we should carry into the DSV41 recipe/bench context.
|
|
|
+
|
|
|
+No new commits today in `local-inference-lab/blackwell-llm-docker`; its latest
|
|
|
+is Oct 20.
|
|
|
+
|
|
|
+## 7. SparkRing — no commits today
|
|
|
+
|
|
|
+Latest commits 2026-10-19: dashboard endpoints, `--checkpoint 1017` of
|
|
|
+`deepseek-v41-flash-sparkcache-tp4`, Install Builder profiles. The todo #14 tag
|
|
|
+(`dev-20261016-sparkcache`) still needs checking against these.
|
|
|
+
|
|
|
+## 8. TensorFold — no commits today
|
|
|
+
|
|
|
+`x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark` latest commits 2026-10-18
|
|
|
+(double production KV pool, dashboard throughput); no 2026-10-26 change.
|
|
|
+`local-inference-lab/b12x` was archived on 2026-10-06; its development moved to
|
|
|
+FlashInfer.
|
|
|
+
|
|
|
+## 9. NVIDIA forum — today's posts
|
|
|
+
|
|
|
+- **384438 "New Deepseek 4.1 flash recipe"** — `huxiaofengtiger` (14:17 UTC)
|
|
|
+ upgraded a 3-node TP3 setup to `1025` and ran the single-stream suite against
|
|
|
+ R6x TP3: **60.94 vs 59.73 tok/s (+2.0%)**, TPOT p50 16.20 ms, TTFT higher;
|
|
|
+ long-prefill workloads next. `rob01` (02:19) reports the crash fix and TP3
|
|
|
+ low-RAM context.
|
|
|
+- **366903 "Introducing Tool Eval Bench CLI"** — v3.0.0 announcement and the
|
|
|
+ card-overwrite bug noted above.
|
|
|
+- **386811 "Welcome Athena Logos & Engine"** — created 2026-10-25; today's
|
|
|
+ comments are user feedback, not a new lane.
|
|
|
+
|
|
|
+## 10. HF model cards — no new 2026-10-26 revision
|
|
|
+
|
|
|
+- `RedHatAI/GLM-5.3-MXFP4xFP8` last modified 2026-10-22 14:05 (`a5cf1930…`).
|
|
|
+- `local-inference-lab/DeepSeek-V4.1-Flash-1017-FLARE` last modified
|
|
|
+ 2026-10-23 00:31 (`6861e802…`).
|
|
|
+
|
|
|
+No new checkpoint today; the package/revision pins stay as they are.
|
|
|
+
|
|
|
+## What this means for the todo list
|
|
|
+
|
|
|
+1. **FLARE tuning is now upstream-ready**: bump `de6f53e` is tempting because
|
|
|
+ DSpark scoring/query expansion/MXFP8 draft head/embedding scales/clock lock
|
|
|
+ all touch our lane, but prod requires a matching package rebuild. Treat as a
|
|
|
+ supervised experiment on dgx5–8 first, not a prod patch.
|
|
|
+2. **tool-eval-bench v3.0.0** is the one launcher-side upstream change today;
|
|
|
+ keep v2.8.0 until the anchor lane is re-baselined.
|
|
|
+3. **GR8 and DGPP** moved today; both are bench-gated bumps, not overnight
|
|
|
+ re-pins.
|
|
|
+4. **vLLM** has no new release; the 0.32.0/0.32.1rc0 picture is unchanged.
|
|
|
+5. **No new checkpoint/package today** — no download or staging action is
|
|
|
+ justified by upstream alone.
|
