 Alexander Korolev
|
e89dfeca78
dsv41-flash-1017-4x-sglang: DeepSeek-V4.1-Flash-1017 drop-in on Aiden SGLang production-1.7.1 (digest-pinned) — aiden16 contract (wrapper + sparse-copy Engram staging) with the 1017 snapshot as the staging src; new per-node dst engram-1017 (old aiden-engram copies are keyed to the old snapshot)
|
6 giorni fa |
|
Alexander Korolev
|
4837e6b6f3
ling-31-flash-fp4: first valid bench 82/100 (159/194, v2.8.0, TP4 dgx58-switch)
|
6 giorni fa |
|
Alexander Korolev
|
39179f27b5
glm53-flash-fp4-local-2x: add cold_args (extra-safe first boot: gmu 0.60, 16384 ctx, 1 seq, 2048 batched) — used by 'serve --cold' over the default args
|
6 giorni fa |
|
Alexander Korolev
|
4300f3b9e4
glm53-flash-fp4-local-2x: GB10-safe limits — card's 500K ctx + GMU 0.87 + 4 seqs stalled both Sparks (unified-memory exhaustion, forum 386057). Cut to GMU 0.75 + 32768 ctx + 2 seqs (bench-ample, host headroom)
|
6 giorni fa |
|
Alexander Korolev
|
6310b23019
ling-31-flash-fp4: fix KV OOM — weights are 86.43 GiB/rank (replicated embeddings/lm_head/dense/MLA on top of 291.7/4), GMU 0.78 left 0.29 GiB KV (65536 needs 0.77, engine init refused). Raise GMU 0.86, drop max-model-len 32768 + max-num-batched-tokens 4096 (bench scenarios short)
|
6 giorni fa |
|
Alexander Korolev
|
3e4fe4dfd8
Add ling-31-flash-fp4 recipe: our own fp8→MXFP4 requant (313.2 GB, 214025 tensors), TP4 on 4 Sparks, stock vLLM v0.31.0 (built-in Ling fp4 loader), HF_HUB_OFFLINE=1 (local-only repo), ling3 tool parser. Corrects the earlier ~110GB/one-Spark estimate — experts dominate, fp4 is 313GB → TP4
|
6 giorni fa |
|
Alexander Korolev
|
6baa5ec725
glm53-flash-fp4-local-2x: complete the DGX Spark x2 card params — add --dtype bfloat16, --quantization modelopt_mixed, --no-enable-flashinfer-autotune, --max-num-seqs 4, --max-num-batched-tokens 4096 (my first transcription grepped a truncated serve block and dropped them); add the card's vllm/flashinfer/triton/b12x cache mounts. Verified against the full '2 x NVIDIA DGX Spark' section (not the RTX PRO 6000 one)
|
6 giorni fa |
|
Alexander Korolev
|
d5276dbb55
Add nvidia/GLM-5.3-Flash-FP4-Local 2x recipe (official NVFP4 QAD, modelopt v0.47.0): TP2, b12x container (new lossless NVFP4-scale fmt not in upstream vLLM), 500K ctx, MTP n5, FP8 KV, glm45/glm47 parsers. A/B vs kindling 2x anchor.
|
6 giorni fa |
|
Alexander Korolev
|
903ed35881
Add 3 single-Spark recipes from NVIDIA forum (10-16): Qwen3.8-Flash-Next-FP4-Local (b12x+MTP n4, 89/100), Nemotron-3.5-Super-VL-120B-NVFP4 (LiLiCorr dflash7, 1M ctx, 90/100), Nemotron-3.5-Bolt-74B-A9B-24GB (QAD 2-bit, LiLiCorr, 88/100). All solo/TP1, eugr/spark-vllm-b12x digest-pinned, port 30000.
|
1 settimana fa |
|
Alexander Korolev
|
81eb20a763
ling recipe: pointer to the fp4 smaller-cut path (~110GB -> one Spark) + ling3 parser
|
1 settimana fa |
|
Alexander Korolev
|
531013f167
ling recipe: add --enable-auto-tool-choice --tool-call-parser ling3 (vLLM v0.31.0 Ling3ToolParser) — makes the serve bench-ready for tool scenarios
|
1 settimana fa |
|
Alexander Korolev
|
b4bcb389ec
live 8x probe results: Ling fp8 feasible on GB10 (serves+generates; bench needs tool-call config + thinking control); Step-5 ruled out (Step5 DSA needs SM90/SM103, GB10=SM121)
|
1 settimana fa |
|
Alexander Korolev
|
e7877b094b
ling-31-flash-nvfp4: live 8x serve on GB10 (stock v0.31.0) — feasibility PROVEN; first bench invalid (94/97 timeout: thinking-on default, 120s/turn) + served-model-name
|
1 settimana fa |
|
Alexander Korolev
|
2401b34ef7
ling/step5 recipes: point at the downloaded weights (Ling fp8 original 535G, Step-5 NVFP4 368G on NAS) + entrypoint=[vllm,serve] (build_tail needs it or the model lands in --entrypoint); Ling 8x TP8 stock v0.31.0
|
1 settimana fa |
|
Alexander Korolev
|
c87238bfd6
8x lane reports: kindling8x-conc-48g (2026-10-12, 95/100 anchor) run package — md/json/log/manifest/recipe.toml
|
1 settimana fa |
|
Alexander Korolev
|
78c48793f7
prep recipes: ling-31-flash-nvfp4 (4x, BailingMoeV3 on v0.31.0, weights pending) + step5-preview-nvfp4 (8x, dedicated step-5-preview image, VL, GB10-unverified)
|
1 settimana fa |
|
Alexander Korolev
|
201b853c79
dgpp recipes: pin [dgpp].repo/rev (3ad63d6b) for sm121 on-demand DGPP provisioning
|
1 settimana fa |
|
Alexander Korolev
|
28a07e0509
glm53-flash-nvfp4-fp8-dgpp-2x: DGPP-engine GLM-5.3-Flash w2 recipe (engine=dgpp, deploy=native, world_size=2, [dgpp.engine] from DGPP w2 example)
|
1 settimana fa |
|
Alexander Korolev
|
c733c0688f
qwen38-27b-nvfp4-dgpp: first-class DGPP-engine recipe (engine=dgpp, deploy=native, [dgpp.engine] knobs → generated cluster JSON, fixed API :18080)
|
1 settimana fa |
|
Alexander Korolev
|
541c5f6dc4
glm53-full-tp4-gr8: first-class gr8-engine recipe (engine=gr8, TP=4 switched, --tp/--peers generated, fixed API :8100)
|
1 settimana fa |
|
Alexander Korolev
|
040e9f26b6
mimo26-flash-mopd-4x-sparkring: rebase bench (2026.10.4 image, v2.8.0) — 90/100, tg 66.2/80.7 c1/c2 (+19% vs dev-20261004)
|
1 settimana fa |
|
Alexander Korolev
|
a09e85fb54
mimo26-flash-mopd-4x-sparkring: rebase to SparkRing 2026.10.4 image (dev-20261013, e21ba936)
|
1 settimana fa |
|
Alexander Korolev
|
6c784467fe
glm53-kindling-2x: second perf pass (post-campaign cooldown) — d0 c1 51.4 vs 39.6 hot: run-variance baseline
|
1 settimana fa |
|
Alexander Korolev
|
71b7f321b0
glm53-kindling-2x: perf-only re-baseline (v0.31.0 image, no label needed)
|
1 settimana fa |
|
Alexander Korolev
|
2e45fa0bca
glm53-kindling-2x: v0.31.0-base bench (v2.8.0, 97 scen) — 95/100 quality (+2 vs 93 control), perf + depth suite 16K..256K
|
1 settimana fa |
|
Alexander Korolev
|
8678fa1883
kindling: AGENTS checklist for changing the vLLM base (overlay rebase + version guards + post-bake smoke)
|
1 settimana fa |
|
Alexander Korolev
|
6e0b9528ac
draft-trunc guard: v0.31.0 is a release build (bare version string, no +g<sha> scm suffix) — pin the guard to the release version, keep the fail-loud semantics
|
1 settimana fa |
|
Alexander Korolev
|
5e04b7bfe9
glm53-kindling: rebase the 23 drifted experimental/ overlays onto the v0.31.0 base (10 files carry real drift; 21 auto-merged, 2 hand-merged: rejection_sampler accept_drafts after the gather_draft_sampled() refactor, sm90 _TRITON early-return after the 3-tuple _kv_lens_host). Unrebased overlays reintroduce ddd6fbca-era import blocks on the v0.31.0 tree -> ImportError at boot
|
1 settimana fa |
|
Alexander Korolev
|
546e7b2025
glm53-kindling: re-pin BASE to the v0.31.0 release — Docker Hub pruned the entire old nightly-<sha> scheme, the 09-26 base tag is gone and the dir was unbuildable. v0.31.0 (cut 10-02) is the first release containing the base commit; drift audit ddd6fbca..v0.31.0: all patched files unchanged, vllm#58720 landed verbatim upstream (patch dropped, verify-base keeps asserting it), flashinfer 0.7.0->0.7.0.post1, torch/NCCL/cutlass-dsl unchanged
|
1 settimana fa |
|
Alexander Korolev
|
b655965c5f
2x control bench on v2.8.0 (97 scen, hardmode, engine-default effort): 93/100 — the same-harness reference for gr8 (89) and DGPP (88)
|
1 settimana fa |
