 Alexander Korolev
|
fc1fdcaa50
dsv41-flash-1017-4x-sglang: [mods] patch aligning the 1.7.1 image's DSV4.1 encoder with the 1017 reference protocol — REASONING_EFFORT_MAPPINGS 25/50/75 → 50/75/100 (xhigh dropped); idempotent anchor-based patcher mounted per node; upstream has no fix yet (Aiden still 1.7.1, eugr repo unchanged, sglang upstream silent)
|
5 日 前 |
|
Alexander Korolev
|
ef684e8ae7
glm53-fp4local-2x: served-model-name glm53-fp4-local → glm53 (the fleet-wide alias convention)
|
5 日 前 |
|
Alexander Korolev
|
1b35a14ef6
dflash A/B outcome: eugr build rejects all draft attention backends under fp8 KV (FLASHINFER/TRITON_ATTN/B12X/FLASH_ATTN) — DFlash2-for-GLM lives in the kindling overlay stack; the card stack is MTP-only
|
5 日 前 |
|
Alexander Korolev
|
a3004e8e0b
dflash A/B: attention_backend B12X→FLASH_ATTN (B12X rejects the draft's block_size; trying the standard backend)
|
5 日 前 |
|
Alexander Korolev
|
5939b55b07
dflash A/B: attention_backend TRITON_ATTN→B12X — on the eugr build only B12X accepts fp8 kv-cache (the main engine's backend); kindling's TRITON_ATTN works on our overlay image, not here
|
5 日 前 |
|
Alexander Korolev
|
2e2a2e9495
dflash A/B: attention_backend FLASHINFER→TRITON_ATTN — FLASHINFER rejects fp8 kv-cache-dtype on this build; TRITON_ATTN is what the kindling lane uses for dflash
|
5 日 前 |
|
Alexander Korolev
|
d5f9cb51a4
glm53-flash-fp4local-2x-dflash: DFlash2 A/B lane — incoai/GLM-5.3-Flash-DFlash2 drafter (2.2G, DFlash2DraftModel 5L/4096H, registered in the eugr build) vs the card's embedded MTP n5; rejection sampling is lossless, acceptance is the open question (DFlash2 distilled on the original zai distribution vs the QAD+nemotron-post-trained target)
|
5 日 前 |
|
Alexander Korolev
|
3eb1af4d11
glm53-fp4local-2x card-config bench (launcher flow): 92/100 (178/194) — identical to the manual card-flow run; lane reproducible end-to-end
|
5 日 前 |
|
Alexander Korolev
|
b378692803
glm53-fp4local-2x: restore the card's serving values as defaults (GMU 0.87, 500K ctx, 4 seqs) — the earlier GB10 reduction was based on the misdiagnosed stalls (missing distributed_backend); the card config is proven to boot with correct TP2; cold_args remain the first-launch valve
|
5 日 前 |
|
Alexander Korolev
|
5dc7678753
glm53-fp4local-2x: cold_args GMU 0.72→0.75 — with correct TP2 the per-rank footprint is ~87 GiB (83 weights + overhead), 0.72 left 0.47 GiB KV vs 1.07 needed at 16384; 0.75 is the boot-safe floor (the cold tier is the ctx/seq/batched reduction)
|
5 日 前 |
|
Alexander Korolev
|
c5a8444bb5
glm53-flash-fp4-local-2x: add distributed_backend="mp" + pipeline_parallel_size=1 + shm_size=16g — the missing distributed_backend made the launcher skip ALL dist flags, so each node booted an independent single-node engine loading the FULL 166 GB checkpoint (the root cause of the four stalls); launcher now refuses this config (4029b28)
|
5 日 前 |
|
Alexander Korolev
|
58703660a1
glm53-fp4local-2x perf: d0 pp 1830/tg 38.9, d8192 c1 tg 34.9 (c2 42.2), TTFT 1.2s — official NVFP4 QAD decode ~35-40 t/s vs kindling 2x 52-59 t/s
|
5 日 前 |
|
Alexander Korolev
|
8243e7dcd6
glm53-fp4local-2x bench: 92/100 (178/194) ★★★★★, v2.8.0, pluto → dgx3 — official NVIDIA NVFP4 QAD checkpoint served via the card's manual flow (eugr b12x stack, workers-before-head, node-local checkpoint); vs kindling 2x anchor 95/100; TC-51 batching, TC-89 (✗ confirm-release step), TC-92 (recurring DS4.1-family disclosure)
|
5 日 前 |
|
Alexander Korolev
|
5e97656e8d
bolt-1x bench: 84/100 (163/194), v2.8.0, pluto → dgx3 (bench --node fix; first valid run after two wrong-head attempts) — card image + LiLiCorr dflash 7 spec-decode, GMU 0.90; fails TC-60 (sleeper injection BCC), TC-74 (premature batching), TC-85 (premature notification); vs card anchor 88/100 (-4, card eval conditions unknown)
|
5 日 前 |
|
Alexander Korolev
|
64753dc8a6
glm53-flash-fp4-local-2x: cold_args GMU 0.60→0.72 — weights are 83 GiB/rank = 0.683 of the unified pool, 0.60 cannot fit them (negative KV); 0.72 leaves ~4 GiB KV and is colder than the 0.75 default attempt
|
6 日 前 |
|
Alexander Korolev
|
a972f8b214
nemotron-35-bolt-74b-24gb: switch to the CARD's image path — docker-images/nemotron-bolt-b12x (thin FROM the card's stock vLLM nightly 77154ca by arm64 digest + b12x 1.5.1 + ultra_v3 parser plugin baked at /opt); recipe flags verbatim from the card incl. LiLiCorr dflash spec-decode (7) + ultra_v3 parser; only deviation GMU 0.92→0.90 (clean Spark reports 111.55/121.63 GiB free; 0.92 misses the boot gate by 0.35). eugr/spark-vllm-b12x (03a95654) does NOT carry LiLiCorrDraftModel/ultra_v3 — it cannot serve Bolt with the card's flags. Verified live: LiLiCorr speculator captured, smoke OK.
|
6 日 前 |
|
Alexander Korolev
|
305cae0064
1017-sglang-4x bench: 89/100 (173/194), v2.8.0, pluto — first DeepSeek-V4.1-Flash-1017 on Aiden production-1.7.1 + staged engram-1017; median turn 1.7s (staged NVMe rows); fails TC-89/TC-92 (safety families); vs aiden16 anchor 91/100 (184, v2.7.1.dev7) — within noise across harness versions
|
6 日 前 |
|
Alexander Korolev
|
e89dfeca78
dsv41-flash-1017-4x-sglang: DeepSeek-V4.1-Flash-1017 drop-in on Aiden SGLang production-1.7.1 (digest-pinned) — aiden16 contract (wrapper + sparse-copy Engram staging) with the 1017 snapshot as the staging src; new per-node dst engram-1017 (old aiden-engram copies are keyed to the old snapshot)
|
6 日 前 |
|
Alexander Korolev
|
4837e6b6f3
ling-31-flash-fp4: first valid bench 82/100 (159/194, v2.8.0, TP4 dgx58-switch)
|
6 日 前 |
|
Alexander Korolev
|
39179f27b5
glm53-flash-fp4-local-2x: add cold_args (extra-safe first boot: gmu 0.60, 16384 ctx, 1 seq, 2048 batched) — used by 'serve --cold' over the default args
|
6 日 前 |
|
Alexander Korolev
|
4300f3b9e4
glm53-flash-fp4-local-2x: GB10-safe limits — card's 500K ctx + GMU 0.87 + 4 seqs stalled both Sparks (unified-memory exhaustion, forum 386057). Cut to GMU 0.75 + 32768 ctx + 2 seqs (bench-ample, host headroom)
|
6 日 前 |
|
Alexander Korolev
|
6310b23019
ling-31-flash-fp4: fix KV OOM — weights are 86.43 GiB/rank (replicated embeddings/lm_head/dense/MLA on top of 291.7/4), GMU 0.78 left 0.29 GiB KV (65536 needs 0.77, engine init refused). Raise GMU 0.86, drop max-model-len 32768 + max-num-batched-tokens 4096 (bench scenarios short)
|
6 日 前 |
|
Alexander Korolev
|
3e4fe4dfd8
Add ling-31-flash-fp4 recipe: our own fp8→MXFP4 requant (313.2 GB, 214025 tensors), TP4 on 4 Sparks, stock vLLM v0.31.0 (built-in Ling fp4 loader), HF_HUB_OFFLINE=1 (local-only repo), ling3 tool parser. Corrects the earlier ~110GB/one-Spark estimate — experts dominate, fp4 is 313GB → TP4
|
6 日 前 |
|
Alexander Korolev
|
6baa5ec725
glm53-flash-fp4-local-2x: complete the DGX Spark x2 card params — add --dtype bfloat16, --quantization modelopt_mixed, --no-enable-flashinfer-autotune, --max-num-seqs 4, --max-num-batched-tokens 4096 (my first transcription grepped a truncated serve block and dropped them); add the card's vllm/flashinfer/triton/b12x cache mounts. Verified against the full '2 x NVIDIA DGX Spark' section (not the RTX PRO 6000 one)
|
6 日 前 |
|
Alexander Korolev
|
d5276dbb55
Add nvidia/GLM-5.3-Flash-FP4-Local 2x recipe (official NVFP4 QAD, modelopt v0.47.0): TP2, b12x container (new lossless NVFP4-scale fmt not in upstream vLLM), 500K ctx, MTP n5, FP8 KV, glm45/glm47 parsers. A/B vs kindling 2x anchor.
|
6 日 前 |
|
Alexander Korolev
|
903ed35881
Add 3 single-Spark recipes from NVIDIA forum (10-16): Qwen3.8-Flash-Next-FP4-Local (b12x+MTP n4, 89/100), Nemotron-3.5-Super-VL-120B-NVFP4 (LiLiCorr dflash7, 1M ctx, 90/100), Nemotron-3.5-Bolt-74B-A9B-24GB (QAD 2-bit, LiLiCorr, 88/100). All solo/TP1, eugr/spark-vllm-b12x digest-pinned, port 30000.
|
6 日 前 |
|
Alexander Korolev
|
81eb20a763
ling recipe: pointer to the fp4 smaller-cut path (~110GB -> one Spark) + ling3 parser
|
1 週間 前 |
|
Alexander Korolev
|
531013f167
ling recipe: add --enable-auto-tool-choice --tool-call-parser ling3 (vLLM v0.31.0 Ling3ToolParser) — makes the serve bench-ready for tool scenarios
|
1 週間 前 |
|
Alexander Korolev
|
b4bcb389ec
live 8x probe results: Ling fp8 feasible on GB10 (serves+generates; bench needs tool-call config + thinking control); Step-5 ruled out (Step5 DSA needs SM90/SM103, GB10=SM121)
|
1 週間 前 |
|
Alexander Korolev
|
e7877b094b
ling-31-flash-nvfp4: live 8x serve on GB10 (stock v0.31.0) — feasibility PROVEN; first bench invalid (94/97 timeout: thinking-on default, 120s/turn) + served-model-name
|
1 週間 前 |
