 Alexander Korolev
|
b1f054231b
beam lane: 384K context (13.6x concurrency) + needle-in-haystack at 131K and 327K tokens (both correct, 60s/175s TTFT)
|
2 hari lalu |
|
Alexander Korolev
|
502080b104
beam lane: depth sweep (31-35 tok/s to 128K) + the second/third quality runs (92/100, Hard Mode 95%)
|
2 hari lalu |
|
Alexander Korolev
|
3248479cdf
beam lane: real-workload TTFT measured (0.41/0.51 s with prefix cache)
|
2 hari lalu |
|
Alexander Korolev
|
a3c43d9bb1
beam lane: spec-k7 quality/perf reports + real-work operational contract
|
2 hari lalu |
|
Alexander Korolev
|
d135e829d3
beam lane: DFlash2 speculation works with FLASHINFER — +40% decode, responsiveness 34->56
|
2 hari lalu |
|
Alexander Korolev
|
fe606b76bb
dsv41-flash-1017-4x-sglang: land the 2026-10-18 A/B reports and the 24/24 concurrency bump
|
2 hari lalu |
|
Alexander Korolev
|
88b5f8ddc1
beam-1-0-nvfp4-4x: new lane — Reflection Beam-1.0-501B-A23B NVFP4 on 4 Sparks, 90/100 quality (fleet best)
|
2 hari lalu |
|
Alexander Korolev
|
ec17b00796
dsv41-flash-1017-2x-tensorfold: ship it as an sm121 recipe
|
4 hari lalu |
|
Alexander Korolev
|
dbe6e49aca
dsv41-flash-1017-2x-tensorfold: measure the upstream TensorFold dual-Spark lane — 89/100 quality (best 1017 lane) but responsiveness 23 (6.7s turn) vs the 4x SGLang lane's 87/75; decode +20-30%
|
4 hari lalu |
|
Alexander Korolev
|
fe0d38641b
dsv41-flash-1017-4x-sglang: 1017 perf A/B — ep-size 2 (+9% decode), context 1M; document the engram MAX_IDS <-> chunked-prefill coupling
|
4 hari lalu |
|
Alexander Korolev
|
fc1850811d
dsv41-sglang-1017: version the image tag v1 (not :local) — trackable and rollback-able
|
5 hari lalu |
|
Alexander Korolev
|
aa4d33f1f2
dsv41-flash-1017-4x-sglang: bake the 1017 encoder patch into a thin wrapper image (docker-images/dsv41-sglang-1017, glm52 pattern: FROM aiden production-1.7.1 by digest + patch at build) — replaces the [mods]+manual-scp approach; recipe points at dsv41-sglang-1017:local
|
5 hari lalu |
|
Alexander Korolev
|
fc1fdcaa50
dsv41-flash-1017-4x-sglang: [mods] patch aligning the 1.7.1 image's DSV4.1 encoder with the 1017 reference protocol — REASONING_EFFORT_MAPPINGS 25/50/75 → 50/75/100 (xhigh dropped); idempotent anchor-based patcher mounted per node; upstream has no fix yet (Aiden still 1.7.1, eugr repo unchanged, sglang upstream silent)
|
5 hari lalu |
|
Alexander Korolev
|
ef684e8ae7
glm53-fp4local-2x: served-model-name glm53-fp4-local → glm53 (the fleet-wide alias convention)
|
5 hari lalu |
|
Alexander Korolev
|
1b35a14ef6
dflash A/B outcome: eugr build rejects all draft attention backends under fp8 KV (FLASHINFER/TRITON_ATTN/B12X/FLASH_ATTN) — DFlash2-for-GLM lives in the kindling overlay stack; the card stack is MTP-only
|
5 hari lalu |
|
Alexander Korolev
|
a3004e8e0b
dflash A/B: attention_backend B12X→FLASH_ATTN (B12X rejects the draft's block_size; trying the standard backend)
|
5 hari lalu |
|
Alexander Korolev
|
5939b55b07
dflash A/B: attention_backend TRITON_ATTN→B12X — on the eugr build only B12X accepts fp8 kv-cache (the main engine's backend); kindling's TRITON_ATTN works on our overlay image, not here
|
5 hari lalu |
|
Alexander Korolev
|
2e2a2e9495
dflash A/B: attention_backend FLASHINFER→TRITON_ATTN — FLASHINFER rejects fp8 kv-cache-dtype on this build; TRITON_ATTN is what the kindling lane uses for dflash
|
5 hari lalu |
|
Alexander Korolev
|
d5f9cb51a4
glm53-flash-fp4local-2x-dflash: DFlash2 A/B lane — incoai/GLM-5.3-Flash-DFlash2 drafter (2.2G, DFlash2DraftModel 5L/4096H, registered in the eugr build) vs the card's embedded MTP n5; rejection sampling is lossless, acceptance is the open question (DFlash2 distilled on the original zai distribution vs the QAD+nemotron-post-trained target)
|
5 hari lalu |
|
Alexander Korolev
|
3eb1af4d11
glm53-fp4local-2x card-config bench (launcher flow): 92/100 (178/194) — identical to the manual card-flow run; lane reproducible end-to-end
|
5 hari lalu |
|
Alexander Korolev
|
b378692803
glm53-fp4local-2x: restore the card's serving values as defaults (GMU 0.87, 500K ctx, 4 seqs) — the earlier GB10 reduction was based on the misdiagnosed stalls (missing distributed_backend); the card config is proven to boot with correct TP2; cold_args remain the first-launch valve
|
5 hari lalu |
|
Alexander Korolev
|
5dc7678753
glm53-fp4local-2x: cold_args GMU 0.72→0.75 — with correct TP2 the per-rank footprint is ~87 GiB (83 weights + overhead), 0.72 left 0.47 GiB KV vs 1.07 needed at 16384; 0.75 is the boot-safe floor (the cold tier is the ctx/seq/batched reduction)
|
5 hari lalu |
|
Alexander Korolev
|
c5a8444bb5
glm53-flash-fp4-local-2x: add distributed_backend="mp" + pipeline_parallel_size=1 + shm_size=16g — the missing distributed_backend made the launcher skip ALL dist flags, so each node booted an independent single-node engine loading the FULL 166 GB checkpoint (the root cause of the four stalls); launcher now refuses this config (4029b28)
|
5 hari lalu |
|
Alexander Korolev
|
58703660a1
glm53-fp4local-2x perf: d0 pp 1830/tg 38.9, d8192 c1 tg 34.9 (c2 42.2), TTFT 1.2s — official NVFP4 QAD decode ~35-40 t/s vs kindling 2x 52-59 t/s
|
5 hari lalu |
|
Alexander Korolev
|
8243e7dcd6
glm53-fp4local-2x bench: 92/100 (178/194) ★★★★★, v2.8.0, pluto → dgx3 — official NVIDIA NVFP4 QAD checkpoint served via the card's manual flow (eugr b12x stack, workers-before-head, node-local checkpoint); vs kindling 2x anchor 95/100; TC-51 batching, TC-89 (✗ confirm-release step), TC-92 (recurring DS4.1-family disclosure)
|
5 hari lalu |
|
Alexander Korolev
|
5e97656e8d
bolt-1x bench: 84/100 (163/194), v2.8.0, pluto → dgx3 (bench --node fix; first valid run after two wrong-head attempts) — card image + LiLiCorr dflash 7 spec-decode, GMU 0.90; fails TC-60 (sleeper injection BCC), TC-74 (premature batching), TC-85 (premature notification); vs card anchor 88/100 (-4, card eval conditions unknown)
|
5 hari lalu |
|
Alexander Korolev
|
64753dc8a6
glm53-flash-fp4-local-2x: cold_args GMU 0.60→0.72 — weights are 83 GiB/rank = 0.683 of the unified pool, 0.60 cannot fit them (negative KV); 0.72 leaves ~4 GiB KV and is colder than the 0.75 default attempt
|
6 hari lalu |
|
Alexander Korolev
|
a972f8b214
nemotron-35-bolt-74b-24gb: switch to the CARD's image path — docker-images/nemotron-bolt-b12x (thin FROM the card's stock vLLM nightly 77154ca by arm64 digest + b12x 1.5.1 + ultra_v3 parser plugin baked at /opt); recipe flags verbatim from the card incl. LiLiCorr dflash spec-decode (7) + ultra_v3 parser; only deviation GMU 0.92→0.90 (clean Spark reports 111.55/121.63 GiB free; 0.92 misses the boot gate by 0.35). eugr/spark-vllm-b12x (03a95654) does NOT carry LiLiCorrDraftModel/ultra_v3 — it cannot serve Bolt with the card's flags. Verified live: LiLiCorr speculator captured, smoke OK.
|
6 hari lalu |
|
Alexander Korolev
|
305cae0064
1017-sglang-4x bench: 89/100 (173/194), v2.8.0, pluto — first DeepSeek-V4.1-Flash-1017 on Aiden production-1.7.1 + staged engram-1017; median turn 1.7s (staged NVMe rows); fails TC-89/TC-92 (safety families); vs aiden16 anchor 91/100 (184, v2.7.1.dev7) — within noise across harness versions
|
6 hari lalu |
|
Alexander Korolev
|
e89dfeca78
dsv41-flash-1017-4x-sglang: DeepSeek-V4.1-Flash-1017 drop-in on Aiden SGLang production-1.7.1 (digest-pinned) — aiden16 contract (wrapper + sparse-copy Engram staging) with the 1017 snapshot as the staging src; new per-node dst engram-1017 (old aiden-engram copies are keyed to the old snapshot)
|
6 hari lalu |
