 Alexander Korolev
|
2f717631ff
report: stale cross-check of run ledger vs live probe (orange + runlock hint); new 'sm121 runlock list|close' offline ledger command
|
1 month ago |
|
Alexander Korolev
|
d5bbe7e54e
bench: dev66 glm53 eugr 2x re-run 161/176 1326pp/28.1tg/1458ms (same image as 09-11, quality lane verdict unchanged) + history (stale ledger fix)
|
1 month ago |
|
Alexander Korolev
|
c7d5cae0ee
bug report draft: r37 tree DS4.1 text garbling (bisect A-C, D pending)
|
1 month ago |
|
Alexander Korolev
|
65529d1c6c
history: HA shows dgx1-4 powered ON - nodes either hung or mgmt segment down; physical check needed
|
1 month ago |
|
Alexander Korolev
|
e0e73cf562
history: cluster-wide outage during Boot D - nodes dark mid-load, investigating recovery
|
1 month ago |
|
Alexander Korolev
|
9bbfae13d1
r37 typo bisect: A (no linear-b12x), B (prod spec K5), C (full prod contract) all still garbled; D (engram off) in progress - cause isolated to the r37 tree, not the flags
|
1 month ago |
|
Alexander Korolev
|
208a173b0c
history: morning R37 typo investigation (LM-head quant found + disproven as sole cause; linear-backend b12x bisect started)
|
1 month ago |
|
Alexander Korolev
|
902e989c68
r37 recipe: VLLM_MXFP8_LM_HEAD=0 - the lab tree's runtime LM-head MXFP8 quantization (default ON, absent from the eugr prod image) is the suspected cause of the typo regression
|
1 month ago |
|
Alexander Korolev
|
4fdabdb105
night shift final: full MXFP4 works on 4x at 131K (158/176, DCP4 blocked by engine bug), R37 = 2.3x prefill champion (3,830 pp) but 150/176 quality; teardown notes
|
1 month ago |
|
Alexander Korolev
|
5adc5325ff
P2 result: R37 DS4.1 4x - 150/176 (85, safety-capped) but 3,830 pp = 2.3x prod prefill, TTFT 832 ms (2.1x better); reports landed (dev65-r37-ds41-4x)
|
1 month ago |
|
Alexander Korolev
|
4cb61b3aef
P0 result: full GLM-5.3 MXFP4 on 4x GB10 - 158/176 (90), 934 pp d0; reports landed (dev65-full-mxfp4-131k)
|
1 month ago |
|
Alexander Korolev
|
bbeb6bd553
mxfp4 full recipe: switch to the r37 image - r35's DCP4 sparse decode path crashes on this model (reshape x4 DCP factor in attention.py during graph capture)
|
1 month ago |
|
Alexander Korolev
|
e2af78e409
night plan interim: full MXFP4 probe booted and passed (51.6 KB/token/rank KV, smoke green); DCP4 interleave fixed to 1
|
1 month ago |
|
Alexander Korolev
|
36bb9825d6
mxfp4 full recipe: cp-kv-cache-interleave-size 1 - the DCP sparse indexer (DSA top-k path) only supports interleave 1 on r35 (4 is the Flash dense-MLA rule, not this path)
|
1 month ago |
|
Alexander Korolev
|
675ccdb307
mxfp4 recipes: --linear-backend marlin - the b12x mxfp4 warmup (ungated) crashes on this checkpoint's shapes (rank 3 vs 4 in b12x dense_gemm); marlin A16 is the guaranteed path
|
1 month ago |
|
Alexander Korolev
|
9cf76da709
mxfp4 probe: KV pin 4->8 GiB - engine needs 6.58 GiB/rank for one 131072-token request (51.6 KB/token/rank measured)
|
1 month ago |
|
Alexander Korolev
|
57e6336047
mxfp4 recipes: --linear-backend b12x (native SM120 GEMM) - the flashinfer mxfp4 linear kernel wrongly claims sm_12x support and crashes in cute-dsl mm_fp4; --moe-backend is inert for the compressed-tensors MoE scheme (it picks MarlinExperts itself)
|
1 month ago |
|
Alexander Korolev
|
17672b1322
mxfp4 recipes: --moe-backend flashinfer_cutlass (the Sept-tree CLI family; _afp8 was not a valid choice on r35 - first boot argparse)
|
1 month ago |
|
Alexander Korolev
|
87e866512c
night shift: P0 probe recipe (no DCP, 131K, 4 GiB KV) - step 1 of the full-model boot ladder
|
1 month ago |
|
Alexander Korolev
|
501e627b2b
night plan: R37 image built+distributed (label 9350429b) and the MXFP4 checkpoint complete on the NAS - both lanes ready, switch needs the go-ahead
|
1 month ago |
|
Alexander Korolev
|
beda9d96ae
web: persist window layout (geometry/minimized/closed/maximized + open optional windows) in localStorage across reloads; recipes panel drops its own title and live chip when hosted (body.embed) and reopens from the menu when minimized; closed named windows keep a dock chip
|
1 month ago |
|
Alexander Korolev
|
b5e08df460
glm53 full MXFP4: explicit --moe-backend flashinfer_cutlass_afp8 + document the MXFP4 ingestion path
|
1 month ago |
|
Alexander Korolev
|
e3e9cbd3b1
web header: guard #maxinfo[hidden] (display:flex defeated the hidden attribute, so the maximize chrome sat in the toolbar permanently); assert header visibility, not the .hidden property, in playwright
|
1 month ago |
|
Alexander Korolev
|
8433106ff4
night-shift 2026-09-13 plan + R37 (jovian) build profile
|
1 month ago |
|
Alexander Korolev
|
5c54bf0af5
web windows: maximized window drops its titlebar too — header carries the title plus restore/minimize/close for that window
|
1 month ago |
|
Alexander Korolev
|
305f9066a6
web windows: maximized window is frameless and flush to the layer (title moves into the header with a restore button), retire the dead dock reserve, clamp px geometry to the layer when the dock appears or the page resizes
|
1 month ago |
|
Alexander Korolev
|
aa49a8b61e
web windows: dock lists every window (recipes/raw/logs no longer minimize into nowhere), add a maximize/restore button to every titlebar, size the Topology window to the map aspect and reflow the default layout (topo+jobs left, state full height right) so nothing is dead space
|
1 month ago |
|
Alexander Korolev
|
07055b199f
web: recipes panel window — the report --serve page mounted at /recipes (one implementation, prefix-rewritten paths/SSE/viewer) + iframe theme inheritance; fix stale raw-window responses overwriting a newer selection; docs + playwright checks
|
1 month ago |
|
Alexander Korolev
|
7535a7d2c4
glm53 full-model lane: retarget to RedHatAI/GLM-5.3-MXFP4 on the r35 jovian stack; delete the OOM-proven NVFP4-Spark variants
|
1 month ago |
|
Alexander Korolev
|
4ca149b977
web: replay cached snapshot on SSE connect (live indicator no longer waits a poll tick) + jobs window groups running containers one row per launcher job with its node list
|
1 month ago |
