 Alexander Korolev
|
f1209f78f2
glm-5.3-flash-spark lanes on the karmic-kraken-beta image (r12 gone, non-rebuildable); 4x variant; gmu 0.85
|
1 месяц назад |
|
Alexander Korolev
|
8ba943b935
full GLM-5.3 native lane on 8x: aiden prod-1.2 image, safetensors loader, dspark k8 @64k
|
1 месяц назад |
|
Alexander Korolev
|
c144bbb479
P2.1: 8x full GLM up @512K DCP1 (KV 972K tokens, 1.85x; no DCP bug), quality gate clean; bench running; NFS rides enP7s7 mgmt NIC (operator's Mikrotik question)
|
1 месяц назад |
|
Alexander Korolev
|
e625c2abd0
P2.3 result: 8x DS4.1 in-memory engram bench 161/176, 2,866 pp / 50.2 tg / 986 ms (prefill +75%, TTFT -43% vs 4x prod)
|
1 месяц назад |
|
Alexander Korolev
|
a5f50a4185
P2.3: 8x DS4.1 in-memory engram up (GMU 0.85 fix, dgx6-8 cache symlink, 122-min NFS load, KV 2.58M tokens @4.93x, quality gate clean); bench 8x running
|
1 месяц назад |
|
Alexander Korolev
|
5937043c71
plan: SGLang DS4.1 4x lane queued (aidenle recipe, production since 09-17 on their fleet); 8x DS4.1 in-memory engram boot in progress
|
1 месяц назад |
|
Alexander Korolev
|
e6e4b43c66
P1 verdict: lab-tree garbling trait (0913 A/B garbled like r37); bug report finalized; dsv41-flash-8x recipe (in-memory engram, GMU 0.91)
|
1 месяц назад |
|
Alexander Korolev
|
8b8f906a82
P1: eugr 0913 A/B recipe (measured r37 contract on lab's infernal-invocation image); boot D infeasible on 4x (engram-off OOM incident, power-cycled, recovered)
|
1 месяц назад |
|
Alexander Korolev
|
4b8e9815bc
P0 result: full GLM-5.3 + DSpark K8 bench 153/176, d0c1 892 pp / 33.2 tg (1.75x decode) / 2,193 ms vs no-spec anchor
|
1 месяц назад |
|
Alexander Korolev
|
960dc62483
glm53-full-mxfp4-4x-dspark8: full GLM-5.3 + DSpark K8 drafter recipe (131K DCP1, 9 GiB KV pin, skip-layers for draft KV, in-container patch for shared-page inheritance); boot 6 up, quality gate clean
|
1 месяц назад |
|
Alexander Korolev
|
bc5aa791bf
8nodes-switch cluster in config (dual-rail 200G, ranks 0-7), dgx7/8 nas mount fixed, night plan 09-18: 8x ready, DS4.1-8x in-memory engram
|
1 месяц назад |
|
Alexander Korolev
|
73dd1db126
night plan 2026-09-17: full GLM-5.3 + DSpark K8 (speculator.dspark, downloaded), r37 typo verdict (boot D + eugr 0913), 8x trials on the new cables, lab lil-catalog policies
|
1 месяц назад |
|
Alexander Korolev
|
523924b032
remove stray file 1 (shell redirect accident)
|
1 месяц назад |
|
Alexander Korolev
|
f011388d0a
web: SSE streams can no longer die silently or stay dead — 15s keepalive comments on both endpoints, live-chip watchdog rebuilds a stale stream / a gave-up EventSource, bounce on visibility/online; supervisor isolates ticks and survives poisoned locks
|
1 месяц назад |
|
Alexander Korolev
|
706b7a5e03
topo: an HA-OFF switch can never have green ports - fabric port dots and 200G cable flow now require the switch to report on (fabPortOn policy + 6 playwright probes)
|
1 месяц назад |
|
Alexander Korolev
|
2f717631ff
report: stale cross-check of run ledger vs live probe (orange + runlock hint); new 'sm121 runlock list|close' offline ledger command
|
1 месяц назад |
|
Alexander Korolev
|
d5bbe7e54e
bench: dev66 glm53 eugr 2x re-run 161/176 1326pp/28.1tg/1458ms (same image as 09-11, quality lane verdict unchanged) + history (stale ledger fix)
|
1 месяц назад |
|
Alexander Korolev
|
c7d5cae0ee
bug report draft: r37 tree DS4.1 text garbling (bisect A-C, D pending)
|
1 месяц назад |
|
Alexander Korolev
|
65529d1c6c
history: HA shows dgx1-4 powered ON - nodes either hung or mgmt segment down; physical check needed
|
1 месяц назад |
|
Alexander Korolev
|
e0e73cf562
history: cluster-wide outage during Boot D - nodes dark mid-load, investigating recovery
|
1 месяц назад |
|
Alexander Korolev
|
9bbfae13d1
r37 typo bisect: A (no linear-b12x), B (prod spec K5), C (full prod contract) all still garbled; D (engram off) in progress - cause isolated to the r37 tree, not the flags
|
1 месяц назад |
|
Alexander Korolev
|
208a173b0c
history: morning R37 typo investigation (LM-head quant found + disproven as sole cause; linear-backend b12x bisect started)
|
1 месяц назад |
|
Alexander Korolev
|
902e989c68
r37 recipe: VLLM_MXFP8_LM_HEAD=0 - the lab tree's runtime LM-head MXFP8 quantization (default ON, absent from the eugr prod image) is the suspected cause of the typo regression
|
1 месяц назад |
|
Alexander Korolev
|
4fdabdb105
night shift final: full MXFP4 works on 4x at 131K (158/176, DCP4 blocked by engine bug), R37 = 2.3x prefill champion (3,830 pp) but 150/176 quality; teardown notes
|
1 месяц назад |
|
Alexander Korolev
|
5adc5325ff
P2 result: R37 DS4.1 4x - 150/176 (85, safety-capped) but 3,830 pp = 2.3x prod prefill, TTFT 832 ms (2.1x better); reports landed (dev65-r37-ds41-4x)
|
1 месяц назад |
|
Alexander Korolev
|
4cb61b3aef
P0 result: full GLM-5.3 MXFP4 on 4x GB10 - 158/176 (90), 934 pp d0; reports landed (dev65-full-mxfp4-131k)
|
1 месяц назад |
|
Alexander Korolev
|
bbeb6bd553
mxfp4 full recipe: switch to the r37 image - r35's DCP4 sparse decode path crashes on this model (reshape x4 DCP factor in attention.py during graph capture)
|
1 месяц назад |
|
Alexander Korolev
|
e2af78e409
night plan interim: full MXFP4 probe booted and passed (51.6 KB/token/rank KV, smoke green); DCP4 interleave fixed to 1
|
1 месяц назад |
|
Alexander Korolev
|
36bb9825d6
mxfp4 full recipe: cp-kv-cache-interleave-size 1 - the DCP sparse indexer (DSA top-k path) only supports interleave 1 on r35 (4 is the Flash dense-MLA rule, not this path)
|
1 месяц назад |
|
Alexander Korolev
|
675ccdb307
mxfp4 recipes: --linear-backend marlin - the b12x mxfp4 warmup (ungated) crashes on this checkpoint's shapes (rank 3 vs 4 in b12x dense_gemm); marlin A16 is the guaranteed path
|
1 месяц назад |
