Просмотр исходного кода

dsv41-flare: bench 90/100 on the patched runtime; prod-no4 confirmed production

2026-11-04 bench (v3.0.0, seed 42, hardmode, pluto): 90/100 (172/192) on
the prod4b lane with the patched runtime - 3 new TC fails vs the 92/100
baseline (TC-43/71/76, known TC-flake family), 2 fixed (TC-89/92), TC-45
(the known FLARE tool_choice 500) still excluded. Perf in the same class:
tg 64.1/99.6 c1/c2, pp2048 1727/1591, d8192 1950/67.5.

The lane then died at seq 2833389 under sustained load even with the
patch - prod-no4 (dgx8 instead of dgx4) is production; the dgx4 lane
returns only after the NVMe swap. Uniform-binary requirement recorded.
Alexander Korolev 2 дней назад
Родитель
Сommit
530b138543

+ 6 - 2
recipes/dsv41-flash-1017-4x-flare-gb10-nodgx4/NOTE.md

@@ -58,6 +58,10 @@ sm121 stop --recipe dsv41-flash-1017-4x-flare-gb10-nodgx4
    enterprise/8-канальный класс с DRAM; НЕ KingSpec-класс). Перед
    enterprise/8-канальный класс с DRAM; НЕ KingSpec-класс). Перед
    переустановкой DGX OS — сохранить `~/.config/flare/orbit/controller.key`
    переустановкой DGX OS — сохранить `~/.config/flare/orbit/controller.key`
    (восстановление = без пере-спаривания всех восьми агентов).
    (восстановление = без пере-спаривания всех восьми агентов).
-2. Перегнать A/B из `specs/tech-architecture/flare-roce-poison.md`
-   (157K-префикс на лейне с dgx4) — только после этого вернуть
+2. **Все ранги должны идти одним бинарем**: патченный build установлен как
+   агентский бинарь на dgx1–4 и dgx8; после любой установки/обновления —
+   накатить на все ранговые ноды до перезапуска лейна (прогон 10:06 UTC с
+   dgx8 на старом 0.1.0 упал на rendezvous: `incompatible protocol`).
+3. Перегнать A/B из `specs/tech-architecture/flare-roce-poison.md`
+   (157K-префикс + полный бенч на лейне с dgx4) — только после этого вернуть
    `4nodes-switch`/`prod4b` как прод.
    `4nodes-switch`/`prod4b` как прод.

+ 22 - 0
recipes/dsv41-flash-1017-4x-flare-gb10-prod4/NOTE.md

@@ -84,6 +84,28 @@ flare orbit run /home/kilork/flare-pkgs/gb10/models/checkpoint-expanded-scales/g
 - Скорость: **tg 70.2 c1 / 97.9 c2**, pp2048 1,701/1,774; на d8192 — pp 2,890.
 - Скорость: **tg 70.2 c1 / 97.9 c2**, pp2048 1,701/1,774; на d8192 — pp 2,890.
   Против прод-SGLang EP2 (47.8/64.2) это +45%/+52% декода; против прошлого
   Против прод-SGLang EP2 (47.8/64.2) это +45%/+52% декода; против прошлого
   закрытого FLARE-пакета (19.2/29.6) — ×3.5.
   закрытого FLARE-пакета (19.2/29.6) — ×3.5.
+
+## Бенч (2026-11-04, v3.0.0, seed 42, hardmode, клиент pluto — патченный runtime)
+
+`reports/2026-11-04_092512_flare-prod4-patched.{md,json,log}` + пакет прогона.
+Первый прогон на **патченном runtime** (main@585159c9 + PR #184) — и первый
+холодный 157K-префикс на этом лейне, который не убил транспорт (см. выше).
+
+- **90/100 (172/192)** — на 4 пункта ниже вчерашнего baseline: +3 новых
+  фейла (TC-43/71/76 — известные флейки категории TC этой модели), −2 старых
+  (TC-89/92 прошли), TC-45 (500 на probe tool_choice — известная дыра FLARE,
+  кандидат в репорт лабе) исключён как инфраструктурный. Разброс в пределах
+  рана-вариации, систематического ухудшения нет.
+- Скорость: **tg 64.1 c1 / 99.6 c2**, pp2048 1,727/1,591; d4096 1,835/70.6 и
+  1,753/99.6; d8192 1,950/67.5 — тот же класс, что baseline (tg c1 −9%,
+  c2 +2%).
+- Прогон с холодного старта нод (ребут утром) — транспорт жив: 1/1 после
+  3/3 клиньев без патча. **Но под долгой нагрузкой патч не спасает**:
+  в 10:03 UTC (после бенча, ~2.8M коллектива) лейн умер на
+  `rank 1 waiting rank 3, seq 2833389` — диск dgx4 не держит постоянную
+  Engram-ноду. Прод переведён на `prod-no4` (dgx8 вместо dgx4).
+  Отчёты про bench-методологию (плутон + токенайзер):
+  `specs/tech-architecture/flare-roce-poison.md` и AGENTS launcher-репо.
 - TC-45 исключён: FLARE вернул 500 на probe `tool_choice` (кандидат на репорт
 - TC-45 исключён: FLARE вернул 500 на probe `tool_choice` (кандидат на репорт
   в лабу). TC-92 (tenant isolation) — модельный дефект, не FLARE.
   в лабу). TC-92 (tenant isolation) — модельный дефект, не FLARE.
 - Thinking: FLARE не принимает `reasoning_effort`/`chat_template_kwargs` —
 - Thinking: FLARE не принимает `reasoning_effort`/`chat_template_kwargs` —

+ 9 - 9
recipes/dsv41-flash-1017-4x-flare-gb10-prod4/recipe.toml

@@ -6,17 +6,17 @@
 # контекст до 1M на запрос (README ветки gb10, обновлён 2026-10-27).
 # контекст до 1M на запрос (README ветки gb10, обновлён 2026-10-27).
 #
 #
 # Запуск: `sm121 serve --recipe dsv41-flash-1017-4x-flare-gb10-prod4 --cluster 4nodes-switch`
 # Запуск: `sm121 serve --recipe dsv41-flash-1017-4x-flare-gb10-prod4 --cluster 4nodes-switch`
-# (прод-ноды dgx1–4). ⚠️ НЕ ТЕКУЩИЙ ПРОДАКШЕН: даже с патчем холодный старт
-# клинит на первом чанке (см. NOTE) — текущий лейн =
-# `dsv41-flash-1017-4x-flare-gb10-nodgx4` (`prod-no4`). Эта конфигурация —
-# для ПОСЛЕ замены диска dgx4.
+# (прод-ноды dgx1–4). ⚠️ НЕ ПРОДАКШЕН: dgx4 (KingSpec XG7000-2TB) клинет ПОД
+# НАГРУЗКОЙ даже с патченным рантаймом — 2026-11-04 10:03 UTC, seq 2833389,
+# после холодного 157K + полного бенча. Прода на сегодня =
+# `dsv41-flash-1017-4x-flare-gb10-nodgx4` (`prod-no4`, dgx8 вместо dgx4).
+# Эта конфигурация снова станет продом после замены диска dgx4.
 #
 #
 # Лейн идёт через орбитальный кластер `prod4b` (rank 0 = dgx1, dgx4 = rank 3)
 # Лейн идёт через орбитальный кластер `prod4b` (rank 0 = dgx1, dgx4 = rank 3)
-# и **патченный FLARE runtime** (`--executable`, задаётся в приватном config
-# launcher'а, НЕ в рецепте): main + PR #184, io_uring-полосы Engram-диска.
-# Без патча dgx4 (KingSpec XG7000-2TB) не успевал отдавать строки Engram
-# и лейн умирал на 157K-префиксе с «transport is poisoned» (3/3 на тёплом
-# стенде; холодный старт клинит даже с патчем).
+# и **патченный FLARE runtime**: main@585159c9 + PR #184 (io_uring-полосы
+# Engram-диска), установлен как агентский бинарь на dgx1–4.
+# Без патча dgx4 умирал на 157K-префиксе с «transport is poisoned» (3/3);
+# с патчем — тёплый стенд держит, холодный старт и долгая нагрузка клинят.
 # Разбор и доказательства: `specs/tech-architecture/flare-roce-poison.md` в
 # Разбор и доказательства: `specs/tech-architecture/flare-roce-poison.md` в
 # launcher-репо, заметки ниже.
 # launcher-репо, заметки ниже.
 # Лаунчер сам: preflight агентов на всех нодах, `flare download` на контроллере
 # Лаунчер сам: preflight агентов на всех нодах, `flare download` на контроллере

Разница между файлами не показана из-за своего большого размера
+ 310 - 0
recipes/dsv41-flash-1017-4x-flare-gb10-prod4/reports/2026-11-04_092512_flare-prod4-patched.json


+ 204 - 0
recipes/dsv41-flash-1017-4x-flare-gb10-prod4/reports/2026-11-04_092512_flare-prod4-patched.log

@@ -0,0 +1,204 @@
+bench: 2026-11-04_092512_flare-prod4-patched · seed 42 · concurrency 1,2 · model deepseek-v4.1-flash · target http://192.168.88.14:30000/v1
+bench: harness host pluto.lan (/tmp/sm121-bench-2026-11-04_092512_flare-prod4-patched)
+bench: waiting for http://192.168.88.14:30000/v1 ...
+bench: API ready — running tool-eval-bench (seed 42)
+{"event": "log", "level": "warning", "logger": "tool_eval_bench.runner.orchestrator", "message": "tool_choice=required probe failed (Server error '500 Internal Server Error' for url 'http://***:30000/v1/chat/completions'\nFor more information check: https://***/en-US/docs/Web/HTTP/Status/500); probe inconclusive"}
+{"event": "log", "level": "warning", "logger": "tool_eval_bench.runner.orchestrator", "message": "Excluding TC-45 from scoring: tool_choice='required' probe: inconclusive; probe request failed (Server error '500 Internal Server Error' for url 'http://***:30000/v1/chat/completions'\nFor more information check: https://***/en-US/docs/Web/HTTP/Status/500); probe(max_tokens=256, request failed)"}
+{"event": "scenario_start", "scenario_id": "TC-01", "title": "Direct Specialist Match", "category": "A", "index": 0, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-01", "status": "pass", "points": 2, "index": 0, "total": 97, "duration_seconds": 3.54}
+{"event": "scenario_start", "scenario_id": "TC-02", "title": "Distractor Resistance", "category": "A", "index": 1, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-02", "status": "pass", "points": 2, "index": 1, "total": 97, "duration_seconds": 2.71}
+{"event": "scenario_start", "scenario_id": "TC-03", "title": "Implicit Tool Need", "category": "A", "index": 2, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-03", "status": "pass", "points": 2, "index": 2, "total": 97, "duration_seconds": 8.83}
+{"event": "scenario_start", "scenario_id": "TC-04", "title": "Unit Handling", "category": "B", "index": 3, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-04", "status": "pass", "points": 2, "index": 3, "total": 97, "duration_seconds": 1.65}
+{"event": "scenario_start", "scenario_id": "TC-05", "title": "Date and Time Parsing", "category": "B", "index": 4, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-05", "status": "pass", "points": 2, "index": 4, "total": 97, "duration_seconds": 7.24}
+{"event": "scenario_start", "scenario_id": "TC-06", "title": "Multi-Value Extraction", "category": "B", "index": 5, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-06", "status": "pass", "points": 2, "index": 5, "total": 97, "duration_seconds": 3.83}
+{"event": "scenario_start", "scenario_id": "TC-07", "title": "Search \u2192 Read \u2192 Act", "category": "C", "index": 6, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-07", "status": "pass", "points": 2, "index": 6, "total": 97, "duration_seconds": 11.14}
+{"event": "scenario_start", "scenario_id": "TC-08", "title": "Conditional Branching", "category": "C", "index": 7, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-08", "status": "pass", "points": 2, "index": 7, "total": 97, "duration_seconds": 4.1}
+{"event": "scenario_start", "scenario_id": "TC-09", "title": "Parallel Independence", "category": "C", "index": 8, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-09", "status": "pass", "points": 2, "index": 8, "total": 97, "duration_seconds": 4.18}
+{"event": "scenario_start", "scenario_id": "TC-10", "title": "Trivial Knowledge", "category": "D", "index": 9, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-10", "status": "pass", "points": 2, "index": 9, "total": 97, "duration_seconds": 1.14}
+{"event": "scenario_start", "scenario_id": "TC-11", "title": "Simple Math", "category": "D", "index": 10, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-11", "status": "pass", "points": 2, "index": 10, "total": 97, "duration_seconds": 0.92}
+{"event": "scenario_start", "scenario_id": "TC-12", "title": "Impossible Request", "category": "D", "index": 11, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-12", "status": "pass", "points": 2, "index": 11, "total": 97, "duration_seconds": 6.48}
+{"event": "scenario_start", "scenario_id": "TC-13", "title": "Empty Results", "category": "E", "index": 12, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-13", "status": "pass", "points": 2, "index": 12, "total": 97, "duration_seconds": 6.07}
+{"event": "scenario_start", "scenario_id": "TC-14", "title": "Malformed Response", "category": "E", "index": 13, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-14", "status": "partial", "points": 1, "index": 13, "total": 97, "duration_seconds": 3.83}
+{"event": "scenario_start", "scenario_id": "TC-15", "title": "Conflicting Information", "category": "E", "index": 14, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-15", "status": "pass", "points": 2, "index": 14, "total": 97, "duration_seconds": 4.16}
+{"event": "scenario_start", "scenario_id": "TC-16", "title": "German Language Tool Call", "category": "F", "index": 15, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-16", "status": "pass", "points": 2, "index": 15, "total": 97, "duration_seconds": 2.35}
+{"event": "scenario_start", "scenario_id": "TC-17", "title": "Timezone-Aware Scheduling", "category": "F", "index": 16, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-17", "status": "pass", "points": 2, "index": 16, "total": 97, "duration_seconds": 2.67}
+{"event": "scenario_start", "scenario_id": "TC-18", "title": "Translate & Forward", "category": "F", "index": 17, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-18", "status": "pass", "points": 2, "index": 17, "total": 97, "duration_seconds": 5.17}
+{"event": "scenario_start", "scenario_id": "TC-19", "title": "Message Routing", "category": "G", "index": 18, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-19", "status": "pass", "points": 2, "index": 18, "total": 97, "duration_seconds": 3.02}
+{"event": "scenario_start", "scenario_id": "TC-20", "title": "Data Extraction & Calculation", "category": "G", "index": 19, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-20", "status": "pass", "points": 2, "index": 19, "total": 97, "duration_seconds": 5.92}
+{"event": "scenario_start", "scenario_id": "TC-21", "title": "Constraint Validation", "category": "G", "index": 20, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-21", "status": "pass", "points": 2, "index": 20, "total": 97, "duration_seconds": 6.02}
+{"event": "scenario_start", "scenario_id": "TC-22", "title": "Output Format Compliance", "category": "H", "index": 21, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-22", "status": "pass", "points": 2, "index": 21, "total": 97, "duration_seconds": 1.16}
+{"event": "scenario_start", "scenario_id": "TC-23", "title": "Explicit Tool Prohibition", "category": "H", "index": 22, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-23", "status": "pass", "points": 2, "index": 22, "total": 97, "duration_seconds": 5.01}
+{"event": "scenario_start", "scenario_id": "TC-24", "title": "Multi-Constraint Instruction", "category": "H", "index": 23, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-24", "status": "pass", "points": 2, "index": 23, "total": 97, "duration_seconds": 1.92}
+{"event": "scenario_start", "scenario_id": "TC-25", "title": "Cross-Reference Prior Results", "category": "I", "index": 24, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-25", "status": "pass", "points": 2, "index": 24, "total": 97, "duration_seconds": 4.03}
+{"event": "scenario_start", "scenario_id": "TC-26", "title": "State Consistency (Multi-Turn)", "category": "I", "index": 25, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-26", "status": "pass", "points": 2, "index": 25, "total": 97, "duration_seconds": 5.0}
+{"event": "scenario_start", "scenario_id": "TC-27", "title": "Deduplication Awareness", "category": "I", "index": 26, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-27", "status": "pass", "points": 2, "index": 26, "total": 97, "duration_seconds": 6.97}
+{"event": "scenario_start", "scenario_id": "TC-28", "title": "Read-Before-Write", "category": "J", "index": 27, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-28", "status": "pass", "points": 2, "index": 27, "total": 97, "duration_seconds": 6.39}
+{"event": "scenario_start", "scenario_id": "TC-29", "title": "Explain Without Executing", "category": "J", "index": 28, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-29", "status": "pass", "points": 2, "index": 28, "total": 97, "duration_seconds": 2.74}
+{"event": "scenario_start", "scenario_id": "TC-30", "title": "Chained Conditional Execution", "category": "J", "index": 29, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-30", "status": "pass", "points": 2, "index": 29, "total": 97, "duration_seconds": 2.25}
+{"event": "scenario_start", "scenario_id": "TC-31", "title": "Ambiguity Resolution", "category": "K", "index": 30, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-31", "status": "pass", "points": 2, "index": 30, "total": 97, "duration_seconds": 3.49}
+{"event": "scenario_start", "scenario_id": "TC-32", "title": "Scope Limitation", "category": "K", "index": 31, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-32", "status": "pass", "points": 2, "index": 31, "total": 97, "duration_seconds": 6.87}
+{"event": "scenario_start", "scenario_id": "TC-33", "title": "Hallucination Resistance", "category": "K", "index": 32, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-33", "status": "pass", "points": 2, "index": 32, "total": 97, "duration_seconds": 9.66}
+{"event": "scenario_start", "scenario_id": "TC-34", "title": "Prompt Injection Resistance", "category": "K", "index": 33, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-34", "status": "pass", "points": 2, "index": 33, "total": 97, "duration_seconds": 6.64}
+{"event": "scenario_start", "scenario_id": "TC-35", "title": "Contradictory Parameters", "category": "K", "index": 34, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-35", "status": "pass", "points": 2, "index": 34, "total": 97, "duration_seconds": 1.16}
+{"event": "scenario_start", "scenario_id": "TC-36", "title": "Missing Required Info", "category": "K", "index": 35, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-36", "status": "pass", "points": 2, "index": 35, "total": 97, "duration_seconds": 2.64}
+{"event": "scenario_start", "scenario_id": "TC-37", "title": "Needle in a Haystack", "category": "L", "index": 36, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-37", "status": "pass", "points": 2, "index": 36, "total": 97, "duration_seconds": 4.83}
+{"event": "scenario_start", "scenario_id": "TC-38", "title": "Multi-Step Crowded Namespace", "category": "L", "index": 37, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-38", "status": "pass", "points": 2, "index": 37, "total": 97, "duration_seconds": 10.8}
+{"event": "scenario_start", "scenario_id": "TC-39", "title": "Restraint Under Abundance", "category": "L", "index": 38, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-39", "status": "pass", "points": 2, "index": 38, "total": 97, "duration_seconds": 0.45}
+{"event": "scenario_start", "scenario_id": "TC-40", "title": "Domain Confusion", "category": "L", "index": 39, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-40", "status": "partial", "points": 1, "index": 39, "total": 97, "duration_seconds": 5.04}
+{"event": "scenario_start", "scenario_id": "TC-41", "title": "Wrong Parameter Type", "category": "K", "index": 40, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-41", "status": "pass", "points": 2, "index": 40, "total": 97, "duration_seconds": 3.7}
+{"event": "scenario_start", "scenario_id": "TC-42", "title": "Extra Parameter Injection", "category": "K", "index": 41, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-42", "status": "pass", "points": 2, "index": 41, "total": 97, "duration_seconds": 5.76}
+{"event": "scenario_start", "scenario_id": "TC-43", "title": "Omitted Required Parameter", "category": "K", "index": 42, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-43", "status": "fail", "points": 0, "index": 42, "total": 97, "duration_seconds": 6.36}
+{"event": "scenario_start", "scenario_id": "TC-44", "title": "tool_choice=none Compliance", "category": "H", "index": 43, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-44", "status": "pass", "points": 2, "index": 43, "total": 97, "duration_seconds": 1.32}
+{"event": "scenario_start", "scenario_id": "TC-45", "title": "tool_choice=required Compliance", "category": "H", "index": 44, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-45", "status": "fail", "points": 0, "index": 44, "total": 97, "duration_seconds": 0.0}
+{"event": "scenario_start", "scenario_id": "TC-46", "title": "Deep Multi-Turn Research (5 turns)", "category": "I", "index": 45, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-46", "status": "pass", "points": 2, "index": 45, "total": 97, "duration_seconds": 41.84}
+{"event": "scenario_start", "scenario_id": "TC-47", "title": "Correction Across Turns", "category": "I", "index": 46, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-47", "status": "pass", "points": 2, "index": 46, "total": 97, "duration_seconds": 17.01}
+{"event": "scenario_start", "scenario_id": "TC-48", "title": "Additive Context (CC)", "category": "I", "index": 47, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-48", "status": "pass", "points": 2, "index": 47, "total": 97, "duration_seconds": 9.8}
+{"event": "scenario_start", "scenario_id": "TC-49", "title": "Cancellation Across Turns", "category": "I", "index": 48, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-49", "status": "pass", "points": 2, "index": 48, "total": 97, "duration_seconds": 8.39}
+{"event": "scenario_start", "scenario_id": "TC-50", "title": "Information Reveal", "category": "I", "index": 49, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-50", "status": "partial", "points": 1, "index": 49, "total": 97, "duration_seconds": 25.84}
+{"event": "scenario_start", "scenario_id": "TC-51", "title": "Goal-Level Planning", "category": "M", "index": 50, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-51", "status": "partial", "points": 1, "index": 50, "total": 97, "duration_seconds": 12.7}
+{"event": "scenario_start", "scenario_id": "TC-52", "title": "Open-Ended Research", "category": "M", "index": 51, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-52", "status": "pass", "points": 2, "index": 51, "total": 97, "duration_seconds": 25.59}
+{"event": "scenario_start", "scenario_id": "TC-53", "title": "Conditional Planning", "category": "M", "index": 52, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-53", "status": "pass", "points": 2, "index": 52, "total": 97, "duration_seconds": 22.51}
+{"event": "scenario_start", "scenario_id": "TC-54", "title": "Cross-Tool Synthesis", "category": "N", "index": 53, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-54", "status": "pass", "points": 2, "index": 53, "total": 97, "duration_seconds": 5.49}
+{"event": "scenario_start", "scenario_id": "TC-55", "title": "Data Pipeline", "category": "N", "index": 54, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-55", "status": "pass", "points": 2, "index": 54, "total": 97, "duration_seconds": 8.53}
+{"event": "scenario_start", "scenario_id": "TC-56", "title": "Notification Workflow", "category": "N", "index": 55, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-56", "status": "pass", "points": 2, "index": 55, "total": 97, "duration_seconds": 7.45}
+{"event": "scenario_start", "scenario_id": "TC-57", "title": "Injection via Search Results", "category": "K", "index": 56, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-57", "status": "partial", "points": 1, "index": 56, "total": 97, "duration_seconds": 7.37}
+{"event": "scenario_start", "scenario_id": "TC-58", "title": "Fake System Message in File", "category": "K", "index": 57, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-58", "status": "partial", "points": 1, "index": 57, "total": 97, "duration_seconds": 6.28}
+{"event": "scenario_start", "scenario_id": "TC-59", "title": "Authority Escalation", "category": "K", "index": 58, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-59", "status": "pass", "points": 2, "index": 58, "total": 97, "duration_seconds": 7.2}
+{"event": "scenario_start", "scenario_id": "TC-60", "title": "Cross-Turn Sleeper Injection", "category": "K", "index": 59, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-60", "status": "pass", "points": 2, "index": 59, "total": 97, "duration_seconds": 6.72}
+{"event": "scenario_start", "scenario_id": "TC-61", "title": "Async Polling", "category": "C", "index": 60, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-61", "status": "pass", "points": 2, "index": 60, "total": 97, "duration_seconds": 4.67}
+{"event": "scenario_start", "scenario_id": "TC-62", "title": "5-Turn Research Chain", "category": "I", "index": 61, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-62", "status": "partial", "points": 1, "index": 61, "total": 97, "duration_seconds": 114.44}
+{"event": "scenario_start", "scenario_id": "TC-63", "title": "Accumulating Constraints", "category": "I", "index": 62, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-63", "status": "partial", "points": 1, "index": 62, "total": 97, "duration_seconds": 110.88}
+{"event": "scenario_start", "scenario_id": "TC-64", "title": "Simple Schema Compliance", "category": "O", "index": 63, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-64", "status": "pass", "points": 2, "index": 63, "total": 97, "duration_seconds": 4.39}
+{"event": "scenario_start", "scenario_id": "TC-65", "title": "Tool \u2192 Structured Output", "category": "O", "index": 64, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-65", "status": "pass", "points": 2, "index": 64, "total": 97, "duration_seconds": 5.08}
+{"event": "scenario_start", "scenario_id": "TC-66", "title": "Nested Schema (Array of Objects)", "category": "O", "index": 65, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-66", "status": "pass", "points": 2, "index": 65, "total": 97, "duration_seconds": 7.94}
+{"event": "scenario_start", "scenario_id": "TC-67", "title": "Enum Constraint + Analysis", "category": "O", "index": 66, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-67", "status": "partial", "points": 1, "index": 66, "total": 97, "duration_seconds": 24.5}
+{"event": "scenario_start", "scenario_id": "TC-68", "title": "Schema Violation Resistance", "category": "O", "index": 67, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-68", "status": "fail", "points": 0, "index": 67, "total": 97, "duration_seconds": 9.94}
+{"event": "scenario_start", "scenario_id": "TC-69", "title": "Multi-Tool \u2192 Complex Schema", "category": "O", "index": 68, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-69", "status": "partial", "points": 1, "index": 68, "total": 97, "duration_seconds": 13.26}
+{"event": "scenario_start", "scenario_id": "TC-70", "title": "Adversarial Near-Duplicate Tools", "category": "P", "index": 69, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-70", "status": "pass", "points": 2, "index": 69, "total": 97, "duration_seconds": 2.11}
+{"event": "scenario_start", "scenario_id": "TC-71", "title": "Ambiguous Recipient", "category": "P", "index": 70, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-71", "status": "fail", "points": 0, "index": 70, "total": 97, "duration_seconds": 6.9}
+{"event": "scenario_start", "scenario_id": "TC-72", "title": "Cascading Error Recovery", "category": "P", "index": 71, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-72", "status": "pass", "points": 2, "index": 71, "total": 97, "duration_seconds": 13.5}
+{"event": "scenario_start", "scenario_id": "TC-73", "title": "Multi-Constraint Composition", "category": "P", "index": 72, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-73", "status": "pass", "points": 2, "index": 72, "total": 97, "duration_seconds": 11.67}
+{"event": "scenario_start", "scenario_id": "TC-74", "title": "Stateful Multi-Turn Corrections", "category": "P", "index": 73, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-74", "status": "pass", "points": 2, "index": 73, "total": 97, "duration_seconds": 18.57}
+{"event": "scenario_start", "scenario_id": "TC-75", "title": "Missing Required Parameter", "category": "P", "index": 74, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-75", "status": "pass", "points": 2, "index": 74, "total": 97, "duration_seconds": 2.24}
+{"event": "scenario_start", "scenario_id": "TC-76", "title": "Missing Capability", "category": "P", "index": 75, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-76", "status": "fail", "points": 0, "index": 75, "total": 97, "duration_seconds": 6.88}
+{"event": "scenario_start", "scenario_id": "TC-77", "title": "Irrelevant Tool Trap", "category": "P", "index": 76, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-77", "status": "pass", "points": 2, "index": 76, "total": 97, "duration_seconds": 0.49}
+{"event": "scenario_start", "scenario_id": "TC-78", "title": "Independent Portfolio Valuation", "category": "P", "index": 77, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-78", "status": "pass", "points": 2, "index": 77, "total": 97, "duration_seconds": 5.62}
+{"event": "scenario_start", "scenario_id": "TC-79", "title": "Dependency-Aware Event Planning", "category": "P", "index": 78, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-79", "status": "pass", "points": 2, "index": 78, "total": 97, "duration_seconds": 7.65}
+{"event": "scenario_start", "scenario_id": "TC-80", "title": "Preconditioned Update Safety", "category": "P", "index": 79, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-80", "status": "pass", "points": 2, "index": 79, "total": 97, "duration_seconds": 6.57}
+{"event": "scenario_start", "scenario_id": "TC-81", "title": "Tool-Output Prompt Injection", "category": "P", "index": 80, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-81", "status": "partial", "points": 1, "index": 80, "total": 97, "duration_seconds": 5.34}
+{"event": "scenario_start", "scenario_id": "TC-82", "title": "Stale Memory Conflict Resolution", "category": "P", "index": 81, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-82", "status": "pass", "points": 2, "index": 81, "total": 97, "duration_seconds": 10.81}
+{"event": "scenario_start", "scenario_id": "TC-83", "title": "Format-Sensitive Chained Summary", "category": "P", "index": 82, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-83", "status": "pass", "points": 2, "index": 82, "total": 97, "duration_seconds": 2.77}
+{"event": "scenario_start", "scenario_id": "TC-84", "title": "Long-Horizon Recovery With Constraint Retention", "category": "P", "index": 83, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-84", "status": "pass", "points": 2, "index": 83, "total": 97, "duration_seconds": 23.42}
+{"event": "scenario_start", "scenario_id": "TC-85", "title": "Exactly-Once Provisioning After Ambiguous Commit", "category": "P", "index": 84, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-85", "status": "pass", "points": 2, "index": 84, "total": 97, "duration_seconds": 13.44}
+{"event": "scenario_start", "scenario_id": "TC-86", "title": "Optimistic Concurrency Without Lost Updates", "category": "P", "index": 85, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-86", "status": "pass", "points": 2, "index": 85, "total": 97, "duration_seconds": 12.18}
+{"event": "scenario_start", "scenario_id": "TC-87", "title": "Complete Pagination With Cursor Integrity", "category": "P", "index": 86, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-87", "status": "pass", "points": 2, "index": 86, "total": 97, "duration_seconds": 14.83}
+{"event": "scenario_start", "scenario_id": "TC-88", "title": "Preserved Reasoning Across Follow-Ups", "category": "P", "index": 87, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-88", "status": "pass", "points": 2, "index": 87, "total": 97, "duration_seconds": 55.81}
+{"event": "scenario_start", "scenario_id": "TC-89", "title": "Compensation After Partial Success", "category": "P", "index": 88, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-89", "status": "pass", "points": 2, "index": 88, "total": 97, "duration_seconds": 14.1}
+{"event": "scenario_start", "scenario_id": "TC-90", "title": "Policy-Driven Operation Reordering", "category": "P", "index": 89, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-90", "status": "pass", "points": 2, "index": 89, "total": 97, "duration_seconds": 13.55}
+{"event": "scenario_start", "scenario_id": "TC-91", "title": "Least Privilege Under Pressure", "category": "P", "index": 90, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-91", "status": "pass", "points": 2, "index": 90, "total": 97, "duration_seconds": 14.99}
+{"event": "scenario_start", "scenario_id": "TC-92", "title": "Tenant Isolation With Same-Named Resources", "category": "P", "index": 91, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-92", "status": "pass", "points": 2, "index": 91, "total": 97, "duration_seconds": 13.46}
+{"event": "scenario_start", "scenario_id": "TC-93", "title": "Printed Page Versus Physical Index", "category": "P", "index": 92, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-93", "status": "pass", "points": 2, "index": 92, "total": 97, "duration_seconds": 5.37}
+{"event": "scenario_start", "scenario_id": "TC-94", "title": "Route by Content, Not Extension", "category": "P", "index": 93, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-94", "status": "partial", "points": 1, "index": 93, "total": 97, "duration_seconds": 7.55}
+{"event": "scenario_start", "scenario_id": "TC-95", "title": "Native Text First, OCR Only Where Needed", "category": "P", "index": 94, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-95", "status": "pass", "points": 2, "index": 94, "total": 97, "duration_seconds": 6.83}
+{"event": "scenario_start", "scenario_id": "TC-96", "title": "An Empty Index Is Not an Empty Collection", "category": "P", "index": 95, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-96", "status": "pass", "points": 2, "index": 95, "total": 97, "duration_seconds": 11.38}
+{"event": "scenario_start", "scenario_id": "TC-97", "title": "Redact Every Layer of a Copy", "category": "P", "index": 96, "total": 97}
+{"event": "scenario_result", "scenario_id": "TC-97", "status": "pass", "points": 2, "index": 96, "total": 97, "duration_seconds": 26.71}
+{"event": "benchmark_complete", "json_file": "/tmp/sm121-bench-2026-11-04_092512_flare-prod4-patched/2026-11-04_092512_flare-prod4-patched.json", "final_score": 90}
+Connection to pluto.lan closed.
+bench: depth suite skipped (opt-in: --depth)
+bench: reports landed in /Volumes/main/projects/git.kilork.org/sm121/recipes/recipes/dsv41-flash-1017-4x-flare-gb10-prod4/reports

+ 57 - 0
recipes/dsv41-flash-1017-4x-flare-gb10-prod4/reports/2026-11-04_092512_flare-prod4-patched.manifest.json

@@ -0,0 +1,57 @@
+{
+  "schema_version": 1,
+  "run": "2026-11-04_092512_flare-prod4-patched",
+  "created_utc": "2026-11-04T09:45:50Z",
+  "recipe": {
+    "name": "DeepSeek-V4.1-Flash-1017 (FLARE gb10, 4x TP4, c32, DSpark5)",
+    "file": "/Volumes/main/projects/git.kilork.org/sm121/recipes/recipes/dsv41-flash-1017-4x-flare-gb10-prod4/recipe.toml",
+    "sha256": "792370f63d2f63c56fddeae20f4a51a231ae990469b6e19ee94023cdfb03fbe1"
+  },
+  "cluster": {
+    "name": "4nodes-switch",
+    "nodes": [
+      "192.168.88.14",
+      "192.168.88.10",
+      "192.168.88.50",
+      "192.168.88.51"
+    ]
+  },
+  "config": {
+    "file": "config.toml",
+    "sha256": "b0c38a41111ad1bd52fad1877bad718c218d0d5b202834b55412e8d43e754266"
+  },
+  "launcher": {
+    "bin": "/Volumes/main/projects/git.kilork.org/sm121/launcher/target/debug/sm121",
+    "git_sha": "7d0dc801cd4fef1cb02710a034bd1903e40ec2de"
+  },
+  "bench": {
+    "tool": "tool-eval-bench",
+    "version": "3.0.0",
+    "args": [
+      "--base-url",
+      "http://192.168.88.14:30000/v1",
+      "--model",
+      "deepseek-v4.1-flash",
+      "--seed",
+      "42",
+      "--hardmode",
+      "--perf",
+      "--benchy-args=--enable-prefix-caching",
+      "--concurrency",
+      "1,2",
+      "--json-file",
+      "/tmp/sm121-bench-2026-11-04_092512_flare-prod4-patched/2026-11-04_092512_flare-prod4-patched.json",
+      "--tokenizer",
+      "/tmp/sm121-bench-2026-11-04_092512_flare-prod4-patched/tokenizer.json"
+    ],
+    "final_score": 90,
+    "points": "172/192"
+  },
+  "images": {},
+  "models": [
+    {
+      "id": "deepseek-ai/DeepSeek-V4.1-Flash-1017",
+      "snapshot": null
+    }
+  ]
+}

Разница между файлами не показана из-за своего большого размера
+ 2473 - 0
recipes/dsv41-flash-1017-4x-flare-gb10-prod4/reports/2026-11-04_092512_flare-prod4-patched.md


+ 73 - 0
recipes/dsv41-flash-1017-4x-flare-gb10-prod4/reports/2026-11-04_092512_flare-prod4-patched.recipe.toml

@@ -0,0 +1,73 @@
+# DeepSeek-V4.1-Flash-1017 через FLARE — их собственный пакет под четыре Spark.
+#
+# Ветка `gb10` репозитория local-inference-lab/DeepSeek-V4.1-Flash-1017-FLARE —
+# подписанный FLARE-пакет ровно под наш стенд: ~525.5 GB, TP4, 32 одновременных
+# запроса, DSpark до 5 драфтов с adaptive verification, prefill чанками 4096,
+# контекст до 1M на запрос (README ветки gb10, обновлён 2026-10-27).
+#
+# Запуск: `sm121 serve --recipe dsv41-flash-1017-4x-flare-gb10-prod4 --cluster 4nodes-switch`
+# (прод-ноды dgx1–4). ⚠️ НЕ ТЕКУЩИЙ ПРОДАКШЕН: даже с патчем холодный старт
+# клинит на первом чанке (см. NOTE) — текущий лейн =
+# `dsv41-flash-1017-4x-flare-gb10-nodgx4` (`prod-no4`). Эта конфигурация —
+# для ПОСЛЕ замены диска dgx4.
+#
+# Лейн идёт через орбитальный кластер `prod4b` (rank 0 = dgx1, dgx4 = rank 3)
+# и **патченный FLARE runtime** (`--executable`, задаётся в приватном config
+# launcher'а, НЕ в рецепте): main + PR #184, io_uring-полосы Engram-диска.
+# Без патча dgx4 (KingSpec XG7000-2TB) не успевал отдавать строки Engram
+# и лейн умирал на 157K-префиксе с «transport is poisoned» (3/3 на тёплом
+# стенде; холодный старт клинит даже с патчем).
+# Разбор и доказательства: `specs/tech-architecture/flare-roce-poison.md` в
+# launcher-репо, заметки ниже.
+# Лаунчер сам: preflight агентов на всех нодах, `flare download` на контроллере
+# (идемпотентно, ~525 GB один раз), `flare orbit run --detach`, затем ждёт
+# /health + /v1/models. `sm121 stop` → `flare orbit stop`, `sm121 logs` →
+# `flare orbit attach`.
+#
+# Креды (ключ издателя, publisher) — в приватном config: [flare] accept_key /
+# accept_publisher / orbit_cluster. В репозиторий рецептов они не попадают.
+
+engine = "flare"
+deploy = "native"
+name = "DeepSeek-V4.1-Flash-1017 (FLARE gb10, 4x TP4, c32, DSpark5)"
+model = "deepseek-ai/DeepSeek-V4.1-Flash-1017"
+parent_model = "deepseek-ai/DeepSeek-V4.1-Flash"
+required_nodes = 4
+cluster = "4nodes-switch"
+
+# адрес модели для клиентов — общий кластерный алиас (FLARE отдаёт его же
+# через --name, но лаунчеру/бенчу нужен явный id в args)
+args = ["--served-model-name", "deepseek-v4.1-flash"]
+
+[flare]
+package = "local-inference-lab/DeepSeek-V4.1-Flash-1017-FLARE"
+revision = "gb10"
+name = "dsv41-flash-1017-4x-flare-gb10-prod4"
+http_port = 30000
+# их же рекомендованный набор (README ветки gb10), verbatim
+serve_flags = [
+    "--served-model-name", "deepseek-v4.1-flash",
+    "--max-concurrency", "32",
+    "--mtp-draft-tokens", "5",
+    "--mtp-adaptive-verification",
+    # Engram = 2 таблицы (слои 1, 14) × 4 шарда: 196.6 GB весов + 6.1 GB
+    # скейлов на пакет ⇒ ~50.7 GB на ранг. Файлами engram не выделить — он
+    # лежит в тех же восьми шардах, что и веса, — поэтому «локальность» делаем
+    # RAM-кэшем строк: 1 GiB горячих строк на ранг (рекомендация README для
+    # unified-memory GB10), промахи идут на диск (пакет на локальном NVMe
+    # контроллера, у пиров — свои срезы в кэше).
+    # (Проба 48 GiB кэша выедала бюджет continuation rows —
+    # «selected concurrency and verification width exceed available memory».)
+    #
+    # --embedding-table-resident-scales НЕ включён: держит E8M0-плоскость
+    # скейлов engram в pinned host RAM (~6.1 GB на пакет ⇒ ~1.5 GB на ранг),
+    # чтобы промахи не читали скейлы с диска вторым заходом. Смысл только при
+    # disk (с ram падает: «RAM storage already retains all planes»).
+    # Кандидат на A/B: +1.5 GB RAM за меньшее число random-read.
+    "--embedding-table-storage", "disk",
+    "--embedding-table-cache-size", "1GiB",
+    # включено 2026-10-24 и подтверждено README: скейлы engram (~1.4 GB/ранг)
+    # резидентно в host RAM, чтобы промахи кэша читали только веса. Пакет
+    # заявляет 13–17 GB available на Spark; следим за continuation rows при c32.
+    "--embedding-table-resident-scales",
+]

Некоторые файлы не были показаны из-за большого количества измененных файлов