|
|
@@ -0,0 +1,82 @@
|
|
|
+# Upstream watch — 2026-10-16 (night shift)
|
|
|
+
|
|
|
+Autonomous pass during the 10-15→16 night shift. Focus: the two queued models
|
|
|
+(Ling-3.1-flash, Step-5-Preview) + repo/forum deltas. Launcher-side only; no
|
|
|
+new node deploys (cluster powers off at end of shift).
|
|
|
+
|
|
|
+## The two queued models — feasibility (the headline)
|
|
|
+
|
|
|
+### Ling-3.1-flash (inclusionAI) — arch SUPPORTED on our base, but no small weights yet
|
|
|
+- Architecture: **`BailingMoeV3ForCausalLM`** (`model_type: bailing_hybrid`,
|
|
|
+ `custom_code`, fp8). Ant Group's Ling/Bailing MoE-V3 line.
|
|
|
+- **vLLM v0.31.0 (our kindling base) already registers it**:
|
|
|
+ `BailingMoeV3ForCausalLM → bailing_moe_v3` AND `BailingMoeV3MTPModel →
|
|
|
+ bailing_moe_v3_mtp` (MTP head supported). So **no nightly vLLM needed** — the
|
|
|
+ earlier "requires nightly" note is stale for the arch. A Ling lane can ride a
|
|
|
+ v0.31.0-based image (kindling or stock).
|
|
|
+- BUT the day-0 **NVFP4 cut `cbfight/Ling-3.1-flash-NVFP4-lossless8` is an
|
|
|
+ EMPTY repo** (only `.gitattributes`, 0 bytes) — reserved, not populated. So
|
|
|
+ the cheap 4x-lane (NVFP4 ≈ GLM-5.3-Flash class) is **not available yet**.
|
|
|
+- The only vLLM-loadable Ling today is the **original `inclusionAI/Ling-3.1-flash`**
|
|
|
+ (fp8, 572B total, ~689 GB) → **TP8+EP8 = all 8 Sparks**. Not a 4x lane.
|
|
|
+- Other cuts are GGUF/EXL3 (llama.cpp / exllamav3), not our vLLM stack:
|
|
|
+ `AtomicChat/Ling-3.1-flash-GGUF`, `AMAImedia/...-BF16-GGUF`,
|
|
|
+ `vcruz305/...-EXL3-SAGE-*`.
|
|
|
+- **Action:** watch `cbfight` (and inclusionAI) for a real NVFP4/FP8-small cut →
|
|
|
+ then a 4x lane on a v0.31.0 base is straightforward (arch already works).
|
|
|
+ A 572B-fp8 8x lane is possible but heavy (689 GB download + all 8 Sparks);
|
|
|
+ not attempted overnight before power-off. Recipe authored as prep:
|
|
|
+ `recipes/ling-31-flash-nvfp4/` (targets the future NVFP4 cut; 4x).
|
|
|
+
|
|
|
+### Step-5-Preview-NVFP4 (stepfun-ai) — VL model, needs the dedicated image
|
|
|
+- Architecture: **`Step5ForConditionalGeneration`** (`model_type: step5_vl`) —
|
|
|
+ a **vision-language** model (8-bit, modelopt/NVFP4), NOT text-only.
|
|
|
+- vLLM v0.31.0 registers `StepVLForConditionalGeneration` (`step_vl`) but **not**
|
|
|
+ `Step5ForConditionalGeneration` (`step5_vl`) — a different, newer arch. So
|
|
|
+ stock v0.31.0 won't load it; needs the **dedicated image
|
|
|
+ `vllm/vllm-openai:step-5-preview`** (or a vLLM new enough to carry step5_vl).
|
|
|
+- 604B, ~473 GB → 8 Sparks (TP8). **GB10/sm_121a unverified** (kernels, UMA
|
|
|
+ memory gate vLLM #48140, GMU). This is a research probe, not a bench target.
|
|
|
+- **Action:** recipe authored as prep `recipes/step5-preview-nvfp4/` (dedicated
|
|
|
+ image, 8x). A live GB10 probe needs the 8x cluster up + a 473 GB download —
|
|
|
+ deferred to a supervised window (wasteful to pull overnight before power-off).
|
|
|
+
|
|
|
+## Repo deltas (GitHub / Docker)
|
|
|
+
|
|
|
+- **vLLM**: latest stable tag still **v0.31.0**; v0.32.0 is **rc0/rc1/rc2** only
|
|
|
+ (no stable). Our kindling base (v0.31.0) is current. Do NOT re-pin overnight —
|
|
|
+ the v0.32.0 stable re-pin + 38-overlay rebase is a supervised, bench-gated task.
|
|
|
+- **HawkBearPig/dgpp**: HEAD moved to **`d9311e72…`** (our recipe pin is
|
|
|
+ `3ad63d6b…`). DGPP is active daily. Consider bumping the pin after reading the
|
|
|
+ changelog (supervised) — the provisioning marker keys on `rev`, so a bump
|
|
|
+ triggers a clean rebuild.
|
|
|
+- **kindlingai/gr8**: HEAD `3df902dc…`, active.
|
|
|
+- **SeraphimSerapis/tool-eval-bench**: latest tag **v2.8.0** = our pin. No newer
|
|
|
+ release; harness stays pinned (do not upgrade casually — re-baseline anchors).
|
|
|
+- **vllm/vllm-openai** (Docker Hub): `step-5-preview` tag is the Step-5 base
|
|
|
+ (dedicated image, per the model's requirement).
|
|
|
+
|
|
|
+## DGPP w2 live-run findings (from the provisioning test, 2nodes-dac)
|
|
|
+
|
|
|
+Provisioning (clone-at-pin → release.sh → install → up) now works end-to-end
|
|
|
+(4 launcher bugs fixed — see `engine-drivers.md`). DGPP's own preflight then
|
|
|
+gates on deployment prereqs the launcher does NOT own:
|
|
|
+1. **Checkpoint missing** — `HawkBearPig/GLM-5.3-Flash-NVFP4-FP8` not in the
|
|
|
+ node HF cache. DGPP: `python3 scripts/download_model.py --config <file>` on
|
|
|
+ rank 0 (or pre-stage the snapshot). ~GLM-5.3-Flash class size.
|
|
|
+2. **RoCE lanes** — "the transport supports at most two lanes; select
|
|
|
+ `DGPP_ROCE_DEVICES` explicitly" on multi-fabric hosts. The site `.env` needs
|
|
|
+ `DGPP_ROCE_DEVICES` set to the two fabric devices (per the DGPP thread
|
|
|
+ 383406). Not yet encoded in the launcher's generated `.env` — a candidate
|
|
|
+ follow-up (derive from config `nccl.ib_hca`, or a recipe `[dgpp].roce_devices`).
|
|
|
+3. Preflight is strict: any FAIL → "no services started". So w2 needs BOTH the
|
|
|
+ model and the RoCE selection before it serves.
|
|
|
+
|
|
|
+## Not done overnight (deferred, supervised)
|
|
|
+
|
|
|
+- Ling 572B-fp8 8x lane (689 GB + all 8 Sparks) and Step-5 8x GB10 probe
|
|
|
+ (473 GB + dedicated image) — both heavy downloads that a power-off would not
|
|
|
+ waste (NVMe caches persist) but are too large/risky to run unsupervised, and
|
|
|
+ the arch/image prerequisites need a supervised base decision.
|
|
|
+- DGPP w2 full serve — needs model download + `DGPP_ROCE_DEVICES`; the launcher
|
|
|
+ side is validated.
|
