|
|
@@ -101,6 +101,23 @@ supported in vLLM v0.31.0 (`MuseGlimmerForCausalLM` / `MuseGlimmerForConditional
|
|
|
on NAS. If we want a Muse lane this shift, Glimmer-30B (single Spark) is the one
|
|
|
that actually loads — not the 816B.
|
|
|
|
|
|
+### SGLang is NOT a workaround today (but is the promising track)
|
|
|
+We are not vLLM-locked — sm121 has `engine = "sglang"` first-class. Checked
|
|
|
+SGLang (latest release **v0.5.21**, 2026-10-02):
|
|
|
+- **MuseSpark: not in a release, not in `main`** (`models/muse_spark.py` = 404;
|
|
|
+ only `muse_glimmer.py`). BUT the monolithic PR **#44530 is being landed as a
|
|
|
+ 5-layer series**: #44664 (Docker) **merged**, #44665 (kernels) **merged**
|
|
|
+ (`456231e2`), #44666 (srt model layer, base of 4/5), #44667 (ROCm, 4/5) open,
|
|
|
+ + a 5/5 CUDA/deploy layer. So the model layer is in-flight, not merged. A
|
|
|
+ SGLang **cookbook for Muse Spark Open** is already merged (#44654). → SGLang
|
|
|
+ will likely run Muse-816B **before** vLLM does (vLLM's only MuseSpark PR is
|
|
|
+ sm80+INT4). **Watch the #44530 series; retry SGLang once `muse_spark.py` lands
|
|
|
+ in a release.**
|
|
|
+- **Step-5: also not in SGLang** — SGLang has `step3_vl`, `step3p5`, `step3p7`
|
|
|
+ (Step3 family) but no Step5 arch. Same blocker as vLLM.
|
|
|
+- SGLang DOES have `bailing_moe_v3.py` (Ling/BailingMoeV3) — a real alternative
|
|
|
+ engine to A/B our Ling fp8/fp4 serve against vLLM (not a blocker, an option).
|
|
|
+
|
|
|
## Ling-3.1-flash fp8→fp4 (our own converter — in progress)
|
|
|
No official 3.1-flash fp4 exists (inclusionAI has fp4 only for 3.0). We wrote a
|
|
|
converter (fp8-block routed experts → MXFP4, "even" scale mode, byte-exact vs
|
