Explorar o código

ling recipe: pointer to the fp4 smaller-cut path (~110GB -> one Spark) + ling3 parser

Alexander Korolev hai 1 semana
pai
achega
81eb20a763
Modificáronse 1 ficheiros con 6 adicións e 0 borrados
  1. 6 0
      recipes/ling-31-flash-nvfp4/recipe.toml

+ 6 - 0
recipes/ling-31-flash-nvfp4/recipe.toml

@@ -23,6 +23,12 @@
 # <bailing parser>` (else tool scenarios 400 on tool_choice=auto) and thinking
 # control (`--no-think` in the harness, or the model's long default reasoning
 # blows the 120 s/turn timeout → 94/97 scenarios excluded). Follow-up.
+#
+# Smaller-cut path (fp4): the parser is `ling3` (vLLM v0.31.0 Ling3ToolParser);
+# a 3.1-flash fp4 (MXFP4 experts + fp8 dense) would be ~110 GB → ONE Spark
+# (vs 534 GB fp8 → 8). No official 3.1 fp4 exists; the fp8→mxfp4-experts
+# converter path + format spec are in
+# specs/tech-architecture/upstream-watch-2026-10-16.md ("making 3.1-flash smaller").
 
 engine = "vllm"
 name = "Ling-3.1-flash fp8 (8x, stock v0.31.0)"