Bladeren bron

refactor(recipes): quant-agnostic served-model-name + rename glm53 to -nvfp4

Align day-0 recipes with the repo convention that quant variants of one model
share a single served name, so swapping quants never changes what agents/
clients pin against:

- rename recipe glm53-flash -> glm53-flash-nvfp4 (matches qwen38-flash-next-*)
- glm53-flash-nvfp4: --served-model-name glm53
- qwen38-flash-next-fp8: --served-model-name qwen3.8-flash-next
- qwen38-flash-next-nvfp4: Qwen3.8-Flash-Next-NVFP4 -> qwen3.8-flash-next
  (was encoding the quant; both quants now serve the same stable name)
Alexander Korolev 1 maand geleden
bovenliggende
commit
095dec1d73

+ 2 - 0
recipes/glm53-flash/recipe.toml → recipes/glm53-flash-nvfp4/recipe.toml

@@ -58,6 +58,8 @@ shm_size = "32gb"
 entrypoint = ["vllm", "serve"]
 
 args = [
+    # Stable served name — quant-agnostic (shared across glm53 quants)
+    "--served-model-name", "glm53",
     # GLM tokenizer
     "--trust-remote-code",
     # Executor (mp, no ray)

+ 2 - 0
recipes/qwen38-flash-next-fp8/recipe.toml

@@ -36,6 +36,8 @@ shm_size = "32gb"
 entrypoint = ["vllm", "serve"]
 
 args = [
+    # Stable served name — quant-agnostic, shared with qwen38-flash-next-nvfp4
+    "--served-model-name", "qwen3.8-flash-next",
     # Expert parallelism — the fix for the FP8 MoE block-divisibility crash
     "--distributed-executor-backend", "mp",
     "--enable-expert-parallel",

+ 3 - 1
recipes/qwen38-flash-next-nvfp4/recipe.toml

@@ -40,7 +40,9 @@ shm_size = "32gb"
 entrypoint = ["python3", "-m", "sglang.launch_server"]
 
 args = [
-    "--served-model-name", "Qwen3.8-Flash-Next-NVFP4",
+    # Stable served name — quant-agnostic, shared with qwen38-flash-next-fp8 so
+    # swapping quants never changes what agents/clients pin against.
+    "--served-model-name", "qwen3.8-flash-next",
     "--trust-remote-code",
     # NVFP4 (ModelOpt) weights
     "--quantization", "modelopt_fp4",