瀏覽代碼

fix: reduce gpu-memory-utilization to 0.75 for deepseek-v4-flash

From author: 0.8+ over-commits the ~121 GiB unified pool → OOM on GB10
Alexander Korolev 3 月之前
父節點
當前提交
ae1285dfdf
共有 1 個文件被更改,包括 1 次插入 和 1 次删除
  1. 1 1
      recipes/deepseek-v4-flash/recipe.toml

+ 1 - 1
recipes/deepseek-v4-flash/recipe.toml

@@ -23,7 +23,7 @@ args = [
     "--moe-backend", "flashinfer_cutlass",
     "--linear-backend", "b12x",
     "--enable-chunked-prefill",
-    "--gpu-memory-utilization", "0.85",
+    "--gpu-memory-utilization", "0.75",
     "--compilation-config", """
 '{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}'""",
     "--enable-prefix-caching",