Преглед изворни кода

fix: reduce gpu-memory-utilization to 0.75 for deepseek-v4-flash

From author: 0.8+ over-commits the ~121 GiB unified pool → OOM on GB10
Alexander Korolev пре 3 месеци
родитељ
комит
ae1285dfdf
1 измењених фајлова са 1 додато и 1 уклоњено
  1. 1 1
      recipes/deepseek-v4-flash/recipe.toml

+ 1 - 1
recipes/deepseek-v4-flash/recipe.toml

@@ -23,7 +23,7 @@ args = [
     "--moe-backend", "flashinfer_cutlass",
     "--linear-backend", "b12x",
     "--enable-chunked-prefill",
-    "--gpu-memory-utilization", "0.85",
+    "--gpu-memory-utilization", "0.75",
     "--compilation-config", """
 '{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}'""",
     "--enable-prefix-caching",