Bladeren bron

glm53-kindling-4x: 30 GiB hot KV pin (measured headroom; pool 5,083,602 tok)

Alexander Korolev 1 week geleden
bovenliggende
commit
94ac882a41
1 gewijzigde bestanden met toevoegingen van 1 en 1 verwijderingen
  1. 1 1
      recipes/glm53-kindling-4x/recipe.toml

+ 1 - 1
recipes/glm53-kindling-4x/recipe.toml

@@ -49,7 +49,7 @@ args = [
     "--max-model-len", "524288",
     "--kv-cache-dtype", "fp8_e4m3",
     "--block-size", "2304",
-    "--kv-cache-memory", "27917287424",
+    "--kv-cache-memory", "32212254720",
     "--trust-remote-code",
     "--enable-prefix-caching",
     "--max-num-batched-tokens", "16384",