Selaa lähdekoodia

r36 recipe: 512K context (our floor; their 131K profile is not usable) + note the b12x vs instanttensor load-format finding (forum 381350 #521: -35% prefill on the b12x loader at TP4, and it cannot load a DFlash2 drafter)

Alexander Korolev 1 kuukausi sitten
vanhempi
sitoutus
6798a62b8a
1 muutettua tiedostoa jossa 3 lisäystä ja 2 poistoa
  1. 3 2
      recipes/dsv41-flash-jovian-r36-4x/recipe.toml

+ 3 - 2
recipes/dsv41-flash-jovian-r36-4x/recipe.toml

@@ -71,9 +71,10 @@ args = [
     # GB10 startup-gate value (see header); KV pool is engine-computed from
     # the remainder (no pin — their profile pins nothing either).
     "--gpu-memory-utilization", "0.88",
-    # Their qualified DS4.1 profile (RTX quartet, TP4/DCP1, 131K):
+    # 512K context (our floor — their qualified 131K is not usable for us;
+    # the KV pool is engine-computed from the memory budget, unchanged):
+    "--max-model-len", "524288",
     "--decode-context-parallel-size", "1",
-    "--max-model-len", "131072",
     "--max-num-seqs", "4",
     "--max-num-batched-tokens", "4096",
     "--prefill-compute-share", "0.4",