|
|
@@ -150,4 +150,24 @@ measurable turn time here and costs a point. Lane stays at 100.
|
|
|
- The 2x alternative: thread 386121 / `x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark`
|
|
|
(TensorFold + vLLM on EXL3 2.9bpw, decode c1 59-103 tok/s on two Sparks) is a
|
|
|
different checkpoint + engine; a dgx1+dgx2 (`2nodes-dac`) lane is the natural
|
|
|
- place to try it.
|
|
|
+ place to try it.
|
|
|
+## The TensorFold 2x alternative (measured 2026-10-19)
|
|
|
+
|
|
|
+Upstream `x00byte/DeepSeek-V4.1-Flash-1017-Dual-Spark` (forum 386121) on
|
|
|
+`2nodes-dac` (dgx1 head + dgx2 worker), EXL3 2.9bpw weights on node-local disk,
|
|
|
+TensorFold 0.6.0 + Jay Leaton patches, 262144 context, thinking on / effort 75.
|
|
|
+Bring-up detail + artifacts: `recipes/dsv41-flash-1017-2x-tensorfold/NOTE.md`.
|
|
|
+
|
|
|
+| metric | TensorFold 2x | 4x SGLang (EP2/1M) |
|
|
|
+|---|---|---|
|
|
|
+| quality (v2.8.0, seed 42, hardmode) | **89/100** | 87/100 |
|
|
|
+| responsiveness / median turn | **23 / 6.7 s** | **75 / 1.4 s** |
|
|
|
+| decode c1 (probe median) | **50.4 tok/s** | 38.8-43.0 tok/s |
|
|
|
+| prefill for the same ~2K prompt | 5.4 s (~380 tok/s) | 0.75 s (~2,700 tok/s) |
|
|
|
+| nodes | 2 | 4 |
|
|
|
+
|
|
|
+So the trade is explicit: **half the cluster, better decode and no quality loss,
|
|
|
+at 4-5x the turn latency** (prefill is ~7x slower — the harness scores it as
|
|
|
+responsiveness 23). TensorFold 2x is the long-generation / batch shape; the 4x
|
|
|
+SGLang lane remains the interactive-agent shape. Not purity-matched: thinking
|
|
|
+effort 75 vs 100, EXL3 2.9bpw vs fp8.
|
