|
|
@@ -0,0 +1,38 @@
|
|
|
+# Throughput Benchmark — glm53
|
|
|
+
|
|
|
+- **Run ID**: `2026-10-15T11-00-30.457153Z_4ddbdfba`
|
|
|
+- **Date**: `2026-10-15T11:00:30.457229+00:00`
|
|
|
+- **Mode**: throughput-only
|
|
|
+- **tool-eval-bench**: `v2.8.0`
|
|
|
+- **Backend**: vllm
|
|
|
+- **Server**: http://***:30000/v1
|
|
|
+- **Model (API)**: `glm53`
|
|
|
+- **Model (Root)**: `nvidia/GLM-5.3-Flash-NVFP4`
|
|
|
+
|
|
|
+## Inference Engine
|
|
|
+
|
|
|
+| Property | Value |
|
|
|
+|---|---|
|
|
|
+| Engine | vLLM 0.31.0 |
|
|
|
+| Max Model Length | 524,288 |
|
|
|
+| Quantization | NVFP4 |
|
|
|
+| Host | `pluto` |
|
|
|
+| Platform | `Linux-7.2.9-arch1-1-x86_64-with-glibc2.44` |
|
|
|
+| Python | 3.12.15 |
|
|
|
+
|
|
|
+## Results
|
|
|
+
|
|
|
+| Test | pp t/s | tg t/s | TTFT (ms) | Total (ms) | Tokens |
|
|
|
+|---|---:|---:|---:|---:|---:|
|
|
|
+| pp2048 tg128 @ d0 | 675* | 39.6 | 3,034 | 6,242 | 2048+128 |
|
|
|
+| pp2048 tg128 @ d0 c2 | 1,369 | 68.5 | 2,843 | 6,323 | 2048+128 |
|
|
|
+| ctx pp4096 tg128 @ d4096 | 0 | 47.0 | 1,606 | 4,308 | 2048+128 |
|
|
|
+| pp2048 tg128 @ d4096 | 0 | 44.6 | 2,603 | 5,453 | 2048+128 |
|
|
|
+| ctx pp4096 tg128 @ d4096 c2 | 2,548 | 66.5 | 3,226 | 6,976 | 2048+127.9 |
|
|
|
+| pp2048 tg128 @ d4096 c2 | 572 | 37.9 | 6,082 | 11,204 | 2048+127.9 |
|
|
|
+| ctx pp8192 tg128 @ d8192 | 0 | 34.4 | 3,171 | 6,853 | 2048+127.7 |
|
|
|
+| pp2048 tg128 @ d8192 | 0 | 54.3 | 4,261 | 6,594 | 2048+127.7 |
|
|
|
+| ctx pp8192 tg128 @ d8192 c2 | 1,968 | 43.3 | 6,863 | 11,358 | 2048+127.8 |
|
|
|
+| pp2048 tg128 @ d8192 c2 | 757 | 43.4 | 4,588 | 9,189 | 2048+127.8 |
|
|
|
+
|
|
|
+\* prefill estimated from e2e_ttft over the tokens counted by this benchmark phase. llama-benchy saw a response chunk before the first content token, so est_ppt can measure that early chunk instead of prefill (eugr/llama-benchy#33). The estimate includes network and queue time.
|