|
|
@@ -0,0 +1,36 @@
|
|
|
+# Throughput Benchmark — glm53
|
|
|
+
|
|
|
+- **Run ID**: `2026-10-15T11-34-50.926324Z_150314ac`
|
|
|
+- **Date**: `2026-10-15T11:34:50.926390+00:00`
|
|
|
+- **Mode**: throughput-only
|
|
|
+- **tool-eval-bench**: `v2.8.0`
|
|
|
+- **Backend**: vllm
|
|
|
+- **Server**: http://***:30000/v1
|
|
|
+- **Model (API)**: `glm53`
|
|
|
+- **Model (Root)**: `nvidia/GLM-5.3-Flash-NVFP4`
|
|
|
+
|
|
|
+## Inference Engine
|
|
|
+
|
|
|
+| Property | Value |
|
|
|
+|---|---|
|
|
|
+| Engine | vLLM 0.31.0 |
|
|
|
+| Max Model Length | 524,288 |
|
|
|
+| Quantization | NVFP4 |
|
|
|
+| Host | `pluto` |
|
|
|
+| Platform | `Linux-7.2.9-arch1-1-x86_64-with-glibc2.44` |
|
|
|
+| Python | 3.12.15 |
|
|
|
+
|
|
|
+## Results
|
|
|
+
|
|
|
+| Test | pp t/s | tg t/s | TTFT (ms) | Total (ms) | Tokens |
|
|
|
+|---|---:|---:|---:|---:|---:|
|
|
|
+| pp2048 tg128 @ d0 | 2,303 | 51.4 | 1,013 | 3,484 | 2048+128 |
|
|
|
+| pp2048 tg128 @ d0 c2 | 1,687 | 48.3 | 3,501 | 7,864 | 2048+127.8 |
|
|
|
+| ctx pp4096 tg128 @ d4096 | 2,746 | 43.8 | 1,616 | 4,518 | 2048+128 |
|
|
|
+| pp2048 tg128 @ d4096 | 613 | 51.7 | 5,810 | 8,266 | 2048+128 |
|
|
|
+| ctx pp4096 tg128 @ d4096 c2 | 2,591 | 59.5 | 3,172 | 7,346 | 2048+128 |
|
|
|
+| pp2048 tg128 @ d4096 c2 | 699 | 47.1 | 5,404 | 10,165 | 2048+128 |
|
|
|
+| ctx pp8192 tg128 @ d8192 | 2,406 | 45.0 | 3,625 | 6,446 | 2048+128 |
|
|
|
+| pp2048 tg128 @ d8192 | 865 | 56.2 | 2,486 | 4,746 | 2048+128 |
|
|
|
+| ctx pp8192 tg128 @ d8192 c2 | 2,037 | 35.2 | 6,556 | 11,995 | 2048+127.9 |
|
|
|
+| pp2048 tg128 @ d8192 c2 | 761 | 36.5 | 3,930 | 8,789 | 2048+127.9 |
|