Răsfoiți Sursa

glm53-kindling-2x: perf-only re-baseline (v0.31.0 image, no label needed)

Alexander Korolev 1 săptămână în urmă
părinte
comite
71b7f321b0

+ 2 - 0
recipes/glm53-kindling-2x/reports/2026-10-15_085825_v031-2x.log

@@ -0,0 +1,2 @@
+^D{"event": "backend_detected", "backend": "vllm"}
+{"event": "error", "error": "run_failed", "message": "llama-benchy error: llama-benchy could not load a tokenizer. llama-benchy needs a tokenizer to construct prompts, and this host is running in offline mode with no tokenizer in the HuggingFace cache. Fixes: - Pass --tokenizer /path/to/tokenizer.json (or a directory containing tokenizer.json) to use a local tokenizer. - Or fetch just the tokenizer once with network access: hf download <org>/<model> --include \"tokenizer*\" \"*config.json\""}

+ 3 - 0
recipes/glm53-kindling-2x/reports/2026-10-15_105249_v031-2x-perf.log

@@ -0,0 +1,3 @@
+Pseudo-terminal will not be allocated because stdin is not a terminal.
+{"event": "backend_detected", "backend": "vllm"}
+{"event": "run_saved", "run_type": "perf", "run_id": "2026-10-15T11-00-30.457153Z_4ddbdfba", "status": "completed", "report_path": "/tmp/sm121-bench-2026-10-15_105249_v031-2x-perf/runs/2026/10/2026-10-15T11-00-30.457153Z_4ddbdfba.md"}

+ 38 - 0
recipes/glm53-kindling-2x/reports/2026-10-15_105249_v031-2x-perf.md

@@ -0,0 +1,38 @@
+# Throughput Benchmark — glm53
+
+- **Run ID**: `2026-10-15T11-00-30.457153Z_4ddbdfba`
+- **Date**: `2026-10-15T11:00:30.457229+00:00`
+- **Mode**: throughput-only
+- **tool-eval-bench**: `v2.8.0`
+- **Backend**: vllm
+- **Server**: http://***:30000/v1
+- **Model (API)**: `glm53`
+- **Model (Root)**: `nvidia/GLM-5.3-Flash-NVFP4`
+
+## Inference Engine
+
+| Property | Value |
+|---|---|
+| Engine | vLLM 0.31.0 |
+| Max Model Length | 524,288 |
+| Quantization | NVFP4 |
+| Host | `pluto` |
+| Platform | `Linux-7.2.9-arch1-1-x86_64-with-glibc2.44` |
+| Python | 3.12.15 |
+
+## Results
+
+| Test | pp t/s | tg t/s | TTFT (ms) | Total (ms) | Tokens |
+|---|---:|---:|---:|---:|---:|
+| pp2048 tg128 @ d0 | 675* | 39.6 | 3,034 | 6,242 | 2048+128 |
+| pp2048 tg128 @ d0 c2 | 1,369 | 68.5 | 2,843 | 6,323 | 2048+128 |
+| ctx pp4096 tg128 @ d4096 | 0 | 47.0 | 1,606 | 4,308 | 2048+128 |
+| pp2048 tg128 @ d4096 | 0 | 44.6 | 2,603 | 5,453 | 2048+128 |
+| ctx pp4096 tg128 @ d4096 c2 | 2,548 | 66.5 | 3,226 | 6,976 | 2048+127.9 |
+| pp2048 tg128 @ d4096 c2 | 572 | 37.9 | 6,082 | 11,204 | 2048+127.9 |
+| ctx pp8192 tg128 @ d8192 | 0 | 34.4 | 3,171 | 6,853 | 2048+127.7 |
+| pp2048 tg128 @ d8192 | 0 | 54.3 | 4,261 | 6,594 | 2048+127.7 |
+| ctx pp8192 tg128 @ d8192 c2 | 1,968 | 43.3 | 6,863 | 11,358 | 2048+127.8 |
+| pp2048 tg128 @ d8192 c2 | 757 | 43.4 | 4,588 | 9,189 | 2048+127.8 |
+
+\* prefill estimated from e2e_ttft over the tokens counted by this benchmark phase. llama-benchy saw a response chunk before the first content token, so est_ppt can measure that early chunk instead of prefill (eugr/llama-benchy#33). The estimate includes network and queue time.