2026-10-12_201240_kindling4x-conc-30g.md 3.3 KB

Throughput Benchmark — glm53

  • Run ID: 2026-10-12T20-55-26.598600Z_ca4374d1
  • Date: 2026-10-12T20:55:26.598660+00:00
  • Mode: throughput-only
  • tool-eval-bench: v2.7.1.dev25+gc8a30ff5c

Run Context

Parameter Value
Backend vllm
Server http://***:30000/v1
Model (API) glm53
Model (Root) nvidia/GLM-5.3-Flash-NVFP4
Temperature 0.0
Seed 42
Max Turns 8
Timeout 120.0s
Scenarios all (92)
Parallel 1 (sequential)
Error Rate 0.0
Thinking enabled
Max Tokens 16384

Inference Engine

Property Value
Engine vLLM 0.30.1rc1.dev193+gddd6fbca1
Max Model Length 524,288
Quantization NVFP4
Host pluto
Platform Linux-7.2.9-arch1-1-x86_64-with-glibc2.44
Python 3.12.15

Results

Test pp t/s tg t/s TTFT (ms) Total (ms) Tokens
pp2048 tg128 @ d0 3,546 84.9 658 2,089 2048+128
pp2048 tg128 @ d0 c2 3,346 97.4 1,012 3,277 2048+128
pp2048 tg128 @ d0 c4 4,242 140.8 1,826 4,905 2048+128
pp2048 tg128 @ d0 c8 3,951 126.6 3,649 8,265 2048+128
pp2048 tg128 @ d0 c16 3,943 147.0 5,806 13,343 2048+128
pp2048 tg128 @ d0 c32 4,194 172.2 9,446 21,951 2048+128
pp2048 tg128 @ d0 c64 4,291 186.4 16,990 39,782 2048+128
pp4096 tg128 @ d4096 3,419 67.9 1,279 3,088 2048+128
pp2048 tg128 @ d4096 1,738 76.9 1,255 2,843 2048+128
pp4096 tg128 @ d4096 c2 3,188 68.0 1,922 4,734 2048+128
pp2048 tg128 @ d4096 c2 1,756 88.3 1,931 4,276 2048+128
pp4096 tg128 @ d4096 c4 4,082 104.5 3,463 7,308 2048+128
pp2048 tg128 @ d4096 c4 1,980 81.0 3,297 7,156 2048+128
pp4096 tg128 @ d4096 c8 4,180 86.3 7,042 12,321 2048+128
pp2048 tg128 @ d4096 c8 2,196 105.1 6,766 11,408 2048+128
pp4096 tg128 @ d4096 c16 4,242 96.1 10,977 20,570 2048+128
pp2048 tg128 @ d4096 c16 2,197 100.5 10,722 19,469 2048+128
pp4096 tg128 @ d4096 c32 4,349 105.5 18,622 35,257 2048+128
pp2048 tg128 @ d4096 c32 2,258 110.4 18,050 33,710 2048+128
pp4096 tg128 @ d4096 c64 4,287 112.0 34,084 64,140 2048+128
pp2048 tg128 @ d4096 c64 2,243 114.9 32,926 61,515 2048+128
pp8192 tg128 @ d8192 4,076 74.8 2,090 3,724 2048+128
pp2048 tg128 @ d8192 1,904 76.8 1,152 2,743 2048+128
pp8192 tg128 @ d8192 c2 3,456 50.5 3,421 6,567 2048+128
pp2048 tg128 @ d8192 c2 1,787 73.3 1,719 4,393 2048+128
pp8192 tg128 @ d8192 c4 4,075 63.9 6,716 10,911 2048+128
pp2048 tg128 @ d8192 c4 2,177 90.4 3,000 6,581 2048+128
pp8192 tg128 @ d8192 c8 4,352 56.3 13,462 18,669 2048+128
pp2048 tg128 @ d8192 c8 2,502 98.7 5,872 10,449 2048+128
pp8192 tg128 @ d8192 c16 4,320 57.5 21,726 33,436 2048+128
pp2048 tg128 @ d8192 c16 2,533 106.5 9,827 18,267 2048+128
pp8192 tg128 @ d8192 c32 4,328 59.8 37,157 59,159 2048+128
pp2048 tg128 @ d8192 c32 2,598 123.6 16,248 30,465 2048+128
pp8192 tg128 @ d8192 c64 4,299 63.0 67,226 107,282 2048+128
pp2048 tg128 @ d8192 c64 2,573 128.2 29,367 55,853 2048+128