Throughput Benchmark — deepseek-v4.1-flash

Run ID: 2026-09-26T02-40-26.026891Z_fb22e087
Date: 2026-09-26T02:40:26.027390+00:00
Mode: throughput-only
tool-eval-bench: v2.7.0

Run Context

Parameter	Value
Backend	sglang
Server	`http://***:30000/v1`
Model (API)	`deepseek-v4.1-flash`
Temperature	0.0
Seed	42
Max Turns	8
Timeout	120.0s
Scenarios	all (88)
Parallel	1 (sequential)
Error Rate	0.0
Thinking	enabled
Max Tokens	16384

Inference Engine

Property	Value
Engine	SGLang
Max Model Length	524,288
Host	`localhost`
Platform	`macOS-26.6.2-arm64-arm-64bit-Mach-O`
Python	3.14.7

Results

Test	pp t/s	tg t/s	TTFT (ms)	Total (ms)	Tokens
pp2048 tg128 @ d0	189	20.2	10,102	16,258	2048+128
pp2048 tg128 @ d0 c2	208	28.5	18,290	26,115	2048+128
pp2048 tg128 @ d4096	206	19.4	26,615	33,038	2048+128
pp2048 tg128 @ d4096 c2	319	14.6	30,911	41,557	2048+128
pp2048 tg128 @ d8192	277	28.5	33,738	38,061	2048+128
pp2048 tg128 @ d8192 c2	359	12.6	45,246	57,719	2048+128