|
@@ -5,9 +5,15 @@ served by sm121 through the launcher, same benchmark invocation:
|
|
|
|
|
|
|
|
```
|
|
```
|
|
|
tool-eval-bench run --base-url http://<head>:30000/v1 --model glm53 \
|
|
tool-eval-bench run --base-url http://<head>:30000/v1 --model glm53 \
|
|
|
- --seed 42 --hardmode --perf --concurrency 1,2
|
|
|
|
|
|
|
+ --seed 42 --hardmode --perf --concurrency 1,2 \
|
|
|
|
|
+ --json-file <abs repo path>/recipes/<lane>/reports/<UTC-stamp>_<label>.json
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
|
|
+Keep BOTH the rich `.md` report (copied from the tool's `runs/` output dir)
|
|
|
|
|
+and the raw JSON (`--json-file`; absolute path — the bench runs in a separate
|
|
|
|
|
+cwd, and the flag keeps stdout clean for the `.log`) in
|
|
|
|
|
+`recipes/<lane>/reports/` under the same base name.
|
|
|
|
|
+
|
|
|
**Tool version pin (2026-09-05, verified against the installed tool):**
|
|
**Tool version pin (2026-09-05, verified against the installed tool):**
|
|
|
tool-eval-bench 2.6.1.dev45+gcf54b4bfe (uv tool; git
|
|
tool-eval-bench 2.6.1.dev45+gcf54b4bfe (uv tool; git
|
|
|
SeraphimSerapis/tool-eval-bench @ cf54b4bfe). Current pool semantics:
|
|
SeraphimSerapis/tool-eval-bench @ cf54b4bfe). Current pool semantics:
|