|
@@ -943,3 +943,17 @@ r27 watch; aiden 159 as the community reference).
|
|
|
- **Eager A/B (our r18 2x lane + --enforce-eager): 159 vs 158** — the eager
|
|
- **Eager A/B (our r18 2x lane + --enforce-eager): 159 vs 158** — the eager
|
|
|
quality rumor does not hold on this stack (+1 = noise); perf at parity
|
|
quality rumor does not hold on this stack (+1 = noise); perf at parity
|
|
|
(tg c1 27.1 vs 24.5, TTFT 1.17 vs 1.24 s). Graphs remain the default.
|
|
(tg c1 27.1 vs 24.5, TTFT 1.17 vs 1.24 s). Graphs remain the default.
|
|
|
|
|
+
|
|
|
|
|
+## Rust frontend (vllm-frontend-rs) — evaluated, deferred (2026-09-07)
|
|
|
|
|
+
|
|
|
|
|
+The r27 tree ships the Rust OpenAI frontend (axum, ZMQ bridge; `vllm-rs`
|
|
|
|
|
+binary is already in our r27 image, built with the wheel; toggle
|
|
|
|
|
+`VLLM_USE_RUST_FRONTEND=1`). Smoke test (dgx5, Qwen3.8-27B, 1 node):
|
|
|
|
|
+fronted boots but fails `engine input registration timed out after 600s`
|
|
|
|
|
+— the Rust client's fixed ready_timeout is shorter than our GB10 boots
|
|
|
|
|
+(NAS weights + JIT + cudagraphs = 10–40 min). Verdict (operator): not
|
|
|
|
|
+pursuing — the fork stays selective about upstream imports and the
|
|
|
|
|
+frontend is upstream-experimental (not feature-complete; parser parity
|
|
|
|
|
+for glm45/glm47 tool/reasoning unverified). Revisit only if upstream
|
|
|
|
|
+matures it (or if per-request CPU overhead ever becomes a bottleneck in
|
|
|
|
|
+our benches).
|
