nemotron-35-bolt-74b-24gb: switch to the CARD's image path — docker-images/nemotron-bolt-b12x (thin FROM the card's stock vLLM nightly 77154ca by arm64 digest + b12x 1.5.1 + ultra_v3 parser plugin baked at /opt); recipe flags verbatim from the card incl. LiLiCorr dflash spec-decode (7) + ultra_v3 parser; only deviation GMU 0.92→0.90 (clean Spark reports 111.55/121.63 GiB free; 0.92 misses the boot gate by 0.35). eugr/spark-vllm-b12x (03a95654) does NOT carry LiLiCorrDraftModel/ultra_v3 — it cannot serve Bolt with the card's flags. Verified live: LiLiCorr speculator captured, smoke OK.

docker-images/nemotron-bolt-b12x/Dockerfile

@@ -0,0 +1,41 @@
+# syntax=docker/dockerfile:1.7
+# Nemotron-3.5-Bolt-74B-A9B-24GB serving image (1x DGX Spark).
+#
+# The model card's Quick Start pins the STOCK vLLM nightly at commit
+# 77154ca6116ae34cbfe314beef1016bde9b63989 (2026-10-14) — the first build
+# that registers LiLiCorrDraftModel (the DFlash LiLiCorr speculator) and the
+# ultra_v3 reasoning parser. eugr/spark-vllm-b12x is a DIFFERENT vLLM fork
+# (commit 8244d1c44) that does NOT carry those, so it cannot serve Bolt with
+# the card's flags. This wrapper reproduces the card exactly: the card's
+# nightly base + `pip install b12x` + `vllm download-kernels`, baked once so
+# every serve is reproducible and needs no runtime pip/network.
+#
+# Pinned by arm64 digest (nightly tags expire); see versions.env.
+
+ARG BASE_IMAGE
+FROM ${BASE_IMAGE}
+
+ARG B12X_VERSION
+
+# The card's runtime setup, baked: b12x wheel + pre-downloaded vLLM kernels.
+# B12X_CACHE_DIR keeps the JIT-compiled native + autotuned kernels per node
+# (the recipe bind-mounts a host dir there, like the other JIT caches).
+ENV B12X_CACHE_DIR=/root/.cache/b12x
+
+# The card's ultra_v3 reasoning-parser plugin (shipped in the model repo).
+# Baked so the serve needs no network; referenced by absolute path from the
+# recipe (--reasoning-parser-plugin /opt/ultra_v3_reasoning_parser.py).
+RUN set -eux; \
+    curl -fsSL -o /opt/ultra_v3_reasoning_parser.py \
+      "https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB/resolve/main/ultra_v3_reasoning_parser.py"; \
+    test -s /opt/ultra_v3_reasoning_parser.py
+
+RUN set -eux; \
+    pip install --no-cache-dir "b12x==${B12X_VERSION}"; \
+    python3 -c "import b12x; print('b12x import OK')"; \
+    vllm download-kernels || echo "vllm download-kernels: nothing to fetch / already present"
+# NOTE: we do NOT pre-build the b12x native here. It links -lcuda (the CUDA
+# *driver* lib), which the nvidia container runtime only mounts at RUNTIME
+# (--gpus all) — it is absent in the build image, so a build-time `load()`
+# fails with `ld: cannot find -lcuda`. b12x JIT-builds the native on the first
+# serve instead (libcuda present then); B12X_CACHE_DIR persists it per node.

docker-images/nemotron-bolt-b12x/build.sh

@@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+# Build the Nemotron-3.5-Bolt B12X wrapper (thin FROM over the model card's
+# pinned stock vLLM nightly).
+#
+# Uses the classic docker builder (not buildx docker-container): the base is a
+# registry image pulled by digest, which the classic builder resolves fine, and
+# this keeps the build identical to the other b12x wrappers in this repo.
+#
+# Generic driver: forwards every KEY=VAL pin in versions.env as a
+# --build-arg to the Dockerfile (no hardcoded variable list). The sm121
+# launcher writes a merged versions.env here when a recipe carries
+# [build.versions] overrides (recipe wins over the image defaults).
+set -euo pipefail
+cd "$(dirname "$0")"
+TAG="${TAG:-nemotron-bolt-b12x:local}"
+
+ARGS=()
+while IFS= read -r line; do
+    # Skip blank lines and comments (# ...).
+    case "$line" in
+        ''|\#*) continue ;;
+    esac
+    key="${line%%=*}"
+    val="${line#*=}"
+    [ -n "$key" ] && ARGS+=(--build-arg "${key}=${val}")
+done < versions.env
+
+echo "=== Building ${TAG} ==="
+docker build "${ARGS[@]}" --progress=plain -t "${TAG}" -f Dockerfile . 2>&1
+echo "=== Done: ${TAG} ==="

docker-images/nemotron-bolt-b12x/versions.env

@@ -0,0 +1,15 @@
+# versions.env - build input for the Nemotron-3.5-Bolt B12X wrapper.
+# Thin FROM wrapper over the model card's pinned STOCK vLLM nightly, adding
+# the b12x kernel package (the card's `pip install b12x` step, baked once).
+#
+# The card's Quick Start pins vllm/vllm-openai:nightly-77154ca6116ae34cbfe314beef1016bde9b63989
+# (2026-10-14) — the first vLLM build that registers LiLiCorrDraftModel (the
+# DFlash LiLiCorr speculator) and the ultra_v3 reasoning parser. NOTE:
+# eugr/spark-vllm-b12x is a DIFFERENT vLLM fork (commit 8244d1c44) that does
+# NOT carry those, so it cannot serve Bolt with the card's flags.
+#
+# We pin the base by arm64 digest (nightly tags have a TTL). arm64 digest of
+# that nightly tag: sha256:1ee85ca80e0fc110017d79ea2081a0cbef9a208a05d7f22724801bcf6a956620
+# Convention: KEY=VAL, no spaces around =.
+BASE_IMAGE=vllm/vllm-openai@sha256:1ee85ca80e0fc110017d79ea2081a0cbef9a208a05d7f22724801bcf6a956620
+B12X_VERSION=1.5.1

recipes/nemotron-35-bolt-74b-24gb/recipe.toml

@@ -1,24 +1,27 @@
 # NVIDIA Nemotron-3.5-Bolt-74B-A9B-24GB — single DGX Spark, VL, experts
 # quantized to 2-bit via QAD (quantization-aware distillation) to fit GB10.
-# Source: NVIDIA forum thread 385978 + model card
-# https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB (DGX Spark
-# section). 24 GB → one Spark (the "star" for Sparks without 128 GB RAM).
-# tool-eval-bench 88/100.
+# Source: model card Quick Start (authoritative) + NVIDIA forum thread 385978.
+# https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB
+# 24 GB → one Spark. Card tool-eval-bench anchor: 88/100.
 #
-# Image: eugr/spark-vllm-b12x (vLLM nightly main + b12x, sm_121), digest-pinned
-# (arm64, built 2026-10-15). b12x linear/MoE
-# backends + flashinfer mamba. LiLiCorr drafter (dflash, 7) is a SECOND HF repo →
-# resolved via the repo-ref ({{lilicorr}}).
+# IMAGE: the card pins the STOCK vLLM nightly at commit
+# 77154ca6116ae34cbfe314beef1016bde9b63989 (2026-10-14) — the first vLLM build
+# that registers LiLiCorrDraftModel (the DFlash LiLiCorr speculator) and the
+# ultra_v3 reasoning parser — then `pip install b12x` + `vllm download-kernels`.
+# We bake that into docker-images/nemotron-bolt-b12x/ (thin FROM the card's
+# nightly by arm64 digest + b12x 1.5.1 + the ultra_v3 plugin), so sm121 builds
+# it from zero and every serve is reproducible.
 #
-# CAVEAT: the card uses a reasoning-parser PLUGIN file `ultra_v3_reasoning_parser.py`
-# (ultra_v3 is NOT built into vLLM). The file ships in the model repo; vLLM
-# resolves it relative to the model dir under --trust-remote-code. If the plugin
-# path does not resolve at launch, point --reasoning-parser-plugin at the
-# snapshot's absolute path (or drop the parser for a text-only smoke).
+# ⚠ eugr/spark-vllm-b12x is a DIFFERENT vLLM fork (commit 8244d1c44) that does
+# NOT carry LiLiCorrDraftModel or the ultra_v3 plugin — it CANNOT serve Bolt
+# with the card's flags (it aborts at SpeculativeConfig validation and on the
+# missing parser). Do not "fix" that by disabling spec-decode/the parser; use
+# this image instead.
 
 engine = "vllm"
 name = "Nemotron-3.5-Bolt-74B-A9B-24GB (1x Spark, QAD 2-bit, LiLiCorr)"
-docker_image = "eugr/spark-vllm-b12x@sha256:03a956544e9f9b4b7ba91690469fbe7848bdbef29a7d2e0f5788368467bb3595"
+docker_image = "nemotron-bolt-b12x:local"
+docker_image_dir = "nemotron-bolt-b12x"
 entrypoint = ["vllm", "serve"]
 model = "nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB"
 parent_model = "nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B"
@@ -29,21 +32,40 @@ tensor_parallel_size = 1
 args = [
     "--served-model-name", "nemotron-3.5-bolt",
     "--trust-remote-code",
+    "--load-format", "auto",
     "--max-model-len", "262144",
     "--max-num-seqs", "1",
+    # GMU: the card relies on vLLM's default 0.92. On THIS Spark a clean node
+    # reports only ~111.5 GiB free (GB10 reserves ~10 GiB of the 121.63 GiB
+    # unified pool), so 0.92 (111.9 GiB) misses by a hair at the startup memory
+    # gate. 0.90 is the minimal fit — the only deviation from the card; every
+    # other flag is verbatim. (Bolt is 24 GB, so KV headroom is huge either
+    # way — this only clears the boot gate, not a capacity limit.)
+    "--gpu-memory-utilization", "0.90",
     "--enable-chunked-prefill",
     "--kv-cache-dtype", "fp8",
     "--linear-backend", "b12x",
     "--moe-backend", "b12x",
+    "--attention-backend", "FLASHINFER",
     "--mamba-backend", "flashinfer",
     "--mamba-ssm-cache-dtype", "bfloat16",
     "--mamba-cache-dtype", "bfloat16",
     "--mamba-cache-mode", "align",
+    "--seed", "42",
+    "--allowed-local-media-path", "/",
     "--limit-mm-per-prompt", '{"video":1,"image":1}',
+    "--video-pruning-rate", "0.5",
+    "--media-io-kwargs", '{"video":{"fps":2,"num_frames":256}}',
     "--enable-auto-tool-choice",
     "--tool-call-parser", "qwen3_coder",
-    "--reasoning-parser-plugin", "ultra_v3_reasoning_parser.py",
+    # ultra_v3 is NOT built into vLLM — it is a plugin file shipped in the
+    # model repo. We bake it into the image at /opt (see the Dockerfile), so
+    # the serve needs no network. The card wgets it and passes the bare name.
+    "--reasoning-parser-plugin", "/opt/ultra_v3_reasoning_parser.py",
     "--reasoning-parser", "ultra_v3",
+    # LiLiCorr DFlash speculator (7 tokens) — the card's headline speed lever.
+    # The drafter is a SECOND HF repo, resolved per node from the HF cache via
+    # the repo-ref below ({{lilicorr}} → container-side snapshot path).
     "--speculative-config", """
 '{"method":"dflash","model":"{{lilicorr}}","num_speculative_tokens":7,"rejection_sample_method":"standard","attention_backend":"FLASHINFER","draft_load_config":{"load_format":"auto"}}'""",
 ]
@@ -53,5 +75,12 @@ args = [
 repo = "nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr"
 container = "lilicorr"
 
+# b12x JIT/kernel cache (per node) — the image sets B12X_CACHE_DIR to this
+# container path; the host dir persists autotuned kernels across serves.
+[[extra_mounts]]
+host = "~/.cache/b12x"
+container = "/root/.cache/b12x"
+
 [env]
+VLLM_USE_V2_MODEL_RUNNER = "1"
 VLLM_ENGINE_READY_TIMEOUT_S = "3600"