ricursiveAI/Ripple-65B-Preview-NVFP4 runs on one DGX Spark. The vendor ships a
hardware-locked SGLang path; vLLM support exists upstream but is not in any
release yet. Numbers below are measured on dgx4, not vendor claims.
| repo | ricursiveAI/Ripple-65B-Preview-NVFP4, public, Apache-2.0, created 04.11.2026 |
| parameters | 39.6 B dense (the "65B" name is marketing), 80 layers, hybrid attention 4 sliding : 1 full |
| size on disk | 53.28 GiB in 6 shards (byte-verified against HF's published sizes) |
| quantisation | MLP NVFP4 static group 16, attention/embed/lm_head BF16, BF16 KV cache |
| draft | ricursiveAI/Ripple-65B-Preview-DFlash2, 4.04 B, 7.6 GB, unquantised |
| licence note | HF tag says "8-bit"; that is wrong — read quantization_config |
inference/local/spark.py refuses any other device:
if not torch.cuda.is_available() or torch.cuda.get_device_capability() != (12, 1):
raise RuntimeError("Run this notebook on a DGX Spark / GB10 GPU")
It also patches SGLang internals for SM121: enable_pdl=False in the flashinfer
softmax ("SM121 PDL softmax can produce NaNs"), KV scales forced to 1.0 for
quantised exports with BF16 KV, and --mem-fraction-static 0.78. Code-writing
helpers (ripple_kernels.py, ripple_tree*.py, ripple_transport.py) must sit
next to spark.py.
python3 -m venv ~/venvs/ripple
~/venvs/ripple/bin/pip install --pre --extra-index-url https://download.pytorch.org/whl/cu130 \
"sglang==0.5.19" "torch==2.13.0+cu130" "transformers==5.12.1" \
"flashinfer-python==0.6.18" "ninja==1.13.2" "nest_asyncio==1.6.0"
# checkpoints (skip the notebook, it is not needed to serve)
~/venvs/ripple/bin/python -c "
from huggingface_hub import snapshot_download as s
s('ricursiveAI/Ripple-65B-Preview-NVFP4', local_dir='/home/kilork/models/Ripple-65B-Preview-NVFP4', ignore_patterns=['*.mp4','*.ipynb'])
s('ricursiveAI/Ripple-65B-Preview-DFlash2', local_dir='/home/kilork/models/Ripple-65B-Preview-DFlash2', ignore_patterns=['*.mp4','*.ipynb'])"
export CUDA_HOME=/usr/local/cuda-13.0
~/venvs/ripple/bin/python ~/models/Ripple-65B-Preview-NVFP4/inference/local/spark.py \
--target ~/models/Ripple-65B-Preview-NVFP4 \
--draft ~/models/Ripple-65B-Preview-DFlash2 \
--decoding tree --draft-tokens 32 \
--host 0.0.0.0 --port 8010
First start compiles flashinfer's fp4_gemm_cutlass for sm121 into
~/.cache/sglang/.cache/flashinfer/0.6.18/121a/cached_ops/ with two cicc
processes — about 10 minutes. Later starts load in ~35 s and capture CUDA graphs
in ~2 min.
Do not stop it with pkill -f "spark.py" from an ssh one-liner: the pattern
matches the ssh shell and kills the session. Use pkill -f "spark[.]py".
| decoding | code prompt (stream) | prose, 128 tok | 243-token prompt | TTFT |
|---|---|---|---|---|
| target only | 4.1 tok/s | 4.05 tok/s | 3.76 tok/s | 0.40 s |
| chain, 16 draft tokens | 18.7 tok/s | 6.8 tok/s | 7.25 tok/s | 0.50 s |
| tree, 32 nodes (vendor default) | 22.6 tok/s | 10.4 tok/s | 11.1 tok/s | 0.50 s |
4 concurrent requests, tree decoding: 39.9 tok/s aggregate (401 tokens in 10.05 s wall), 110 / 82 / 128 / 81 tokens per request.
Memory: 103 GiB of 121 GiB with target + draft loaded (--mem-fraction-static
0.78); the target alone is 98 GiB. Load time 35 s from NVMe (1.5 GB/s).
The spread between "code" and "prose" is acceptance rate, not a benchmark artefact: predictable text accepts 4–6 draft tokens per step, prose accepts far fewer.
vLLM implements the architecture natively — RicursiveForCausalLM and
DFlash2RicursiveDraftModel in vllm/model_executor/models/registry.py, with
NVFP4, TP/PP, torch.compile, CUDA graphs, --reasoning-parser ripple /
--tool-call-parser ripple and no --trust-remote-code. It merged 05.11.2026
(PR #65745), so it is not in 0.33.0 (02.11). Options when we want it:
vllm-0.34.0rc1.dev210+gbafa5c6c3-cp38-abi3-manylinux_2_28_aarch64.whl (229 MiB) from https://wheels.vllm.ai/nightly/vllm/;0.34.0.ModelOptNvFp4Config.get_min_capability() is 75, so sm_121 is not gated out. The
PR was validated on B300 only, so NVFP4 kernel selection on GB10 is the open
risk — flashinfer's sm121 cutlass FP4 path working under SGLang suggests it will
land on a real kernel rather than a marlin fallback.
--context-length), although the config claims
2 097 152 positions; SGLang corrected max_total_tokens down to its profiled
134 119 at load time.