Decode · tok/s
65
Prefill
—
tok/100W
27.1
NVIDIA's Qwen3.8-Flash-Next on Spark 1m context 1 million tokens read from SSD (on the Spark) into KV-cache, TTFT just 8.5 seconds (5.5 directly from the cache) and then 65 tok/s! (YaRN setting factor 4.0 over 262,144)
First-person DGX Spark. Rank is the quoted 65 tok/s after loading 1M into KV. Quant is NVIDIA FP4-Local. Runtime not named beyond the official stack. YaRN 4.0.
HF FP4-Local