Three optimizations raise Ling-3.0-flash throughput on DGX Spark

NVIDIA tested three combined techniques for mixture-of-experts inference on a single DGX Spark: Humming MXFP4 MoE kernels, an FP8 version of the language-model head, and built-in NEXTN speculative decoding. The test used Ant Group inclusionAI’s Ling-3.0-flash-fp4, a 124B-parameter model with 5.1B active parameters per token, running at batch size one on a system with 128 GB of unified memory.

On the 1,319-question GSM8K evaluation, decode throughput increased from 37.81 to 53.89 tokens per second, a gain of 42.5%. Mean latency fell from 15.21 to 11.49 seconds, while accuracy changed from 96.89% to 96.82%, which NVIDIA described as statistically undetectable. The model’s FP8 language-model head reduces weight traffic, while speculative decoding verifies several positions together.

Synthetic testing showed a larger 185.1% improvement, but accepted draft length declined from 3.90 to 2.68 on GSM8K. NVIDIA said the difference reflects less repetitive sampled outputs and shorter answers. The company also warned that FP8 and MTP must share the full language-model-head module; otherwise the framework can silently fall back and lose the performance benefit.