Nunchux AI unveils low-bit attention kernel for video diffusion models
Nunchux AI has introduced VC-Attention, a training-free low-bit attention kernel for video Diffusion Transformers. The method addresses value quantization errors and the FP32 softmax stage, two remaining costs after low-bit Tensor Cores accelerate the QK and PV matrix products. The company says attention accounts for more than 64% of generation time for a 5-second 720p Wan2.2-14B clip on an RTX 5090.
VC-Attention combines V-Smooth with ExpCast-FP8. V-Smooth groups value tokens online, subtracts the mean from 128-token blocks, and quantizes only the residual using E4M3 at 8 bits or NVFP4 at 4 bits. ExpCast-FP8 replaces the FP32 exponent-and-cast path with a fused multiply-add for the 8-bit kernel. Across 204.8K Wan2.2 attention rows, the reported average total-variation error is 1.6%.
Tests on four open-weight video DiTs reported attention speedups of 1.59× on a B200 with 8-bit precision and 3.58× on an RTX 5090 with 4-bit precision. End-to-end speedups were 1.19× and 1.70%, respectively. At 8 bits, V-Smooth improved PSNR over SageAttention2 on Wan2.2 and HunyuanVideo-1.5. No public kernel release is available; Nunchux runs a proprietary extension in its stack.
Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.