Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer

雷达摘要

NVIDIA Nemotron 3.5 Lightning achieves up to 4x faster throughput with the NVFP4 checkpoint compressed to 22 GB from the 66 GB full-precision version. Quantization-aware distillation (QAD) recovers accuracy lost during aggressive post-training quantization, enabling W4A16 quantization of Mamba linear layers while preserving model quality. The two-stage QAD…