When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving

雷达摘要

NVIDIA Dynamo implements encode-prefill-decode (EPD) disaggregation to separate vision encoding from LLM prefill and decode stages for multimodal inference. EPD disaggregation delivers up to 5x faster time to first token and 7x faster end-to-end response time for image-heavy prompts, short-to-medium outputs, and quantized mixture-of-experts models. Three en…