Omni-Flow: A Unified Workflow Orchestration and Distributed KV Cache Sharing Framework for Multimodal Inference

Bin Xiao, Jingfu Dong, Changran Wang +5

Multimodal models increasingly integrate heterogeneous components, from encoders and LLMs to diffusion models and media decoders. Serving these models efficiently requires flexible…