LiveServe: Interaction-Aware Serving for Real-Time Omni-Modal LLMs

Xiangyu Zhi, Peiqi Yin, Sheng Guan +3

Realtime omni-modal LMs support speech-centric conversations where users stream inputs, hear generated audio, and interrupt freely. Existing Omni-LM serving systems still rely on t…