Metronome: Bound the Cache, Keep the Beat for Real-Time Interaction Model Serving

Jiaying Meng, Bojie Li

Real-time interaction models -- Moshi, MiniCPM-o, Qwen-Omni -- turn serving into a periodic real-time task: on every frame a session ingests streaming audio and must respond by a r…