OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming

Zongshang Shen, Wangsong Yin, Daliang Xu +2

On-device streaming omni-modal inference safeguards user privacy and eliminates prohibitive per-token API costs, but faces a critical bottleneck: the continuous influx of multimoda…