MoST: Mixing Speech and Text with Modality-Aware Mixture of Experts

Yuxuan Lou, Kai Yang, Yang You

We present MoST (Mixture of Speech and Text), a novel multimodal large language model that seamlessly integrates speech and text processing through our proposed Modality-Aware Mixt…