Interleaved Speech Language Models Latently Work In Text

Talia Sternberg, Gallil Maimon, Yossi Adi

Speech language models (SLMs) increasingly combine speech and text, often by interleaving their tokens within a single sequence. Yet how these two modalities interact in the model'…