Textually Pretrained Speech Language Models

Michael Hassid, Tal Remez, Tu Anh Nguyen +9

Speech language models (SpeechLMs) process and generate acoustic data only, without textual supervision. In this work, we propose TWIST, a method for training SpeechLMs using a war…