Scaling Spoken Language Models with Syllabic Speech Tokenization

Nicholas Lee, Cheol Jun Cho, Alan W Black +1

Spoken language models (SLMs) typically discretize speech into high-frame-rate tokens extracted from SSL speech models. As the most successful LMs are based on the Transformer arch…