Residual Tokens Enhance Masked Autoencoders for Speech Modeling

Samir Sadok, Stéphane Lathuilière, Xavier Alameda-Pineda

Recent speech modeling relies on explicit attributes such as pitch, content, and speaker identity, but these alone cannot capture the full richness of natural speech. We introduce…