NEST-RQ: Next Token Prediction for Speech Self-Supervised Pre-Training

Minglun Han, Ye Bai, Chen Shen +6

Speech self-supervised pre-training can effectively improve the performance of downstream tasks. However, previous self-supervised learning (SSL) methods for speech, such as HuBERT…