Exploring Prediction Targets in Masked Pre-Training for Speech Foundation Models

Li-Wei Chen, Takuya Higuchi, He Bai +6

Speech foundation models, such as HuBERT and its variants, are pre-trained on large amounts of unlabeled speech data and then used for a range of downstream tasks. These models use…