3.6k citations · 6.7k across the 72 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2022★ 1 cited
Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR
Dongseong Hwang, Khe Chai Sim, Yu Zhang +1
Knowledge distillation is an effective machine learning technique to transfer knowledge from a teacher model to a smaller student model, especially with unlabeled data. In this pap…
cs.LG2021★ 15 cited
W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training
Yu-An Chung, Yu Zhang, Wei Han +4
Motivated by the success of masked language modeling~(MLM) in pre-training natural language processing models, we propose w2v-BERT that explores MLM for self-supervised speech repr…