387 citations · 1.2k across the 27 of their papers we have counts for
3 papers · 1 filter
W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training
Yu-An Chung, Yu Zhang, Wei Han +4
Motivated by the success of masked language modeling~(MLM) in pre-training natural language processing models, we propose w2v-BERT that explores MLM for self-supervised speech repr…
Distilling Interpretable Models into Human-Readable Code
Walker Ravina, Ethan Sterling, Olexiy Oryeshko +5
The goal of model distillation is to faithfully transfer teacher model knowledge to a model which is faster, more generalizable, more interpretable, or possesses other desirable ch…
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Jonathan Shen, Patrick Nguyen, Yonghui Wu +88
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Lingvo models a…