168 citations · 471 across the 17 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021★ 50 cited
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training
Ankur Bapna, Yu-an Chung, Nan Wu +7
Unsupervised pre-training is now the predominant approach for both text and speech understanding. Self-attention models pre-trained on large amounts of unannotated data have been h…
cs.CL2021★ 133 cited
CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation
Jonathan H. Clark, Dan Garrette, Iulia Turc +1
Pipelined NLP systems have largely been superseded by end-to-end neural modeling, yet nearly all commonly-used models still require an explicit tokenization step. While recent toke…