602 citations · 879 across the 7 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021★ 1 cited
nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?
Mihir Kale, Aditya Siddhant, Noah Constant +3
Recently, mT5 - a massively multilingual version of T5 - leveraged a unified text-to-text format to attain state-of-the-art results on a wide variety of multilingual NLP tasks. In…
cs.CL2021★ 69 cited
ByT5: Towards a token-free future with pre-trained byte-to-byte models
Linting Xue, Aditya Barua, Noah Constant +5
Most widely-used pre-trained language models operate on sequences of tokens corresponding to word or subword units. By comparison, token-free models that operate directly on raw te…