5.7k citations · 5.7k across the 2 of their papers we have counts for
3 papers
cs.CL2018
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing
Taku Kudo, John Richardson
This paper describes SentencePiece, a language-independent subword tokenizer and detokenizer designed for Neural-based text processing, including Neural Machine Translation. It pro…
cs.CL2018★ 7 cited
Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates
Taku Kudo
Subword units are an effective way to alleviate the open vocabulary problems in neural machine translation (NMT). While sentences are usually converted into unique subword sequence…
cs.CL2016★ 5.7k cited
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Yonghui Wu, Mike Schuster, Zhifeng Chen +28
Neural Machine Translation (NMT) is an end-to-end learning approach for automated translation, with the potential to overcome many of the weaknesses of conventional phrase-based tr…