607 citations · 3.5k across the 64 of their papers we have counts for
12 papers · 1 filter
Understanding Knowledge Distillation in Non-autoregressive Machine Translation
Chunting Zhou, Graham Neubig, Jiatao Gu
Non-autoregressive machine translation (NAT) systems predict a sequence of output tokens in parallel, achieving substantial improvements in generation speed compared to autoregress…
Harnessing Indirect Training Data for End-to-End Automatic Speech Translation: Tricks of the Trade
Juan Pino, Liezl Puzon, Jiatao Gu +3
For automatic speech translation (AST), end-to-end approaches are outperformed by cascaded models that transcribe with automatic speech recognition (ASR), then translate with machi…
Depth-Adaptive Transformer
Maha Elbayad, Jiatao Gu, Edouard Grave +1
State of the art sequence-to-sequence models for large scale tasks perform a fixed number of computations for each input sequence regardless of whether it is easy or hard to proces…
Monotonic Multihead Attention
Xutai Ma, Juan Pino, James Cross +2
Simultaneous machine translation models start generating a target sequence before they have encoded or read the source sequence. Recent approaches for this task either apply a fixe…
Improved Variational Neural Machine Translation by Promoting Mutual Information
Arya D. McCarthy, Xian Li, Jiatao Gu +1
Posterior collapse plagues VAEs for text, especially for conditional text generation with strong autoregressive decoders. In this work, we address this problem in variational neura…
VizSeq: A Visual Analysis Toolkit for Text Generation Tasks
Changhan Wang, Anirudh Jain, Danlu Chen +1
Automatic evaluation of text generation tasks (e.g. machine translation, text summarization, image captioning and video description) usually relies heavily on task-specific metrics…
Revisiting Self-Training for Neural Sequence Generation
Junxian He, Jiatao Gu, Jiajun Shen +1
Self-training is one of the earliest and simplest semi-supervised methods. The key idea is to augment the original labeled dataset with unlabeled data paired with the model's predi…
The Source-Target Domain Mismatch Problem in Machine Translation
Jiajun Shen, Peng-Jen Chen, Matt Le +5
While we live in an increasingly interconnected world, different places still exhibit strikingly different cultures and many events we experience in our every day life pertain only…
Neural Machine Translation with Byte-Level Subwords
Changhan Wang, Kyunghyun Cho, Jiatao Gu
Almost all existing machine translation models are built on top of character-based vocabularies: characters, subwords or words. Rare characters from noisy text or character-rich la…
Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations
Jiatao Gu, Yong Wang, Kyunghyun Cho +1
Zero-shot translation, translating between language pairs on which a Neural Machine Translation (NMT) system has never been trained, is an emergent property when training the syste…
Levenshtein Transformer
Jiatao Gu, Changhan Wang, Jake Zhao
Modern neural sequence generation models are built to either generate tokens step-by-step from scratch or (iteratively) modify a sequence of tokens bounded by a fixed length. In th…
Insertion-based Decoding with automatically Inferred Generation Order
Jiatao Gu, Qi Liu, Kyunghyun Cho
Conventional neural autoregressive decoding commonly assumes a fixed left-to-right generation order, which may be sub-optimal. In this work, we propose a novel decoding algorithm -…