106 citations · 139 across the 7 of their papers we have counts for
14 papers
Examining Scaling and Transfer of Language Model Architectures for Machine Translation
Biao Zhang, Behrooz Ghorbani, Ankur Bapna +4
Natural language understanding and generation models follow one of the two dominant architectural paradigms: language models (LMs) that process concatenated sequences in a single s…
Data Scaling Laws in NMT: The Effect of Noise and Architecture
Yamini Bansal, Behrooz Ghorbani, Ankush Garg +5
In this work, we study the effect of varying the architecture and training data quality on the data scaling properties of Neural Machine Translation (NMT). First, we establish that…
Sparse Attention with Linear Units
Biao Zhang, Ivan Titov, Rico Sennrich
Recently, it has been argued that encoder-decoder models can be made more interpretable by replacing the softmax function in the attention with its sparse variants. In this work, w…
Fast Interleaved Bidirectional Sequence Generation
Biao Zhang, Ivan Titov, Rico Sennrich
Independence assumptions during sequence generation can speed up inference, but parallel generation of highly inter-dependent tokens comes at a cost in quality. Instead of assuming…
Adaptive Feature Selection for End-to-End Speech Translation
Biao Zhang, Ivan Titov, Barry Haddow +1
Information in speech signals is not evenly distributed, making it an additional challenge for end-to-end (E2E) speech translation (ST) to learn to focus on informative features. I…
On Sparsifying Encoder Outputs in Sequence-to-Sequence Models
Biao Zhang, Ivan Titov, Rico Sennrich
Sequence-to-sequence models usually transfer all encoder outputs to the decoder for generation. In this work, by contrast, we hypothesize that these encoder outputs can be compress…