52 citations · 89 across the 9 of their papers we have counts for
17 papers · 1 filter
Simple Recurrence Improves Masked Language Models
Tao Lei, Ran Tian, Jasmijn Bastings +1
In this work, we explore whether modeling recurrence into the Transformer architecture can both be beneficial and efficient, by building an extremely simple recurrent module into t…
Learning Compact Metrics for MT
Amy Pu, Hyung Won Chung, Ankur P. Parikh +2
Recent developments in machine translation and multilingual text generation have led researchers to adopt trained metrics such as COMET or BLEURT, which treat evaluation as a regre…
Shatter: An Efficient Transformer Encoder with Single-Headed Self-Attention and Relative Sequence Partitioning
Ran Tian, Joshua Maynez, Ankur P. Parikh
The highly popular Transformer architecture, based on self-attention, is the foundation of large pretrained models such as BERT, that have become an enduring paradigm in NLP. While…
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal +53
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly evolving ecosystem of auto…
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution
Xavier Garcia, Noah Constant, Ankur P. Parikh +1
We propose a straightforward vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models, paving the way towards efficient continual lea…
Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task
Thibault Sellam, Amy Pu, Hyung Won Chung +5
The quality of machine translation systems has dramatically improved over the last decade, and as a result, evaluation has become an increasingly challenging problem. This paper de…