177 citations · 392 across the 27 of their papers we have counts for
33 papers
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Yong +14
Recent breakthroughs in large language models (LLMs) have centered around a handful of data-rich languages. What does it take to broaden access to breakthroughs beyond first-class…
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning
Shivalika Singh, Freddie Vargus, Daniel Dsouza +30
Datasets are foundational to many breakthroughs in modern artificial intelligence. Many recent achievements in the space of natural language processing (NLP) can be attributed to t…
Intriguing Properties of Compression on Multilingual Models
Kelechi Ogueji, Orevaoghene Ahia, Gbemileke Onilude +3
Multilingual models are often particularly dependent on scaling to generalize to a growing number of languages. Compression techniques are widely relied upon to reconcile the growt…
Domain Curricula for Code-Switched MT at MixMT 2022
Lekan Raheem, Maab Elrashid
In multilingual colloquial settings, it is a habitual occurrence to compose expressions of text or speech containing tokens or phrases of different languages, a phenomenon popularl…
JoeyS2T: Minimalistic Speech-to-Text Modeling with JoeyNMT
Mayumi Ohta, Julia Kreutzer, Stefan Riezler
JoeyS2T is a JoeyNMT extension for speech-to-text tasks such as automatic speech recognition and end-to-end speech translation. It inherits the core philosophy of JoeyNMT, a minima…
Building Machine Translation Systems for the Next Thousand Languages
Ankur Bapna, Isaac Caswell, Julia Kreutzer +21
In this paper we share findings from our effort to build practical machine translation (MT) systems capable of translating across over one thousand languages. We describe results i…
A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation
David Ifeoluwa Adelani, Jesujoba Oluwadara Alabi, Angela Fan +42
Recent advances in the pre-training of language models leverage large-scale datasets to create multilingual models. However, low-resource languages are mostly left out in these dat…
Can Multilinguality benefit Non-autoregressive Machine Translation?
Sweta Agrawal, Julia Kreutzer, Colin Cherry
Non-autoregressive (NAR) machine translation has recently achieved significant improvements, and now outperforms autoregressive (AR) models on some benchmarks, providing an efficie…
Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits
Julia Kreutzer, David Vilar, Artem Sokolov
Training data for machine translation (MT) is often sourced from a multitude of large corpora that are multi-faceted in nature, e.g. containing contents from multiple domains or di…
The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation
Orevaoghene Ahia, Julia Kreutzer, Sara Hooker
A "bigger is better" explosion in the number of parameters in deep neural networks has made it increasingly challenging to make state-of-the-art networks accessible in compute-rest…
Evaluating Multiway Multilingual NMT in the Turkic Languages
Jamshidbek Mirzakhalov, Anoop Babu, Aigiz Kunafin +11
Despite the increasing number of large and comprehensive machine translation (MT) systems, evaluation of these methods in various languages has been restrained by the lack of high-…
Modelling Latent Translations for Cross-Lingual Transfer
Edoardo Maria Ponti, Julia Kreutzer, Ivan Vulić +1
While achieving state-of-the-art results in multiple tasks and languages, translation-based cross-lingual transfer is often overlooked in favour of massively multilingual pre-train…
Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation
Samuel Kiegeland, Julia Kreutzer
Policy gradient algorithms have found wide adoption in NLP, but have recently become subject to criticism, doubting their suitability for NMT. Choshen et al. (2020) identify multip…
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Julia Kreutzer, Isaac Caswell, Lisa Wang +49
With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text dataset…
MasakhaNER: Named Entity Recognition for African Languages
David Ifeoluwa Adelani, Jade Abbott, Graham Neubig +58
We take a step towards addressing the under-representation of the African continent in NLP research by creating the first large publicly available high-quality dataset for named en…
Neural Machine Translation for Extremely Low-Resource African Languages: A Case Study on Bambara
Allahsera Auguste Tapo, Bakary Coulibaly, Sébastien Diarra +6
Low-resource languages present unique challenges to (neural) machine translation. We discuss the case of Bambara, a Mande language for which training data is scarce and requires si…
Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages
Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila +45
Research in NLP lacks geographic diversity, and the question of how NLP can be scaled to low-resourced languages has not yet been adequately solved. "Low-resourced"-ness is a compl…
Offline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Tasks
Julia Kreutzer, Stefan Riezler, Carolin Lawrence
Large volumes of interaction logs can be collected from NLP systems that are deployed in the real world. How can this wealth of information be leveraged? Using such interaction log…
KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundi
Rubungo Andre Niyongabo, Hong Qu, Julia Kreutzer +1
Recent progress in text classification has been focused on high-resource languages such as English and Chinese. For low-resource languages, amongst them most African languages, the…
Inference Strategies for Machine Translation with Conditional Masking
Julia Kreutzer, George Foster, Colin Cherry
Conditional masked language model (CMLM) training has proven successful for non-autoregressive and semi-autoregressive sequence generation tasks, such as machine translation. Given…
Correct Me If You Can: Learning from Error Corrections and Markings
Julia Kreutzer, Nathaniel Berger, Stefan Riezler
Sequence-to-sequence learning involves a trade-off between signal strength and annotation cost of training data. For example, machine translation data range from costly expert-gene…
On Optimal Transformer Depth for Low-Resource Language Translation
Elan van Biljon, Arnu Pretorius, Julia Kreutzer
Transformers have shown great promise as an approach to Neural Machine Translation (NMT) for low-resource languages. However, at the same time, transformer models remain difficult…
Masakhane -- Machine Translation For Africa
Iroro Orife, Julia Kreutzer, Blessing Sibanda +22
Africa has over 2000 languages. Despite this, African languages account for a small portion of available resources and publications in Natural Language Processing (NLP). This is du…
Joey NMT: A Minimalist NMT Toolkit for Novices
Julia Kreutzer, Jasmijn Bastings, Stefan Riezler
We present Joey NMT, a minimalist neural machine translation toolkit based on PyTorch that is specifically designed for novices. Joey NMT provides many popular NMT features in a sm…