1.3k citations · 3.7k across the 43 of their papers we have counts for
9 papers · 1 filter
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor
Or Honovich, Thomas Scialom, Omer Levy +1
Instruction tuning enables pretrained language models to perform new tasks from inference-time natural language descriptions. These approaches rely on vast amounts of human supervi…
LMentry: A Language Model Benchmark of Elementary Language Tasks
Avia Efrat, Or Honovich, Omer Levy
As the performance of large language models rapidly improves, benchmarks are getting larger and more complex as well. We present LMentry, a benchmark that avoids this "arms race" b…
A Simple Baseline for Beam Search Reranking
Lior Vassertail, Omer Levy
Reranking methods in machine translation aim to close the gap between common evaluation metrics (e.g. BLEU) and maximum likelihood learning and decoding algorithms. Prior works add…
Causes and Cures for Interference in Multilingual Translation
Uri Shaham, Maha Elbayad, Vedanuj Goswami +2
Multilingual machine translation models can benefit from synergy between different language pairs, but also suffer from interference. While there is a growing number of sophisticat…
Instruction Induction: From Few Examples to Natural Language Task Descriptions
Or Honovich, Uri Shaham, Samuel R. Bowman +1
Large language models are able to perform a task by conditioning on a few input-output demonstrations - a paradigm known as in-context learning. We show that language models can ex…
Breaking Character: Are Subwords Good Enough for MRLs After All?
Omri Keren, Tal Avinari, Reut Tsarfaty +1
Large pretrained language models (PLMs) typically tokenize the input string into contiguous subwords before any pretraining or inference. However, previous studies have claimed tha…