2.7k citations · 3.3k across the 22 of their papers we have counts for
23 papers
Better and Worse with Scale: How Contextual Entrainment Diverges with Model Size
Dikshant Kukreja, Kshitij Sah, Gautam Gupta +5
Larger language models become simultaneously better and worse at handling contextual information -- better at ignoring false claims, worse at ignoring irrelevant tokens. We formali…
RouteLLM: Learning to Route LLMs with Preference Data
Isaac Ong, Amjad Almahairi, Vincent Wu +5
Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and c…
"Sorry, Come Again?" Prompting -- Enhancing Comprehension and Diminishing Hallucination with [PAUSE]-injected Optimal Paraphrasing
Vipula Rawte, S. M Towhidul Islam Tonmoy, S M Mehedi Zaman +4
Hallucination has emerged as the most vulnerable aspect of contemporary Large Language Models (LLMs). In this paper, we introduce the Sorry, Come Again (SCA) prompting, aimed to av…
Towards Artwork Explanation in Large-scale Vision Language Models
Kazuki Hayashi, Yusuke Sakai, Hidetaka Kamigaito +2
Large-scale Vision-Language Models (LVLMs) output text from images and instructions, demonstrating capabilities in text generation and comprehension. However, it has not been clari…
When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards
Norah Alzahrani, Hisham Abdullah Alyahya, Yazeed Alnumay +9
Large Language Model (LLM) leaderboards based on benchmark rankings are regularly used to guide practitioners in model selection. Often, the published leaderboard rankings are take…
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model
Shraman Pramanick, Guangxing Han, Rui Hou +6
The ability of large language models (LLMs) to process visual inputs has given rise to general-purpose vision systems, unifying various vision-language (VL) tasks by instruction tu…
Llama 2: Open Foundation and Fine-Tuned Chat Models
Hugo Touvron, Louis Martin, Kevin Stone +65
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our f…
Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization
Anastasia Razdaibiedina, Yuning Mao, Rui Hou +4
Prompt tuning is one of the successful approaches for parameter-efficient tuning of pre-trained language models. Despite being arguably the most parameter-efficient (tuned soft pro…
Learning Easily Updated General Purpose Text Representations with Adaptable Task-Specific Prefixes
Kuan-Hao Huang, Liang Tan, Rui Hou +3
Many real-world applications require making multiple predictions from the same text. Fine-tuning a large pre-trained language model for each downstream task causes computational bu…
Progressive Prompts: Continual Learning for Language Models
Anastasia Razdaibiedina, Yuning Mao, Rui Hou +3
We introduce Progressive Prompts - a simple and efficient approach for continual learning in language models. Our method allows forward transfer and resists catastrophic forgetting…
Uniform Masking Prevails in Vision-Language Pretraining
Siddharth Verma, Yuchen Lu, Rui Hou +4
Masked Language Modeling (MLM) has proven to be an essential component of Vision-Language (VL) pretraining. To implement MLM, the researcher must make two design choices: the maski…
Logical Satisfiability of Counterfactuals for Faithful Explanations in NLI
Suzanna Sia, Anton Belyy, Amjad Almahairi +3
Evaluating an explanation's faithfulness is desired for many reasons such as trust, interpretability and diagnosing the sources of model's errors. In this work, which focuses on th…
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning
Yuning Mao, Lambert Mathias, Rui Hou +5
Recent parameter-efficient language model tuning (PELT) methods manage to match the performance of fine-tuning with much fewer trainable parameters and perform especially well when…
Unsupervised Learning of Dense Visual Representations
Pedro O. Pinheiro, Amjad Almahairi, Ryan Y. Benmalek +2
Contrastive self-supervised learning has emerged as a promising approach to unsupervised visual representation learning. In general, these methods learn global (image-level) repres…
The Impact of Preprocessing on Arabic-English Statistical and Neural Machine Translation
Mai Oudah, Amjad Almahairi, Nizar Habash
Neural networks have become the state-of-the-art approach for machine translation (MT) in many languages. While linguistically-motivated tokenization techniques were shown to have…
Adversarial Computation of Optimal Transport Maps
Jacob Leygonie, Jennifer She, Amjad Almahairi +2
Computing optimal transport maps between high-dimensional and continuous distributions is a challenging problem in optimal transport (OT). Generative adversarial networks (GANs) ar…
A Closer Look at the Optimization Landscapes of Generative Adversarial Networks
Hugo Berard, Gauthier Gidel, Amjad Almahairi +2
Generative adversarial networks have been very successful in generative modeling, however they remain relatively challenging to train compared to standard deep neural networks. In…
Learning Distributed Representations from Reviews for Collaborative Filtering
Amjad Almahairi, Kyle Kastner, Kyunghyun Cho +1
Recent work has shown that collaborative filter-based recommender systems can be improved by incorporating side information, such as natural language reviews, as a way of regulariz…
Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni +2
Learning inter-domain mappings from unpaired data can improve performance in structured prediction tasks, such as image segmentation, by reducing the need for paired data. CycleGAN…
Calibrating Energy-based Generative Adversarial Networks
Zihang Dai, Amjad Almahairi, Philip Bachman +2
In this paper, we propose to equip Generative Adversarial Networks with the ability to produce direct energy estimates for samples.Specifically, we propose a flexible adversarial t…
First Result on Arabic Neural Machine Translation
Amjad Almahairi, Kyunghyun Cho, Nizar Habash +1
Neural machine translation has become a major alternative to widely used phrase-based statistical machine translation. We notice however that much of research on neural machine tra…
Theano: A Python framework for fast computation of mathematical expressions
The Theano Development Team, Rami Al-Rfou, Guillaume Alain +110
Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Since its introduction, it has bee…
Dynamic Capacity Networks
Amjad Almahairi, Nicolas Ballas, Tim Cooijmans +3
We introduce the Dynamic Capacity Network (DCN), a neural network that can adaptively assign its capacity across different portions of the input data. This is achieved by combining…