180 citations · 192 across the 6 of their papers we have counts for
6 papers
Attention Is All You Need But You Don't Need All Of It For Inference of Large Language Models
Georgy Tyukin, Gbetondji J-S Dovonon, Jean Kaddour +1
The inference demand for LLMs has skyrocketed in recent months, and serving models with low latencies remains challenging due to the quadratic input length complexity of the attent…
Challenges and Applications of Large Language Models
Jean Kaddour, Joshua Harris, Maximilian Mozes +3
Large Language Models (LLMs) went from non-existent to ubiquitous in the machine learning discourse within a few years. Due to the fast pace of the field, it is difficult to identi…
TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models
Yuwei Yin, Jean Kaddour, Xiang Zhang +4
Data augmentation has been established as an efficacious approach to supplement useful information for low-resource datasets. Traditional augmentation techniques such as noise inje…
The MiniPile Challenge for Data-Efficient Language Models
Jean Kaddour
The ever-growing diversity of pre-training text corpora has equipped language models with generalization capabilities across various downstream tasks. However, such diverse dataset…
Spawrious: A Benchmark for Fine Control of Spurious Correlation Biases
Aengus Lynch, Gbètondji J-S Dovonon, Jean Kaddour +1
The problem of spurious correlations (SCs) arises when a classifier relies on non-predictive features that happen to be correlated with the labels in the training data. For example…
DAG Learning on the Permutahedron
Valentina Zantedeschi, Luca Franceschi, Jean Kaddour +2
We propose a continuous optimization framework for discovering a latent directed acyclic graph (DAG) from observational data. Our approach optimizes over the polytope of permutatio…