28 citations · 45 across the 12 of their papers we have counts for
16 papers
Tracing Computation Density in LLMs
Corentin Kervadec, Iuliia Lysova, Iuri Macocco +2
Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs, but it is not clear that they exploit their f…
Sparse or Dense? A Mechanistic Estimation of Computation Density in Transformer-based LLMs
Corentin Kervadec, Iuliia Lysova, Marco Baroni +1
Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs. Several studies on LLM efficiency optimizatio…
Evil twins are not that evil: Qualitative insights into machine-generated prompts
Nathanaël Carraz Rakotonirina, Corentin Kervadec, Francesca Franzon +1
It has been widely observed that language models (LMs) respond in predictable ways to algorithmically generated prompts that are seemingly unintelligible. This is both a sign that…
Emergence of a High-Dimensional Abstraction Phase in Language Transformers
Emily Cheng, Diego Doimo, Corentin Kervadec +4
A language model (LM) is a mapping from a linguistic context to an output token. However, much remains to be known about this mapping, including how its geometric properties relate…
Bridging Information-Theoretic and Geometric Compression in Language Models
Emily Cheng, Corentin Kervadec, Marco Baroni
For a language model (LM) to faithfully model human language, it must compress vast, potentially infinite information into relatively few dimensions. We propose analyzing compressi…
Unnatural language processing: How do language models handle machine-generated prompts?
Corentin Kervadec, Francesca Franzon, Marco Baroni
Language model prompt optimization research has shown that semantically and grammatically well-formed manually crafted prompts are routinely outperformed by automatically generated…
An experimental study of the vision-bottleneck in VQA
Pierre Marza, Corentin Kervadec, Grigory Antipov +2
As in many tasks combining vision and language, both modalities play a crucial role in Visual Question Answering (VQA). To properly solve the task, a given model should both unders…
Supervising the Transfer of Reasoning Patterns in VQA
Corentin Kervadec, Christian Wolf, Grigory Antipov +2
Methods for Visual Question Anwering (VQA) are notorious for leveraging dataset biases rather than performing reasoning, hindering generalization. It has been recently shown that b…
How Transferable are Reasoning Patterns in VQA?
Corentin Kervadec, Theo Jaunet, Grigory Antipov +3
Since its inception, Visual Question Answering (VQA) is notoriously known as a task, where models are prone to exploit biases in datasets to find shortcuts instead of performing hi…
VisQA: X-raying Vision and Language Reasoning in Transformers
Theo Jaunet, Corentin Kervadec, Romain Vuillemot +3
Visual Question Answering systems target answering open-ended textual questions given input images. They are a testbed for learning high-level reasoning with a primary use in HCI,…
Estimating semantic structure for the VQA answer space
Corentin Kervadec, Grigory Antipov, Moez Baccouche +1
Since its appearance, Visual Question Answering (VQA, i.e. answering a question posed over an image), has always been treated as a classification problem over a set of predefined a…
Roses Are Red, Violets Are Blue... but Should Vqa Expect Them To?
Corentin Kervadec, Grigory Antipov, Moez Baccouche +1
Models for Visual Question Answering (VQA) are notorious for their tendency to rely on dataset biases, as the large and unbalanced diversity of questions and concepts involved and…
Weak Supervision helps Emergence of Word-Object Alignment and improves Vision-Language Tasks
Corentin Kervadec, Grigory Antipov, Moez Baccouche +1
The large adoption of the self-attention (i.e. transformer model) and BERT-like training principles has recently resulted in a number of high performing models on a large panoply o…
The Many Moods of Emotion
Valentin Vielzeuf, Corentin Kervadec, Stéphane Pateux +1
This paper presents a novel approach to the facial expression generation problem. Building upon the assumption of the psychological community that emotion is intrinsically continuo…
An Occam's Razor View on Learning Audiovisual Emotion Recognition with Small Training Sets
Valentin Vielzeuf, Corentin Kervadec, Stéphane Pateux +2
This paper presents a light-weight and accurate deep neural model for audiovisual emotion recognition. To design this model, the authors followed a philosophy of simplicity, drasti…
CAKE: Compact and Accurate K-dimensional representation of Emotion
Corentin Kervadec, Valentin Vielzeuf, Stéphane Pateux +2
Numerous models describing the human emotional states have been built by the psychology community. Alongside, Deep Neural Networks (DNN) are reaching excellent performances and are…