3 citations · 6 across the 13 of their papers we have counts for
14 papers
SLAD : Shared LoRA Adapters for Task Specific Distillation
Reda Bensaid, Yassir Bendou, Vincent Gripon +1
In the context of resource-constrained environments such as embedded systems, adapting reduced-size foundation models to downstream tasks has become increasingly popular. This has…
Efficient Few-Shot Learning for Edge AI via Knowledge Distillation on MobileViT
Shuhei Tsuyuki, Reda Bensaid, Jérémy Morlier +4
Efficient and adaptable deep learning models are an important area of deep learning research, driven by the need for highly efficient models on edge devices. Few-shot learning enab…
Residual Connections and the Causal Shift: Uncovering a Structural Misalignment in Transformers
Jonathan Lys, Vincent Gripon, Bastien Pasdeloup +4
Large Language Models (LLMs) are trained with next-token prediction, implemented in autoregressive Transformers via causal masking for parallelism. This creates a subtle misalignme…
Inner Loop Inference for Pretrained Transformers: Unlocking Latent Capabilities Without Training
Jonathan Lys, Vincent Gripon, Bastien Pasdeloup +4
Deep Learning architectures, and in particular Transformers, are conventionally viewed as a composition of layers. These layers are actually often obtained as the sum of two contri…
D5P4: Partition Determinantal Point Process for Diversity in Parallel Discrete Diffusion Decoding
Jonathan Lys, Vincent Gripon, Axel Marmoret +4
Discrete diffusion models are promising alternatives to autoregressive approaches for text generation, yet their decoding methods remain under-studied. Standard autoregressive sear…
MUKA: Multi Kernel Audio Adaptation Of Audio-Language Models
Reda Bensaid, Amine Ouasfi, Yassir Bendou +4
Multimodal foundation models have demonstrated impressive generalization capabilities, yet efficiently adapting them to new tasks in a few-shot setting remains a critical challenge…
REVE: A Foundation Model for EEG -- Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects
Yassine El Ouahidi, Jonathan Lys, Philipp Thölke +5
Foundation models have transformed AI by reducing reliance on task-specific data through large-scale pretraining. While successful in language and vision, their adoption in EEG has…
TensLoRA: Tensor Alternatives for Low-Rank Adaptation
Axel Marmoret, Reda Bensaid, Jonathan Lys +2
Low-Rank Adaptation (LoRA) is widely used to efficiently adapt Transformers by adding trainable low-rank matrices to attention projections. While effective, these matrices are cons…
ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models
Yassir Bendou, Amine Ouasfi, Vincent Gripon +1
The growing popularity of Contrastive Language-Image Pretraining (CLIP) has led to its widespread application in various visual downstream tasks. To enhance CLIP's effectiveness an…
Oops, I Sampled it Again: Reinterpreting Confidence Intervals in Few-Shot Learning
Raphael Lafargue, Luke Smith, Franck Vermet +4
The predominant method for computing confidence intervals (CI) in few-shot learning (FSL) is based on sampling the tasks with replacement, i.e.\ allowing the same samples to appear…
LLM meets Vision-Language Models for Zero-Shot One-Class Classification
Yassir Bendou, Giulia Lioi, Bastien Pasdeloup +4
We consider the problem of zero-shot one-class visual classification, extending traditional one-class classification to scenarios where only the label of the target class is availa…
Unsupervised Adaptive Deep Learning Method For BCI Motor Imagery Decoding
Yassine El Ouahidi, Giulia Lioi, Nicolas Farrugia +2
In the context of Brain-Computer Interfaces, we propose an adaptive method that reaches offline performance level while being usable online without requiring supervision. Interesti…
Few and Fewer: Learning Better from Few Examples Using Fewer Base Classes
Raphael Lafargue, Yassir Bendou, Bastien Pasdeloup +4
When training data is scarce, it is common to make use of a feature extractor that has been pre-trained on a large base dataset, either by fine-tuning its parameters on the ``targe…
A Novel Benchmark for Few-Shot Semantic Segmentation in the Era of Foundation Models
Reda Bensaid, Vincent Gripon, François Leduc-Primeau +3
Few-shot semantic segmentation (FSS) is a crucial challenge in computer vision, driving extensive research into a diverse range of methods, from advanced meta-learning techniques t…