6 papers
RoPE is Dead, Long Live RoPE: Towards Scalable Data-aware Positional Encodings
Jarod Lévy, Mathurin Videau, Jad Yehya +3
Transformers process tokens without any inherent notion of order, making positional encoding a fundamental requirement rather than an architectural refinement. Rotary Position Embe…
Accurate Decoding of Natural Sentences from Non-Invasive Brain Recordings
Mingfang Zhang, Jarod Lévy, Cedric Rommel +9
Restoring communication for people who have lost the ability to speak or move after a brain injury is a major challenge. While intracranial implants now enable high-performing brai…
DANCE: Detect and Classify Events in EEG
Jarod Lévy, Hubert Banville, Jérémy Rapin +3
Event identification in continuous neural recordings is a critical task in neuroscience. Decoding in EEG is dominated by classifying windows aligned to known event onsets. However,…
Geometry-Aware Discretization Error of Diffusion Models
Samuel Hurault, Thomas Moreau, Gabriel Peyré
Practical diffusion sampling requires simulating a reverse-time ODE or SDE with a limited number of denoising steps, making the choice of sampling parameters crucial for minimizing…
RoseCDL: Robust and Scalable Convolutional Dictionary Learning for Rare event and Anomaly Detection
Jad Yehya, Mansour Benbakoura, Cédric Allain +3
Detecting rare events and anomalies in large-scale signals is essential in fields such as astronomy, physical simulations, and biomedical science. In many cases, this problem natur…
Analysis and Synthesis Denoisers for Forward-Backward Plug-and-Play Algorithms
Matthieu Kowalski, Benoît Malézieux, Thomas Moreau +1
In this work we study the behavior of the forward-backward (FB) algorithm when the proximity operator is replaced by a sub-iterative procedure to approximate a Gaussian denoiser, i…