488 citations · 990 across the 37 of their papers we have counts for
13 papers · 1 filter
The Design Space of Tri-Modal Masked Diffusion Models
Louis Bethune, Victor Turrisi, Bruno Kacper Mlodozeniec +21
Discrete diffusion models have emerged as strong alternatives to autoregressive language models, with recent work initializing and fine-tuning a base unimodal model for bimodal gen…
ParaRNN: Unlocking Parallel Training of Nonlinear RNNs for Large Language Models
Federico Danieli, Pau Rodriguez, Miguel Sarabia +2
Recurrent Neural Networks (RNNs) laid the foundation for sequence modeling, but their intrinsic sequential nature restricts parallel computation, creating a fundamental barrier to…
Understanding Input Selectivity in Mamba: Impact on Approximation Power, Memorization, and Associative Recall Capacity
Ningyuan Huang, Miguel Sarabia, Abhinav Moudgil +3
State-Space Models (SSMs), and particularly Mamba, have recently emerged as a promising alternative to Transformers. Mamba introduces input selectivity to its SSM layer (S6) and in…
Controlling Language and Diffusion Models by Transporting Activations
Pau Rodriguez, Arno Blaas, Michal Klein +4
The increasing capabilities of large generative models and their ever more widespread deployment have raised concerns about their reliability, safety, and potential misuse. To addr…
GEO-Bench: Toward Foundation Models for Earth Monitoring
Alexandre Lacoste, Nils Lehmann, Pau Rodriguez +14
Recent progress in self-supervision has shown that pre-training large neural networks on vast amounts of unsupervised data can lead to substantial increases in generalization to do…
CADet: Fully Self-Supervised Out-Of-Distribution Detection With Contrastive Learning
Charles Guille-Escuret, Pau Rodriguez, David Vazquez +2
Handling out-of-distribution (OOD) samples has become a major stake in the real-world deployment of machine learning systems. This work explores the use of self-supervised contrast…
Constraining Representations Yields Models That Know What They Don't Know
Joao Monteiro, Pau Rodriguez, Pierre-Andre Noel +2
A well-known failure mode of neural networks is that they may confidently return erroneous predictions. Such unsafe behaviour is particularly frequent when the use case slightly di…
Toward Foundation Models for Earth Monitoring: Proposal for a Climate Change Benchmark
Alexandre Lacoste, Evan David Sherwin, Hannah Kerner +9
Recent progress in self-supervision shows that pre-training large neural networks on vast amounts of unsupervised data can lead to impressive increases in generalisation for downst…
Continual Learning via Local Module Composition
Oleksiy Ostapenko, Pau Rodriguez, Massimo Caccia +1
Modularity is a compelling solution to continual learning (CL), the problem of modeling sequences of related tasks. Learning and then composing modules to solve different tasks pro…
Sequoia: A Software Framework to Unify Continual Learning Research
Fabrice Normandin, Florian Golemo, Oleksiy Ostapenko +10
The field of Continual Learning (CL) seeks to develop algorithms that accumulate knowledge and skills over time through interaction with non-stationary environments. In practice, a…
Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations
Pau Rodriguez, Massimo Caccia, Alexandre Lacoste +4
Explainability for machine learning models has gained considerable attention within the research community given the importance of deploying more reliable machine-learning systems.…
TADAM: Task dependent adaptive metric for improved few-shot learning
Boris N. Oreshkin, Pau Rodriguez, Alexandre Lacoste
Few-shot learning has become essential for producing models that generalize from few examples. In this work, we identify that metric scaling and metric task conditioning are import…
Regularizing CNNs with Locally Constrained Decorrelations
Pau Rodríguez, Jordi Gonzàlez, Guillem Cucurull +2
Regularization is key for deep learning since it allows training more complex models while keeping lower levels of overfitting. However, the most prevalent regularizations do not l…