575 citations · 714 across the 27 of their papers we have counts for
27 papers
Sliding-window beats linear attention
Alexia Jolicoeur-Martineau, Rhea Sanjay Sukthanker, Pashmina Cameron +1
Due to the nature of quadratic attention, Large Language Models (LLMs) consume a lot of memory and energy: every new token costs more than the previous one, and its keys and values…
Probabilistic Tiny Recursive Model
Amin Sghaier, Ali Parviz, Alexia Jolicoeur-Martineau
Tiny Recursive Models (TRM) solve complex reasoning tasks with a fraction of the parameters of modern large language models (LLMs) by iteratively refining a latent state and final…
One Pass Is Not Enough: Recursive Latent Refinement for Generative Models
Mehdi Esmaeilzadeh, Alexia Jolicoeur-Martineau, Chirag Vashist +1
Despite remarkable progress, image generation is far from solved. The dominant metric, FID, conflates sample fidelity with mode coverage and is close to being saturated. Yet a mode…
On the Sample Efficiency of Inverse Dynamics Models for Semi-Supervised Imitation Learning
Sacha Morin, Moonsub Byeon, Alexia Jolicoeur-Martineau +1
Semi-supervised imitation learning (SSIL) consists in learning a policy from a small dataset of action-labeled trajectories and a much larger dataset of action-free trajectories. S…
Multi-Agent Game Generation and Evaluation via Audio-Visual Recordings
Alexia Jolicoeur-Martineau
While AI excels at generating text, audio, images, and videos, creating interactive audio-visual content such as video games remains challenging. Current LLMs can generate JavaScri…
Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes
Anthony Gosselin, Ge Ya Luo, Luis Lara +5
Video diffusion techniques have advanced significantly in recent years; however, they struggle to generate realistic imagery of car crashes due to the scarcity of accident events i…
Generating -Functional Molecules Using STGG+ with Active Learning
Alexia Jolicoeur-Martineau, Yan Zhang, Boris Knyazev +2
Generating novel molecules with out-of-distribution properties is a major challenge in molecular discovery. While supervised learning methods generate high-quality molecules simila…
Generating Tabular Data Using Heterogeneous Sequential Feature Forest Flow Matching
Ange-Clément Akazan, Alexia Jolicoeur-Martineau, Ioannis Mitliagkas
Privacy and regulatory constraints make data generation vital to advancing machine learning without relying on real-world datasets. A leading approach for tabular data generation i…
Beyond FVD: Enhanced Evaluation Metrics for Video Generation Quality
Ge Ya Luo, Gian Mario Favero, Zhi Hao Luo +2
The Fréchet Video Distance (FVD) is a widely adopted metric for evaluating video generation distribution quality. However, its effectiveness relies on critical assumptions. Our ana…
Understanding Adam Requires Better Rotation Dependent Assumptions
Tianyue H. Zhang, Lucas Maes, Alan Milligan +5
Despite its widespread adoption, Adam's advantage over Stochastic Gradient Descent (SGD) lacks a comprehensive theoretical explanation. This paper investigates Adam's sensitivity t…
Any-Property-Conditional Molecule Generation with Self-Criticism using Spanning Trees
Alexia Jolicoeur-Martineau, Aristide Baratin, Kisoo Kwon +2
Generating novel molecules is challenging, with most representations leading to generative models producing many invalid molecules. Spanning Tree-based Graph Generation (STGG) is a…
Ctrl-V: Higher Fidelity Video Generation with Bounding-Box Controlled Object Motion
Ge Ya Luo, Zhi Hao Luo, Anthony Gosselin +2
Controllable video generation has attracted significant attention, largely due to advances in video diffusion models. In domains such as autonomous driving, it is essential to deve…
LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters
Xinyu Zhou, Boris Knyazev, Alexia Jolicoeur-Martineau +1
A good initialization of deep learning models is essential since it can help them converge better and faster. However, pretraining large models is unaffordable for many researchers…
Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Trees
Alexia Jolicoeur-Martineau, Kilian Fatras, Tal Kachman
Tabular data is hard to acquire and is subject to missing values. This paper introduces a novel approach for generating and imputing mixed-type (continuous and categorical) tabular…
Diffusion models with location-scale noise
Alexia Jolicoeur-Martineau, Kilian Fatras, Ke Li +1
Diffusion Models (DMs) are powerful generative models that add Gaussian noise to the data and learn to remove it. We wanted to determine which noise distribution (Gaussian or non-G…
PopulAtion Parameter Averaging (PAPA)
Alexia Jolicoeur-Martineau, Emy Gervais, Kilian Fatras +2
Ensemble methods combine the predictions of multiple models to improve performance, but they require significantly higher computation costs at inference time. To avoid these costs,…
CNT (Conditioning on Noisy Targets): A new Algorithm for Leveraging Top-Down Feedback
Alexia Jolicoeur-Martineau, Alex Lamb, Vikas Verma +1
We propose a novel regularizer for supervised learning called Conditioning on Noisy Targets (CNT). This approach consists in conditioning the model on a noisy version of the target…
MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation
Vikram Voleti, Alexia Jolicoeur-Martineau, Christopher Pal
Video prediction is a challenging task. The quality of video frames from current state-of-the-art (SOTA) generative models tends to be poor and generalization beyond the training d…
Gotta Go Fast When Generating Data with Score-Based Models
Alexia Jolicoeur-Martineau, Ke Li, Rémi Piché-Taillefer +2
Score-based (denoising diffusion) generative models have recently gained a lot of success in generating realistic and diverse data. These approaches define a forward diffusion proc…
Adversarial score matching and improved sampling for image generation
Alexia Jolicoeur-Martineau, Rémi Piché-Taillefer, Rémi Tachet des Combes +1
Denoising Score Matching with Annealed Langevin Sampling (DSM-ALS) has recently found success in generative modeling. The approach works by first training a neural network to estim…
Stochastic Hamiltonian Gradient Methods for Smooth Games
Nicolas Loizou, Hugo Berard, Alexia Jolicoeur-Martineau +3
The success of adversarial formulations in machine learning has brought renewed motivation for smooth games. In this work, we focus on the class of stochastic Hamiltonian methods a…
Gradient penalty from a maximum margin perspective
Alexia Jolicoeur-Martineau, Ioannis Mitliagkas
A popular heuristic for improved performance in Generative adversarial networks (GANs) is to use some form of gradient penalty on the discriminator. This gradient penalty was origi…
On Relativistic -Divergences
Alexia Jolicoeur-Martineau
This paper provides a more rigorous look at Relativistic Generative Adversarial Networks (RGANs). We prove that the objective function of the discriminator is a statistical diverge…
GANs beyond divergence minimization
Alexia Jolicoeur-Martineau
Generative adversarial networks (GANs) can be interpreted as an adversarial game between two players, a discriminator D and a generator G, in which D learns to classify real from f…