13 papers
Normalization Equivariance for Arbitrary Backbones, with Application to Image Denoising
Youssef Saied, François Fleuret
Normalization Equivariance (NE) is a structural prior that improves robustness to distribution shift in image-to-image tasks. A function is normalization equivariant iff $f(a y…
AA-SVD : Anchored and Adaptive SVD for Large Language Model Compression
Atul Kumar Sinha, François Fleuret
We introduce a fast low-rank factorization-based framework for compressing large language models that enables rapid compression of billion-parameter models without retraining. Unli…
Laminating Representation Autoencoders for Efficient Diffusion
Ramón Calvo-González, François Fleuret
Recent work has shown that diffusion models can generate high-quality images by operating directly on SSL patch features rather than pixel-space latents. However, the dense patch g…
Leveraging the true depth of LLMs
Ramón Calvo González, Daniele Paliotta, Matteo Pagliardini +2
The remarkable capabilities of Large Language Models (LLMs) are overshadowed by their immense computational cost. While recent work has shown that many LLM layers can be reordered…
The Free Transformer
François Fleuret
We propose an extension of the decoder Transformer that conditions its generative process on random latent variables which are learned without supervision thanks to a variational p…
Solvation Free Energies from Neural Thermodynamic Integration
Bálint Máté, François Fleuret, Tristan Bereau
We present a method for computing free-energy differences using thermodynamic integration with a neural network potential that interpolates between two target Hamiltonians. The int…