927 citations · 2.7k across the 88 of their papers we have counts for
8 papers · 1 filter
Weight-Sharing Regularization
Mehran Shakerinava, Motahareh Sohrabi, Siamak Ravanbakhsh +1
Weight-sharing is ubiquitous in deep learning. Motivated by this, we propose a "weight-sharing regularization" penalty on the weights of a neural network, defi…
Balancing Act: Constraining Disparate Impact in Sparse Models
Meraj Hashemizadeh, Juan Ramirez, Rohan Sukumaran +3
Model pruning is a popular approach to enable the deployment of large deep learning models on edge devices with restricted computational or storage capacities. Although sparse mode…
Promoting Exploration in Memory-Augmented Adam using Critical Momenta
Pranshu Malviya, Gonçalo Mordido, Aristide Baratin +5
Adaptive gradient-based optimizers, notably Adam, have left their mark in training large-scale deep learning models, offering fast convergence and robustness to hyperparameter sett…
Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation
Sébastien Lachapelle, Divyat Mahajan, Ioannis Mitliagkas +1
We tackle the problems of latent variables identification and ``out-of-support'' image generation in representation learning. We show that both are possible for a class of decoders…
On the Identifiability of Quantized Factors
Vitória Barin-Pacela, Kartik Ahuja, Simon Lacoste-Julien +1
Disentanglement aims to recover meaningful latent ground-truth factors from the observed distribution solely, and is formalized through the theory of identifiability. The identifia…
PopulAtion Parameter Averaging (PAPA)
Alexia Jolicoeur-Martineau, Emy Gervais, Kilian Fatras +2
Ensemble methods combine the predictions of multiple models to improve performance, but they require significantly higher computation costs at inference time. To avoid these costs,…
Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?
Boris Knyazev, Doha Hwang, Simon Lacoste-Julien
Pretraining a neural network on a large dataset is becoming a cornerstone in machine learning that is within the reach of only a few communities with large-resources. We aim at an…
Unlocking Slot Attention by Changing Optimal Transport Costs
Yan Zhang, David W. Zhang, Simon Lacoste-Julien +2
Slot attention is a powerful method for object-centric modeling in images and videos. However, its set-equivariance limits its ability to handle videos with a dynamic number of obj…