1.1k citations · 1.4k across the 16 of their papers we have counts for
3 papers · 1 filter
A Closer Look at Codistillation for Distributed Training
Shagun Sodhani, Olivier Delalleau, Mahmoud Assran +3
Codistillation has been proposed as a mechanism to share knowledge among concurrently trained models by encouraging them to represent the same function through an auxiliary loss. T…
Supervision Accelerates Pre-training in Contrastive Semi-Supervised Learning of Visual Representations
Mahmoud Assran, Nicolas Ballas, Lluis Castrejon +1
We investigate a strategy for improving the efficiency of contrastive learning of visual representations by leveraging a small amount of supervised information during pre-training.…
On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings
Mahmoud Assran, Michael Rabbat
We study Nesterov's accelerated gradient method with constant step-size and momentum parameters in the stochastic approximation setting (unbiased gradients with bounded variance) a…