2k citations · 4.8k across the 35 of their papers we have counts for
3 papers · 1 filter
A Closer Look at Codistillation for Distributed Training
Shagun Sodhani, Olivier Delalleau, Mahmoud Assran +3
Codistillation has been proposed as a mechanism to share knowledge among concurrently trained models by encouraging them to represent the same function through an auxiliary loss. T…
Revisiting Loss Modelling for Unstructured Pruning
César Laurent, Camille Ballas, Thomas George +2
By removing parameters from deep neural networks, unstructured pruning methods aim at cutting down memory footprint and computational cost, while maintaining prediction accuracy. I…
Supervision Accelerates Pre-training in Contrastive Semi-Supervised Learning of Visual Representations
Mahmoud Assran, Nicolas Ballas, Lluis Castrejon +1
We investigate a strategy for improving the efficiency of contrastive learning of visual representations by leveraging a small amount of supervised information during pre-training.…