4 papers
SING: A Plug-and-Play DNN Learning Technique
Adrien Courtois, Damien Scieur, Jean-Michel Morel +2
We propose SING (StabIlized and Normalized Gradient), a plug-and-play technique that improves the stability and generalization of the Adam(W) optimizer. SING is straightforward to…
Improving Pixel-Level Contrastive Learning by Leveraging Exogenous Depth Information
Ahmed Ben Saad, Kristina Prokopetc, Josselin Kherroubi +3
Self-supervised representation learning based on Contrastive Learning (CL) has been the subject of much attention in recent years. This is due to the excellent results obtained on…
Can neural networks extrapolate? Discussion of a theorem by Pedro Domingos
Adrien Courtois, Jean-Michel Morel, Pablo Arias
Neural networks trained on large datasets by minimizing a loss have become the state-of-the-art approach for resolving data science problems, particularly in computer vision, image…
Investigating Neural Architectures by Synthetic Dataset Design
Adrien Courtois, Jean-Michel Morel, Pablo Arias
Recent years have seen the emergence of many new neural network structures (architectures and layers). To solve a given task, a network requires a certain set of abilities reflecte…