4 papers
Module-wise Training of Neural Networks via the Minimizing Movement Scheme
Skander Karkar, Ibrahim Ayed, Emmanuel de Bézenac +1
Greedy layer-wise or module-wise training of neural networks is compelling in constrained and on-device settings where memory is limited, as it circumvents a number of problems of…
Adversarial Sample Detection Through Neural Network Transport Dynamics
Skander Karkar, Patrick Gallinari, Alain Rakotomamonjy
We propose a detector of adversarial samples that is based on the view of neural networks as discrete dynamic systems. The detector tells clean inputs from abnormal ones by compari…
Block-wise Training of Residual Networks via the Minimizing Movement Scheme
Skander Karkar, Ibrahim Ayed, Emmanuel de Bézenac +1
End-to-end backpropagation has a few shortcomings: it requires loading the entire model during training, which can be impossible in constrained settings, and suffers from three loc…
A Principle of Least Action for the Training of Neural Networks
Skander Karkar, Ibrahim Ayed, Emmanuel de Bézenac +1
Neural networks have been achieving high generalization performance on many tasks despite being highly over-parameterized. Since classical statistical learning theory struggles to…