5 papers
REVE: Regularizing Deep Learning with Variational Entropy Bound
Antoine Saporta, Yifu Chen, Michael Blot +1
Studies on generalization performance of machine learning algorithms under the scope of information theory suggest that compressed representations can guarantee good generalization…
A Characterization of Mean Squared Error for Estimator with Bagging
Martin Mihelich, Charles Dognin, Yan Shu +1
Bagging can significantly improve the generalization performance of unstable machine learning algorithms such as trees or neural networks. Though bagging is now widely used in prac…
SHADE: Information Based Regularization for Deep Learning
Michael Blot, Thomas Robert, Nicolas Thome +1
Regularization is a big issue for training deep neural networks. In this paper, we propose a new information-theory-based regularization scheme named SHADE for SHAnnon DEcay. The o…
SHADE: Information-Based Regularization for Deep Learning
Michael Blot, Thomas Robert, Nicolas Thome +1
Regularization is a big issue for training deep neural networks. In this paper, we propose a new information-theory-based regularization scheme named SHADE for SHAnnon DEcay. The o…
GoSGD: Distributed Optimization for Deep Learning with Gossip Exchange
Michael Blot, David Picard, Matthieu Cord
We address the issue of speeding up the training of convolutional neural networks by studying a distributed method adapted to stochastic gradient descent. Our parallel optimization…