Gradient target propagation
arXiv:1810.09284
Abstract
We report a learning rule for neural networks that computes how much each neuron should contribute to minimize a giving cost function via the estimation of its target value. By theoretical analysis, we show that this learning rule contains backpropagation, Hebian learning, and additional terms. We also give a general technique for weights initialization. Our results are at least as good as those obtained with backpropagation. The neural networks are trained and tested in three problems: MNIST, MNIST-Fashion, and CIFAR-10 datasets. The associated code is available at https://github.com/tiago939/target.
arXiv comment12 pages, 4 figures
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- On weight initialization in deep neural networks
- How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation
- A Theory of Local Learning, the Learning Channel, and the Optimality of Backpropagation
- On the Selection of Initialization and Activation Function for Deep Neural Networks
- Hyperparameters Optimization in Deep Convolutional Neural Network / Bayesian Approach with Gaussian Process Prior
- Conducting Credit Assignment by Aligning Local Representations