Optimization of neural networks via finite-value quantum fluctuations
arXiv:1807.00414
Abstract
We numerically test an optimization method for deep neural networks (DNNs) using quantum fluctuations inspired by quantum annealing. For efficient optimization, our method utilizes the quantum tunneling effect beyond the potential barriers. The path integral formulation of the DNN optimization generates an attracting force to simulate the quantum tunneling effect. In the standard quantum annealing method, the quantum fluctuations will vanish at the last stage of optimization. In this study, we propose a learning protocol that utilizes a finite value for quantum fluctuations strength to obtain higher generalization performance, which is a type of robustness. We demonstrate the performance of our method using two well-known open datasets: the MNIST dataset and the Olivetti face dataset. Although computational costs prevent us from testing our method on large datasets with high-dimensional data, results show that our method can enhance generalization performance by induction of the finite value for quantum fluctuations.
arXiv comment11 pages, 3 figures
References in corpus (15)
- ADADELTA: An Adaptive Learning Rate Method
- Quantum Annealing in the Transverse Ising Model
- Mathematical Foundation of Quantum Annealing
- Experimental Investigation of an Eight Qubit Unit Cell in a Superconducting Optimization Processor
- Quantum Speedup by Quantum Annealing
- A scalable control system for a superconducting adiabatic quantum optimization processor
- Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes
- Quantum annealing with antiferromagnetic fluctuations
- Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
- A scalable readout system for a superconducting adiabatic quantum optimization system
- Entropy-SGD: Biasing Gradient Descent Into Wide Valleys
- Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods
- Efficiency of quantum versus classical annealing in non-convex learning problems
- Quantum annealing with Jarzynski equality
- Localization in non-Hermitian quantum mechanics and flux-line pinning in superconductors