Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlaps
arXiv:1805.07798
Abstract
We propose a new algorithm to learn a one-hidden-layer convolutional neural network where both the convolutional weights and the outputs weights are parameters to be learned. Our algorithm works for a general class of (potentially overlapping) patches, including commonly used structures for computer vision tasks. Our algorithm draws ideas from (1) isotonic regression for learning neural networks and (2) landscape analysis of non-convex matrix factorization problems. We believe these findings may inspire further development in designing provable algorithms for learning neural networks and other complex models.
References in corpus (16)
- The loss surface of deep and wide neural networks
- On the Power of Over-parametrization in Neural Networks with Quadratic Activation
- Spurious Local Minima are Common in Two-Layer ReLU Neural Networks
- Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods
- Recovery Guarantees for One-hidden-layer Neural Networks
- Learning One-hidden-layer Neural Networks with Landscape Design
- Global Optimality in Tensor Factorization, Deep Learning, and Beyond
- Learning ReLUs via Gradient Descent
- An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis
- When is a Convolutional Filter Easy To Learn?
- Reliably Learning the ReLU in Polynomial Time
- Learning One Convolutional Layer with Overlapping Patches
- Learning Non-overlapping Convolutional Neural Networks with Multiple Kernels
- Learning Neural Networks with Two Nonlinear Layers in Polynomial Time
- Learning Halfspaces and Neural Networks with Random Initialization
- The Landscape of Deep Learning Algorithms