High-Performance Neural Networks for Visual Object Classification

Abstract

We present a fast, fully parameterizable GPU implementation of Convolutional Neural Network variants. Our feature extractors are neither carefully designed nor pre-wired, but rather learned in a supervised way. Our deep hierarchical architectures achieve the best published results on benchmarks for object classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple back-propagation perform better than more shallow ones. Learning is surprisingly rapid. NORB is completely trained within five epochs. Test error rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs, respectively.

12 pages, 2 figures, 5 tables

Cited by in corpus (34)

• Striving for Simplicity: The All Convolutional Net
• Deep Learning using Linear Support Vector Machines
• Predicting Parameters in Deep Learning
• Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
• Compressing Neural Networks with the Hashing Trick
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• Spinal cord gray matter segmentation using deep dilated convolutions
• Improvements to context based self-supervised learning
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• Perceiver: General Perception with Iterative Attention
• Learning Invariant Representations with Local Transformations
• Structured Transforms for Small-Footprint Deep Learning
• The Power of Sparsity in Convolutional Neural Networks
• Dynamic Routing Between Capsules
• Data Augmentation in Emotion Classification Using Generative Adversarial Networks
• Automatic Data Augmentation for Generalization in Deep Reinforcement Learning
• The RAdial Velocity Experiment (RAVE): Parameterisation of RAVE spectra based on convolutional neural networks
• Rice Classification Using Spatio-Spectral Deep Convolutional Neural Network
• Machine Learning of Two-Dimensional Spectroscopic Data
• CHAOS: A Parallelization Scheme for Training Convolutional Neural Networks on Intel Xeon Phi
• Autoencoder-Driven Weather Clustering for Source Estimation during Nuclear Events
• Knowledge Projection for Deep Neural Networks
• An Open-source Tool for Hyperspectral Image Augmentation in Tensorflow
• Latent Model Ensemble with Auto-localization
• A Multigrid Method for Efficiently Training Video Models
• An Algorithm for Routing Capsules in All Domains
• Self-Supervised Representation Learning for Visual Anomaly Detection
• Continuous Transition: Improving Sample Efficiency for Continuous Control Problems via MixUp
• Developing a Bubble Chamber Particle Discriminator Using Semi-Supervised Learning
• Significance of Data Augmentation for Improving Cleft Lip and Palate Speech Recognition
• Automated Seed Quality Testing System using GAN & Active Learning
• Exponential Discriminative Metric Embedding in Deep Learning
• Learning Inward Scaled Hypersphere Embedding: Exploring Projections in Higher Dimensions
• Neural Network Structure Design based on N-Gauss Activation Function