Fractional Max-Pooling

Abstract

Convolutional networks almost always incorporate some form of spatial pooling, and very often it is alpha times alpha max-pooling with alpha=2. Max-pooling act on the hidden layers of the network, reducing their size by an integer multiplicative factor alpha. The amazing by-product of discarding 75% of your data is that you build into the network a degree of invariance with respect to translations and elastic distortions. However, if you simply alternate convolutional layers with max-pooling layers, performance is limited due to the rapid reduction in spatial size, and the disjoint nature of the pooling regions. We have formulated a fractional version of max-pooling where alpha is allowed to take non-integer values. Our version of max-pooling is stochastic as there are lots of different ways of constructing suitable pooling regions. We find that our form of fractional max-pooling reduces overfitting on a variety of datasets: for instance, we improve on the state-of-the art for CIFAR-100 without even using dropout.

References in corpus (2)

• Going Deeper with Convolutions
• Spatially-sparse convolutional neural networks

Cited by in corpus (87)

• Deep Learning with Differential Privacy
• Striving for Simplicity: The All Convolutional Net
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Federated Learning with Non-IID Data
• Wide Residual Networks
• Temporal Ensembling for Semi-Supervised Learning
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Deep image mining for diabetic retinopathy screening
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• Models Genesis
• Grid Long Short-Term Memory
• Deep Networks with Stochastic Depth
• ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
• Stacked What-Where Auto-encoders
• Identity Mappings in Deep Residual Networks
• Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks
• Further advantages of data augmentation on convolutional neural networks
• Deep Residual Networks with Exponential Linear Unit
• Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree
• Fixup Initialization: Residual Learning Without Normalization
• Review: Deep Learning in Electron Microscopy
• Study of Different Deep Learning Approach with Explainable AI for Screening Patients with COVID-19 Symptoms: Using CT Scan and Chest X-ray Image Dataset
• Data augmentation instead of explicit regularization
• Regularizing CNNs with Locally Constrained Decorrelations
• Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks
• Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification
• Learning Feature Pyramids for Human Pose Estimation
• Toward high-performance online HCCR: a CNN approach with DropDistortion, path signature and spatial stochastic max-pooling
• A Survey on Deep Learning Methods for Robot Vision
• Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey
• Learning-Based Real-Time Event Identification Using Rich Real PMU Data
• Fuzzy Pooling
• Deep Pyramidal Residual Networks
• Resolution learning in deep convolutional networks using scale-space theory
• 1-Dimensional polynomial neural networks for audio signal related problems
• Automated Strabismus Detection for Telemedicine Applications
• PointAugment: an Auto-Augmentation Framework for Point Cloud Classification
• Improving adversarial robustness of deep neural networks by using semantic information
• Towards the AlexNet Moment for Homomorphic Encryption: HCNN, theFirst Homomorphic CNN on Encrypted Data with GPUs
• Optimization of Convolutional Neural Network using Microcanonical Annealing Algorithm
• Proof of Federated Learning: A Novel Energy-recycling Consensus Algorithm
• Active Convolution: Learning the Shape of Convolution for Image Classification
• LocalDrop: A Hybrid Regularization for Deep Neural Networks
• Sparse 3D convolutional neural networks
• Convolutional Residual Memory Networks
• Efficient ConvNets for Analog Arrays
• Ablation of a Robot's Brain: Neural Networks Under a Knife
• Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning
• Improving Transformation Invariance in Contrastive Representation Learning
• Automatic Extraction of Road Networks by using Teacher-Student Adaptive Structural Deep Belief Network and Its Application to Landslide Disaster
• Multipartite Pooling for Deep Convolutional Neural Networks
• Improving Adversarial Robustness in Weight-quantized Neural Networks
• Automatic Configuration of Deep Neural Networks with EGO
• Fitted Learning: Models with Awareness of their Limits
• CIFAR-10: KNN-based Ensemble of Classifiers
• Transfer entropy-based feedback improves performance in artificial neural networks
• The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs
• Rethinking Recurrent Neural Networks and Other Improvements for Image Classification
• Efficient batchwise dropout training using submatrices
• Convolution Aware Initialization
• Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation
• Parallel Grid Pooling for Data Augmentation
• Rectified Factor Networks
• DelugeNets: Deep Networks with Efficient and Flexible Cross-layer Information Inflows
• Maxmin convolutional neural networks for image classification
• Improving the Resolution of CNN Feature Maps Efficiently with Multisampling
• LIP: Local Importance-based Pooling
• Neural Style Representations and the Large-Scale Classification of Artistic Style
• A Survey of Techniques All Classifiers Can Learn from Deep Networks: Models, Optimizations, and Regularization
• Learning Less-Overlapping Representations
• S3Pool: Pooling with Stochastic Spatial Sampling
• Transfer Incremental Learning using Data Augmentation
• Design of Kernels in Convolutional Neural Networks for Image Classification
• A Fully Trainable Network with RNN-based Pooling
• Frequency Pooling: Shift-Equivalent and Anti-Aliasing Downsampling
• Feedback Learning for Improving the Robustness of Neural Networks
• LogAvgExp Provides a Principled and Performant Global Pooling Operator
• Integrating Multiple Receptive Fields through Grouped Active Convolution
• Feature Fusion Use Unsupervised Prior Knowledge to Let Small Object Represent
• DNN Quantization with Attention
• Meta-learning of Pooling Layers for Character Recognition
• Will there be a construction? Predicting road constructions based on heterogeneous spatiotemporal data
• ResumeNet: A Learning-based Framework for Automatic Resume Quality Assessment
• Recurrent Existence Determination Through Policy Optimization
• On the Exploration of Convolutional Fusion Networks for Visual Recognition
• Emergence of Selective Invariance in Hierarchical Feed Forward Networks