A guide to convolution arithmetic for deep learning

Abstract

We introduce a guide to help deep learning practitioners understand and manipulate convolutional neural network architectures. The guide clarifies the relationship between various properties (input shape, kernel shape, zero padding, strides and output shape) of convolutional, pooling and transposed convolutional layers, as well as the relationship between convolutional and transposed convolutional layers. Relationships are derived for various cases, and are illustrated in order to make them intuitive.

Cited by in corpus (214)

• Deep Learning for Single Image Super-Resolution: A Brief Review
• Rapid identification of pathogenic bacteria using Raman spectroscopy and deep learning
• Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation
• TractSeg - Fast and accurate white matter tract segmentation
• Adversarially Learned Inference
• Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
• A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems
• Perceptual Adversarial Networks for Image-to-Image Transformation
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Machine Learning in Aerodynamic Shape Optimization
• Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors
• MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
• Fully Convolutional Networks with Sequential Information for Robust Crop and Weed Detection in Precision Farming
• A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
• Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
• Regularization for Deep Learning: A Taxonomy
• Combined tract segmentation and orientation mapping for bundle-specific tractography
• Towards a Robust Parameterization for Conditioning Facies Models Using Deep Variational Autoencoders and Ensemble Smoother
• Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics
• An Introduction to Image Synthesis with Generative Adversarial Nets
• A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU
• Unsupervised Predictive Memory in a Goal-Directed Agent
• Data Augmentation of Spectral Data for Convolutional Neural Network (CNN) Based Deep Chemometrics
• Convolutional neural networks for fluid flow analysis: toward effective metamodeling and low-dimensionalization
• Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks
• STConvS2S: Spatiotemporal Convolutional Sequence to Sequence Network for Weather Forecasting
• EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking
• Is the deconvolution layer the same as a convolutional layer?
• Texture Synthesis with Spatial Generative Adversarial Networks
• Review: Deep Learning in Electron Microscopy
• Quaternion Convolutional Neural Networks for Heterogeneous Image Processing
• Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning
• Noise Reduction in Gravitational-wave Data via Deep Learning
• A convolutional neural network approach to deblending seismic data
• A Comparison of Neural Network Architectures for Data-Driven Reduced-Order Modeling
• Actionable and Interpretable Fault Localization for Recurring Failures in Online Service Systems
• Learning to Denoise Astronomical Images with U-nets
• How deep is deep enough? -- Quantifying class separability in the hidden layers of deep neural networks
• Poisson CNN: Convolutional neural networks for the solution of the Poisson equation on a Cartesian mesh
• End-to-End Pixel-Based Deep Active Inference for Body Perception and Action
• Deep Semantic Segmentation in an AUV for Online Posidonia Oceanica Meadows identification
• Deep Learning for Plasma Tomography and Disruption Prediction from Bolometer Data
• Generative Modeling of Turbulence
• Forecasting Megaelectron-Volt Electrons inside Earth's Outer Radiation Belt: PreMevE 2.0 Based on Supervised Machine Learning Algorithms
• Deep Predictive Coding Network for Object Recognition
• Segmentation of coronal holes in solar disk images with a convolutional neural network
• Characterizing the Decision Boundary of Deep Neural Networks
• AirLab: Autograd Image Registration Laboratory
• Machine learning astrophysics from 21 cm lightcones: impact of network architectures and signal contamination
• A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA
• Improving Electron Micrograph Signal-to-Noise with an Atrous Convolutional Encoder-Decoder
• Robust Iris Segmentation Based on Fully Convolutional Networks and Generative Adversarial Networks
• TasNet: time-domain audio separation network for real-time, single-channel speech separation
• Deep learning approach for identification of HII regions during reionization in 21-cm observations
• RiemannONets: Interpretable Neural Operators for Riemann Problems
• Machine Learning Architectures for the Estimation of Predicted Occupancy Grids in Road Traffic
• CASI: A Convolutional Neural Network Approach for Shell Identification
• InGAN: Capturing and Remapping the "DNA" of a Natural Image
• CHS-Net: A Deep learning approach for hierarchical segmentation of COVID-19 infected CT images
• Full-pulse Tomographic Reconstruction with Deep Neural Networks
• High-fidelity Prediction of Megapixel Longitudinal Phase-space Images of Electron Beams using Encoder-Decoder Neural Networks
• Alleviating the transit timing variation bias in transit surveys. I. RIVERS: Method and detection of a pair of resonant super-Earths around Kepler-1705
• Fully Automated Segmentation of Hyperreflective Foci in Optical Coherence Tomography Images
• On the Number of Linear Regions of Convolutional Neural Networks
• AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search
• Self-Supervised Deep Pose Corrections for Robust Visual Odometry
• Learning Uncertain Convolutional Features for Accurate Saliency Detection
• Binary Document Image Super Resolution for Improved Readability and OCR Performance
• Convolutional-network models to predict wall-bounded turbulence from wall quantities
• Neural Network Evolution Strategy for Solving Quantum Sign Structures
• End-to-end Deep Learning from Raw Sensor Data: Atrial Fibrillation Detection using Wearables
• Fully Automated 2D and 3D Convolutional Neural Networks Pipeline for Video Segmentation and Myocardial Infarction Detection in Echocardiography
• Fast Generation for Convolutional Autoregressive Models
• Combining Convolution and Recursive Neural Networks for Sentiment Analysis
• Reconstructing Patchy Reionization with Deep Learning
• Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net
• Revealing Fundamental Physics from the Daya Bay Neutrino Experiment using Deep Neural Networks
• Convolution, attention and structure embedding
• Generalised gravitational burst generation with Generative Adversarial Networks
• A semi-supervised approach to dark matter searches in direct detection data with machine learning
• A Bayesian Deep Learning Approach to Near-Term Climate Prediction
• Deep UL2DL: Channel Knowledge Transfer from Uplink to Downlink
• Deep Learning Architecture Based Approach For 2D-Simulation of Microwave Plasma Interaction
• Radio source-component association for the LOFAR Two-metre Sky Survey with region-based convolutional neural networks
• Prototype-based Neural Network Layers: Incorporating Vector Quantization
• Density of photonic states in aperiodic structures
• Deep Generative Adversarial Neural Networks for Realistic Prostate Lesion MRI Synthesis
• Setting an attention region for convolutional neural networks using region selective features, for recognition of materials within glass vessels
• Pulsar Candidate Identification with Artificial Intelligence Techniques
• Hierarchical Associative Memory
• Eformer: Edge Enhancement based Transformer for Medical Image Denoising
• Machine Learning and Polymer Self-Consistent Field Theory in Two Spatial Dimensions
• Uniformly convex neural networks and non-stationary iterated network Tikhonov (iNETT) method
• Statistics of Deep Generated Images
• VAEs in the Presence of Missing Data
• A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasets
• Using deep generative neural networks to account for model errors in Markov chain Monte Carlo inversion
• CNN2Gate: Toward Designing a General Framework for Implementation of Convolutional Neural Networks on FPGA
• Sparse Weight Activation Training
• Order and Chaos: NTK views on DNN Normalization, Checkerboard and Boundary Artifacts
• Overview of image-to-image translation by use of deep neural networks: denoising, super-resolution, modality conversion, and reconstruction in medical imaging
• The Impact of Background Removal on Performance of Neural Networks for Fashion Image Classification and Segmentation
• A Continuous Convolutional Trainable Filter for Modelling Unstructured Data
• Machine Learning for Detection of 3D Features using sparse X-ray data
• Applications of Deep Learning to Nuclear Fusion Research
• SIRe-Networks: Convolutional Neural Networks Architectural Extension for Information Preservation via Skip/Residual Connections and Interlaced Auto-Encoders
• COALA: Co-Aligned Autoencoders for Learning Semantically Enriched Audio Representations
• The Devil is in the Decoder: Classification, Regression and GANs
• A Competitive Edge: Can FPGAs Beat GPUs at DCNN Inference Acceleration in Resource-Limited Edge Computing Applications?
• Learning to Reconstruct Confocal Microscopy Stacks from Single Light Field Images
• PolSAR Image Classification Based on Dilated Convolution and Pixel-Refining Parallel Mapping network in the Complex Domain
• Rethinking Floating Point Overheads for Mixed Precision DNN Accelerators
• HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular Multi-Person 3D Pose Estimation
• Deep Learning in Information Security
• CSTNet: A Dual-Branch Convolutional Network for Imaging of Reactive Flows using Chemical Species Tomography
• Search for the edge-on galaxies using an artificial neural network
• Autoencoder based image compression: can the learning be quantization independent?
• Blind Source Separation of Radar Signals in Time Domain Using Deep Learning
• Predicting CMB dust foreground using galactic 21 cm data
• Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter
• Learnable Descent Algorithm for Nonsmooth Nonconvex Image Reconstruction
• Stochastic Adversarial Koopman Model for Dynamical Systems
• Attention-based Image Upsampling
• Graph Autoencoders with Deconvolutional Networks
• Spatial-temporal wind field prediction by Artificial Neural Networks
• Hierarchical semantic segmentation using modular convolutional neural networks
• Adversarial Image Alignment and Interpolation
• Sketch2code: Generating a website from a paper mockup
• DEEPCHORUS: A Hybrid Model of Multi-scale Convolution and Self-attention for Chorus Detection
• FlatteNet: A Simple Versatile Framework for Dense Pixelwise Prediction
• Deep Generalized Convolutional Sum-Product Networks
• Using machine learning to study the kinematics of cold gas in galaxies
• Pelta: Shielding Transformers to Mitigate Evasion Attacks in Federated Learning
• Deformable Image Registration using Unsupervised Deep Learning for CBCT-guided Abdominal Radiotherapy
• A multi-path 2.5 dimensional convolutional neural network system for segmenting stroke lesions in brain MRI images
• Semantic Segmentation of Skin Lesions using a Small Data Set
• Towards Disentangled Representations for Human Retargeting by Multi-view Learning
• A Foreground Inference Network for Video Surveillance Using Multi-View Receptive Field
• Anti-aliasing Deep Image Classifiers using Novel Depth Adaptive Blurring and Activation Function
• Deep learning for pedestrians: backpropagation in CNNs
• Detection of Deepfake Videos Using Long Distance Attention
• Facial Expression Editing with Continuous Emotion Labels
• A Deep Factorization of Style and Structure in Fonts
• Deep Adversarial Koopman Model for Reaction-Diffusion systems
• SAVERS: SAR ATR with Verification Support Based on Convolutional Neural Network
• A Frobenius norm regularization method for convolutional kernels to avoid unstable gradient problem
• Walking the Tightrope: An Investigation of the Convolutional Autoencoder Bottleneck
• Deep Learning for Robust Motion Segmentation with Non-Static Cameras
• NeRV: Neural Representations for Videos
• Convolutional Recurrent Residual U-Net Embedded with Attention Mechanism and Focal Tversky Loss Function for Cancerous Nuclei Detection
• Real-world Video Deblurring: A Benchmark Dataset and An Efficient Recurrent Neural Network
• Object Recognition for Economic Development from Daytime Satellite Imagery
• UWSpeech: Speech to Speech Translation for Unwritten Languages
• Spurious samples in deep generative models: bug or feature?
• Infinite-channel deep stable convolutional neural networks
• A Novel Learnable Gradient Descent Type Algorithm for Non-convex Non-smooth Inverse Problems
• Pores for thought: The use of generative adversarial networks for the stochastic reconstruction of 3D multi-phase electrode microstructures with periodic boundaries
• Rethinking Image-Scaling Attacks: The Interplay Between Vulnerabilities in Machine Learning Systems
• Learning scale-variant features for robust iris authentication with deep learning based ensemble framework
• On the Robustness and Generalization of Deep Learning Driven Full Waveform Inversion
• Adaptive Dilated Convolution For Human Pose Estimation
• Automatic Recognition of the Supraspinatus Tendinopathy from Ultrasound Images using Convolutional Neural Networks
• HybridNet: Classification and Reconstruction Cooperation for Semi-Supervised Learning
• Pressure Predictions of Turbine Blades with Deep Learning
• Evaluating Transformer-Based Multilingual Text Classification
• A Light-Weighted Convolutional Neural Network for Bitemporal SAR Image Change Detection
• A Differential Game Theoretic Neural Optimizer for Training Residual Networks
• Make Your Bone Great Again : A study on Osteoporosis Classification
• Learning A 3D-CNN and Transformer Prior for Hyperspectral Image Super-Resolution
• Hybrid Distortion Aggregated Visual Comfort Assessment for Stereoscopic Image Retargeting
• Building Compact and Robust Deep Neural Networks with Toeplitz Matrices
• Unsupervised Object Learning via Common Fate
• DeepCore: Convolutional Neural Network for high $p_T$ jet tracking
• Data-Driven Construction of Data Center Graph of Things for Anomaly Detection
• On regularization for a convolutional kernel in neural networks
• Reproduction of Lateral Inhibition-Inspired Convolutional Neural Network for Visual Attention and Saliency Detection
• An analysis of full-size Russian complexly NER labelled corpus of Internet user reviews on the drugs based on deep learning and language neural nets
• Provably Convergent Learned Inexact Descent Algorithm for Low-Dose CT Reconstruction
• Unsupervised Domain Alignment to Mitigate Low Level Dataset Biases
• LLM4Laser: Large Language Models Automate the Design of Lasers
• Handwritten Character Recognition from Wearable Passive RFID
• Sensitivity Analysis of Lift and Drag Coefficients for Flow over Elliptical Cylinders of Arbitrary Aspect Ratio and Angle of Attack using Neural Network
• Novel and Effective CNN-Based Binarization for Historically Degraded As-built Drawing Maps
• Optimizations of Autoencoders for Analysis and Classification of Microscopic In Situ Hybridization Images
• HUGE2: a Highly Untangled Generative-model Engine for Edge-computing
• Hyperspectral imaging with Raman scattered photons: A new paradigm in Raman analysis
• Modular Block-diagonal Curvature Approximations for Feedforward Architectures
• Finite Difference Nets: A Deep Recurrent Framework for Solving Evolution PDEs
• Deployment Optimization for Meta-material Based Internet of Things
• Simple Multi-Resolution Representation Learning for Human Pose Estimation
• Intrusive deconvolutional neural networks for enhancing PIC/FLIP solutions
• BitNet: Learning-Based Bit-Depth Expansion
• Revising FUNSD dataset for key-value detection in document images
• Cross-filter compression for CNN inference acceleration
• Comb Convolution for Efficient Convolutional Architecture
• Deep Generative Variational Autoencoding for Replay Spoof Detection in Automatic Speaker Verification
• A new smart-cropping pipeline for prostate segmentation using deep learning networks
• Towards Design Methodology of Efficient Fast Algorithms for Accelerating Generative Adversarial Networks on FPGAs
• Sinogram Denoise Based on Generative Adversarial Networks
• Coarse-to-fine Optimization for Speech Enhancement
• Deep learning networks for selection of persistent scatterer pixels in multi-temporal SAR interferometric processing
• Fully Automated 3D Segmentation of MR-Imaged Calf Muscle Compartments: Neighborhood Relationship Enhanced Fully Convolutional Network
• Automated airway segmentation by learning graphical structure
• Orthogonally Regularized Deep Networks For Image Super-resolution
• Computational Long Exposure Mobile Photography
• Convolutional encoder decoder network for the removal of coherent seismic noise
• State Representation Learning from Demonstration
• A Semi-Supervised Approach for Automatic Crystal Structure Classification
• A Study on Trees's Knots Prediction from their Bark Outer-Shape
• A Brief Survey and an Application of Semantic Image Segmentation for Autonomous Driving
• Regularization for convolutional kernel tensors to avoid unstable gradient problem in convolutional neural networks
• Modeling of Rakugo Speech and Its Limitations: Toward Speech Synthesis That Entertains Audiences
• A Survey On 3D Inner Structure Prediction from its Outer Shape
• dictNN: A Dictionary-Enhanced CNN Approach for Classifying Hate Speech on Twitter