ARMA Nets: Expanding Receptive Field for Dense Prediction

Abstract

Global information is essential for dense prediction problems, whose goal is to compute a discrete or continuous label for each pixel in the images. Traditional convolutional layers in neural networks, initially designed for image classification, are restrictive in these problems since the filter size limits their receptive fields. In this work, we propose to replace any traditional convolutional layer with an autoregressive moving-average (ARMA) layer, a novel module with an adjustable receptive field controlled by the learnable autoregressive coefficients. Compared with traditional convolutional layers, our ARMA layer enables explicit interconnections of the output neurons and learns its receptive field by adapting the autoregressive coefficients of the interconnections. ARMA layer is adjustable to different types of tasks: for tasks where global information is crucial, it is capable of learning relatively large autoregressive coefficients to allow for an output neuron's receptive field covering the entire input; for tasks where only local information is required, it can learn small or near zero autoregressive coefficients and automatically reduces to a traditional convolutional layer. We show both theoretically and empirically that the effective receptive field of networks with ARMA layers (named as ARMA networks) expands with larger autoregressive coefficients. We also provably solve the instability problem of learning and prediction in the ARMA layer through a re-parameterization mechanism. Additionally, we demonstrate that ARMA networks substantially improve their baselines on challenging dense prediction tasks including video prediction and semantic segmentation.

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Attention U-Net: Learning Where to Look for the Pancreas
• Spectral Normalization for Generative Adversarial Networks
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Pixel Recurrent Neural Networks
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• Grid Long Short-Term Memory
• Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation
• Highly Efficient Forward and Backward Propagation of Convolutional Neural Networks for Pixelwise Classification
• Understanding Convolution for Semantic Segmentation