DCFNet: Deep Neural Network with Decomposed Convolutional Filters

Abstract

Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.

arXiv commentPublished at ICML 2018

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Densely Connected Convolutional Networks
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Compressing Neural Networks with the Hashing Trick
• Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
• The Power of Sparsity in Convolutional Neural Networks

Cited by in corpus (12)

• ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning
• RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks
• A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data using Unstructured Spatial Discretizations
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
• Structured Convolutions for Efficient Neural Network Design
• Graph Convolution with Low-rank Learnable Local Filters
• ACDC: Weight Sharing in Atom-Coefficient Decomposed Convolution
• Stochastic Conditional Generative Networks with Basis Decomposition
• A Dictionary Approach to Domain-Invariant Learning in Deep Networks
• Deformation Robust Roto-Scale-Translation Equivariant CNNs
• AdjointBackMapV2: Precise Reconstruction of Arbitrary CNN Unit's Activation via Adjoint Operators
• Deeply Shared Filter Bases for Parameter-Efficient Convolutional Neural Networks