Deep Competitive Pathway Networks

Abstract

In the design of deep neural architectures, recent studies have demonstrated the benefits of grouping subnetworks into a larger network. For examples, the Inception architecture integrates multi-scale subnetworks and the residual network can be regarded that a residual unit combines a residual subnetwork with an identity shortcut. In this work, we embrace this observation and propose the Competitive Pathway Network (CoPaNet). The CoPaNet comprises a stack of competitive pathway units and each unit contains multiple parallel residual-type subnetworks followed by a max operation for feature competition. This mechanism enhances the model capability by learning a variety of features in subnetworks. The proposed strategy explicitly shows that the features propagate through pathways in various routing patterns, which is referred to as pathway encoding of category information. Moreover, the cross-block shortcut can be added to the CoPaNet to encourage feature reuse. We evaluated the proposed CoPaNet on four object recognition benchmarks: CIFAR-10, CIFAR-100, SVHN, and ImageNet. CoPaNet obtained the state-of-the-art or comparable results using similar amounts of parameters. The code of CoPaNet is available at: https://github.com/JiaRenChang/CoPaNet.

To appear in ACML17

References in corpus (11)

• Densely Connected Convolutional Networks
• Wide Residual Networks
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Residual Networks Behave Like Ensembles of Relatively Shallow Networks
• Aggregated Residual Transformations for Deep Neural Networks
• Deep Networks with Stochastic Depth
• Identity Mappings in Deep Residual Networks
• Batch-normalized Maxout Network in Network
• Multi-Residual Networks: Improving the Speed and Accuracy of Residual Networks
• PolyNet: A Pursuit of Structural Diversity in Very Deep Networks
• Understanding Locally Competitive Networks