Lightweight Residual Densely Connected Convolutional Neural Network

Abstract

Extremely efficient convolutional neural network architectures are one of the most important requirements for limited-resource devices (such as embedded and mobile devices). The computing power and memory size are two important constraints of these devices. Recently, some architectures have been proposed to overcome these limitations by considering specific hardware-software equipment. In this paper, the lightweight residual densely connected blocks are proposed to guaranty the deep supervision, efficient gradient flow, and feature reuse abilities of convolutional neural network. The proposed method decreases the cost of training and inference processes without using any special hardware-software equipment by just reducing the number of parameters and computational operations while achieving a feasible accuracy. Extensive experimental results demonstrate that the proposed architecture is more efficient than the AlexNet and VGGNet in terms of model size, required parameters, and even accuracy. The proposed model has been evaluated on the ImageNet, MNIST, Fashion MNIST, SVHN, CIFAR-10, and CIFAR-100. It achieves state-of-the-art results on Fashion MNIST dataset and reasonable results on the others. The obtained results show the superiority of the proposed method to efficient models such as the SqueezNet. It is also comparable with state-of-the-art efficient models such as CondenseNet and ShuffleNet.

References in corpus (23)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Fully Convolutional Networks for Semantic Segmentation
• Squeeze-and-Excitation Networks
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Wide Residual Networks
• A Survey of Deep Learning Techniques for Autonomous Driving
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Training Very Deep Networks
• Random Erasing Data Augmentation
• Predicting Parameters in Deep Learning
• Pruning Filters for Efficient ConvNets
• Compressing Neural Networks with the Hashing Trick
• Highway Networks
• Designing Neural Network Architectures using Reinforcement Learning
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
• EIE: Efficient Inference Engine on Compressed Deep Neural Network
• Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey
• DENSER: Deep Evolutionary Network Structured Representation
• Residual Dense Network for Image Restoration
• Competitive Inner-Imaging Squeeze and Excitation for Residual Network

Cited by in corpus (2)

• Compacting Deep Neural Networks for Internet of Things: Methods and Applications
• Be Your Own Best Competitor! Multi-Branched Adversarial Knowledge Transfer