Deep Global-Connected Net With The Generalized Multi-Piecewise ReLU Activation in Deep Learning
arXiv:1807.03116
Abstract
Recent Progress has shown that exploitation of hidden layer neurons in convolution neural networks incorporating with a carefully designed activation function can yield better classification results in the field of computer vision. The paper firstly introduces a novel deep learning architecture aiming to mitigate the gradient-vanishing problem, in which the earlier hidden layer neurons could be directly connected with the last hidden layer and feed into the last layer for classification. We then design a generalized linear rectifier function as the activation function that can approximate arbitrary complex functions via training of the parameters. We will show that our design can achieve similar performance in a number of object recognition and video action benchmark tasks, under significantly less number of parameters and shallower network infrastructure, which is not only promising in training in terms of computation burden and memory usage, but is also applicable to low-computation, low-memory mobile scenarios.
arXiv comment9 pages, 3 figures and 5 tables
References in corpus (17)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Densely Connected Convolutional Networks
- Wide Residual Networks
- Training Very Deep Networks
- Deeply-Supervised Nets
- Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
- Resnet in Resnet: Generalizing Residual Architectures
- FractalNet: Ultra-Deep Neural Networks without Residuals
- Stochastic Pooling for Regularization of Deep Convolutional Neural Networks
- Convolutional Kernel Networks
- Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification
- Deep Learning with S-shaped Rectified Linear Activation Units
- Improving Deep Neural Networks with Probabilistic Maxout Units
- Deeply-Fused Nets