Funnel Activation for Visual Recognition
arXiv:2007.11824
Abstract
We present a conceptually simple but effective funnel activation for image recognition tasks, called Funnel activation (FReLU), that extends ReLU and PReLU to a 2D activation by adding a negligible overhead of spatial condition. The forms of ReLU and PReLU are y = max(x, 0) and y = max(x, px), respectively, while FReLU is in the form of y = max(x,T(x)), where T(x) is the 2D spatial condition. Moreover, the spatial condition achieves a pixel-wise modeling capacity in a simple way, capturing complicated visual layouts with regular convolutions. We conduct experiments on ImageNet, COCO detection, and semantic segmentation tasks, showing great improvements and robustness of FReLU in the visual recognition tasks. Code is available at https://github.com/megvii-model/FunnelAct.
ECCV 2020
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Improving neural networks by preventing co-adaptation of feature detectors
- Neural Architecture Search with Reinforcement Learning
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Empirical Evaluation of Rectified Activations in Convolutional Network
- Conditional Image Generation with PixelCNN Decoders
- Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
- Searching for Activation Functions
- Object Detectors Emerge in Deep Scene CNNs
- Learning Activation Functions to Improve Deep Neural Networks