Agile Amulet: Real-Time Salient Object Detection with Contextual Attention

Abstract

This paper proposes an Agile Aggregating Multi-Level feaTure framework (Agile Amulet) for salient object detection. The Agile Amulet builds on previous works to predict saliency maps using multi-level convolutional features. Compared to previous works, Agile Amulet employs some key innovations to improve training and testing speed while also increase prediction accuracy. More specifically, we first introduce a contextual attention module that can rapidly highlight most salient objects or regions with contextual pyramids. Thus, it effectively guides the learning of low-layer convolutional features and tells the backbone network where to look. The contextual attention module is a fully convolutional mechanism that simultaneously learns complementary features and predicts saliency scores at each pixel. In addition, we propose a novel method to aggregate multi-level deep convolutional features. As a result, we are able to use the integrated side-output features of pre-trained convolutional networks alone, which significantly reduces the model parameters leading to a model size of 67 MB, about half of Amulet. Compared to other deep learning based saliency methods, Agile Amulet is of much lighter-weight, runs faster (30 fps in real-time) and achieves higher performance on seven public benchmarks in terms of both quantitative and qualitative evaluation.

arXiv comment10 pages, 4 figures and 3 tables

References in corpus (22)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Deep Residual Learning for Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Densely Connected Convolutional Networks
• Multi-Scale Context Aggregation by Dilated Convolutions
• Hierarchical Question-Image Co-Attention for Visual Question Answering
• ParseNet: Looking Wider to See Better
• Recurrent Models of Visual Attention
• Fully Convolutional Networks for Semantic Segmentation
• Multiple Object Recognition with Visual Attention
• DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection
• Pyramid Scene Parsing Network
• ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
• Visual Saliency Based on Multiscale Deep Features
• Stacked Attention Networks for Image Question Answering
• What is a salient object? A dataset and a baseline model for salient object detection
• Deep Contrast Learning for Salient Object Detection
• Attention for Fine-Grained Categorization
• Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection
• Aligning where to see and what to tell: image caption with region-based attention and scene factorization
• Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering

Cited by in corpus (5)

• Salient Object Detection with Lossless Feature Reflection and Weighted Structural Loss
• Towards High-Resolution Salient Object Detection
• Cascaded Context Pyramid for Full-Resolution 3D Semantic Scene Completion
• Adaptive Fusion for RGB-D Salient Object Detection
• Self-Attention Recurrent Network for Saliency Detection