AttendSeg: A Tiny Attention Condenser Neural Network for Semantic Segmentation on the Edge
arXiv:2104.14623
Abstract
In this study, we introduce \textbf{AttendSeg}, a low-precision, highly compact deep neural network tailored for on-device semantic segmentation. AttendSeg possesses a self-attention network architecture comprising of light-weight attention condensers for improved spatial-channel selective attention at a very low complexity. The unique macro-architecture and micro-architecture design properties of AttendSeg strike a strong balance between representational power and efficiency, achieved via a machine-driven design exploration strategy tailored specifically for the task at hand. Experimental results demonstrated that the proposed AttendSeg can achieve segmentation accuracy comparable to much larger deep neural networks with greater complexity while possessing a significantly lower architecture and computational complexity (requiring as much as >27x fewer MACs, >72x fewer parameters, and >288x lower weight memory requirements), making it well-suited for TinyML applications on the edge.
arXiv comment5 pages
References in corpus (14)
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Deep Residual Learning for Image Recognition
- Pyramid Scene Parsing Network
- MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
- ProjectionNet: Learning Efficient On-Device Deep Networks Using Neural Projections
- Tiny SSD: A Tiny Single-shot Detection Deep Convolutional Neural Network for Real-time Embedded Object Detection
- Two-Bit Networks for Deep Learning on Resource-Constrained Embedded Devices
- TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devices
- SquishedNets: Squishing SqueezeNet further for edge device scenarios via deep evolutionary synthesis
- AttendNets: Tiny Deep Image Recognition Neural Networks for the Edge via Visual Attention Condensers
- Implications of Computer Vision Driven Assistive Technologies Towards Individuals with Visual Impairment
- EdgeSegNet: A Compact Network for Semantic Segmentation