FixyNN: Efficient Hardware for Mobile Computer Vision via Transfer Learning

Abstract

The computational demands of computer vision tasks based on state-of-the-art Convolutional Neural Network (CNN) image classification far exceed the energy budgets of mobile devices. This paper proposes FixyNN, which consists of a fixed-weight feature extractor that generates ubiquitous CNN features, and a conventional programmable CNN accelerator which processes a dataset-specific CNN. Image classification models for FixyNN are trained end-to-end via transfer learning, with the common feature extractor representing the transfered part, and the programmable part being learnt on the target dataset. Experimental results demonstrate FixyNN hardware can achieve very high energy efficiencies up to 26.6 TOPS/W ($4.81\times$ better than iso-area programmable accelerator). Over a suite of six datasets we trained models via transfer learning with an accuracy loss of $<1\%$ resulting in up to 11.2 TOPS/W - nearly $2\times$ more efficient than a conventional programmable CNN accelerator of the same area.

10 pages, 8 figures, paper accepted at SysML2019 conference

References in corpus (4)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• How transferable are features in deep neural networks?
• Fine-Grained Visual Classification of Aircraft
• Learning multiple visual domains with residual adapters

Cited by in corpus (12)

• MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
• LEEP: A New Measure to Evaluate Transferability of Learned Representations
• Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation
• Searching for Winograd-aware Quantized Networks
• EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference
• Eudoxus: Characterizing and Accelerating Localization in Autonomous Machines
• Sparse Systolic Tensor Array for Efficient CNN Hardware Acceleration
• ISP4ML: Understanding the Role of Image Signal Processing in Efficient Deep Learning Vision Systems
• Systolic Tensor Array: An Efficient Structured-Sparse GEMM Accelerator for Mobile CNN Inference
• A Machine Learning Imaging Core using Separable FIR-IIR Filters
• Doping: A technique for efficient compression of LSTM models using sparse structured additive matrices
• Can a powerful neural network be a teacher for a weaker neural network?