A Data and Compute Efficient Design for Limited-Resources Deep Learning

Abstract

Thanks to their improved data efficiency, equivariant neural networks have gained increased interest in the deep learning community. They have been successfully applied in the medical domain where symmetries in the data can be effectively exploited to build more accurate and robust models. To be able to reach a much larger body of patients, mobile, on-device implementations of deep learning solutions have been developed for medical applications. However, equivariant models are commonly implemented using large and computationally expensive architectures, not suitable to run on mobile devices. In this work, we design and test an equivariant version of MobileNetV2 and further optimize it with model quantization to enable more efficient inference. We achieve close-to state of the art performance on the Patch Camelyon (PCam) medical dataset while being more computationally efficient.

arXiv commentAccepted for poster presentation at the Practical Machine Learning for Developing Countries (PML4DC) workshop, ICLR 2020

References in corpus (15)

• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Group Equivariant Convolutional Networks
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Exploiting Cyclic Symmetry in Convolutional Neural Networks
• General $E(2)$-Equivariant Steerable CNNs
• 3D G-CNNs for Pulmonary Nodule Detection
• Deep Scale-spaces: Equivariance Over Scale
• Attentive Group Equivariant Convolutional Networks
• Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization
• ZeroQ: A Novel Zero Shot Quantization Framework
• Relaxed Quantization for Discretized Neural Networks
• Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks
• Sample Efficient Semantic Segmentation using Rotation Equivariant Convolutional Networks
• Deeply Supervised Rotation Equivariant Network for Lesion Segmentation in Dermoscopy Images

Cited by in corpus (2)

• CoAtNet: Marrying Convolution and Attention for All Data Sizes
• Translational Equivariance in Kernelizable Attention