FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computing
arXiv:2302.10681 · doi:10.1109/TMC.2024.3381952
Abstract
The rise of mobile AI accelerators allows latency-sensitive applications to execute lightweight Deep Neural Networks (DNNs) on the client side. However, critical applications require powerful models that edge devices cannot host and must therefore offload requests, where the high-dimensional data will compete for limited bandwidth. This work proposes shifting away from focusing on executing shallow layers of partitioned DNNs. Instead, it advocates concentrating the local resources on variational compression optimized for machine interpretability. We introduce a novel framework for resource-conscious compression models and extensively evaluate our method in an environment reflecting the asymmetric resource distribution between edge devices and servers. Our method achieves 60% lower bitrate than a state-of-the-art SC method without decreasing accuracy and is up to 16x faster than offloading with existing codec standards.
arXiv commentSubmission to IEEE Transactions on Mobile Computing
References in corpus (20)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Distilling the Knowledge in a Neural Network
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
- Striving for Simplicity: The All Convolutional Net
- The information bottleneck method
- FitNets: Hints for Thin Deep Nets
- Variational image compression with a scale hyperprior
- End-to-end Optimized Image Compression
- Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks
- Neural Image Compression via Non-Local Attention Optimization and Improved Context Modeling
- Non-local Attention Optimized Deep Image Compression
- Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges
- SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud
- CompressAI: a PyTorch library and evaluation platform for end-to-end compression research
- JALAD: Joint Accuracy- and Latency-Aware Deep Structure Decoupling for Edge-Cloud Execution
- Supervised Compression for Resource-Constrained Edge Computing Systems
- DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device
- torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation
- SC2 Benchmark: Supervised Compression for Split Computing