Computer Vision Model Compression Techniques for Embedded Systems: A Survey

Abstract

Deep neural networks have consistently represented the state of the art in most computer vision problems. In these scenarios, larger and more complex models have demonstrated superior performance to smaller architectures, especially when trained with plenty of representative data. With the recent adoption of Vision Transformer (ViT) based architectures and advanced Convolutional Neural Networks (CNNs), the total number of parameters of leading backbone architectures increased from 62M parameters in 2012 with AlexNet to 7B parameters in 2024 with AIM-7B. Consequently, deploying such deep architectures faces challenges in environments with processing and runtime constraints, particularly in embedded systems. This paper covers the main model compression techniques applied for computer vision tasks, enabling modern models to be used in embedded systems. We present the characteristics of compression subareas, compare different approaches, and discuss how to choose the best technique and expected variations when analyzing it on various embedded devices. We also share codes to assist researchers and new practitioners in overcoming initial implementation challenges for each subarea and present trends for Model Compression. Case studies for compression models are available at \href{https://github.com/venturusbr/cv-model-compression}{https://github.com/venturusbr/cv-model-compression}.

References in corpus (27)

• Distilling the Knowledge in a Neural Network
• Knowledge Distillation: A Survey
• LoRA: Low-Rank Adaptation of Large Language Models
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Rethinking the Value of Network Pruning
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Predicting Parameters in Deep Learning
• Knowledge Distillation by On-the-Fly Native Ensemble
• Quantization and Deployment of Deep Neural Networks on Microcontrollers
• Efficient Medical Image Segmentation Based on Knowledge Distillation
• Knowledge Distillation from A Stronger Teacher
• A Learning Framework for n-bit Quantized Neural Networks toward FPGAs
• Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer
• Enabling Mixed-Precision Quantized Neural Networks in Extreme-Edge Devices
• Structural Knowledge Distillation for Object Detection
• Knowledge Diffusion for Distillation
• Language model compression with weighted low-rank factorization
• Efficient Execution of Quantized Deep Learning Models: A Compiler Approach
• Structural Pruning for Diffusion Models
• Model Compression and Efficient Inference for Large Language Models: A Survey
• Benchmarking Quantized Neural Networks on FPGAs with FINN
• Self-supervised Video-centralised Transformer for Video Face Clustering
• Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey