Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models

Abstract

The rapid advancement of artificial intelligence (AI) technologies has led to an increasing deployment of AI models on edge and terminal devices, driven by the proliferation of the Internet of Things (IoT) and the need for real-time data processing. This survey comprehensively explores the current state, technical challenges, and future trends of on-device AI models. We define on-device AI models as those designed to perform local data processing and inference, emphasizing their characteristics such as real-time performance, resource constraints, and enhanced data privacy. The survey is structured around key themes, including the fundamental concepts of AI models, application scenarios across various domains, and the technical challenges faced in edge environments. We also discuss optimization and implementation strategies, such as data preprocessing, model compression, and hardware acceleration, which are essential for effective deployment. Furthermore, we examine the impact of emerging technologies, including edge computing and foundation models, on the evolution of on-device AI models. By providing a structured overview of the challenges, solutions, and future directions, this survey aims to facilitate further research and application of on-device AI, ultimately contributing to the advancement of intelligent systems in everyday life.

This paper has been accepted by ACM Computing Surveys

References in corpus (31)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Language Models are Few-Shot Learners
• Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
• HealthFog: An Ensemble Deep Learning based Smart Healthcare System for Automatic Diagnosis of Heart Diseases in Integrated IoT and Fog Computing Environments
• Machine Learning at the Network Edge: A Survey
• Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
• PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning
• Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• High-Throughput CNN Inference on Embedded ARM big.LITTLE Multi-Core Processors
• Edge Intelligence: Architectures, Challenges, and Applications
• DynaBERT: Dynamic BERT with Adaptive Width and Depth
• A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services
• Active label cleaning for improved dataset quality under resource constraints
• Once-for-All: Train One Network and Specialize it for Efficient Deployment
• FSpiNN: An Optimization Framework for Memory- and Energy-Efficient Spiking Neural Networks
• EdgeDRNN: Recurrent Neural Network Accelerator for Edge Inference
• Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AI
• Comparison and Benchmarking of AI Models and Frameworks on Mobile Devices
• Fusion-Catalyzed Pruning for Optimizing Deep Learning on Intelligent Edge Devices
• Mobile Foundation Model as Firmware
• FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training
• Mobile Edge Intelligence and Computing for the Internet of Vehicles
• Hidet: Task-Mapping Programming Paradigm for Deep Learning Tensor Programs
• EfficientTDNN: Efficient Architecture Search for Speaker Recognition
• MicroNet: Towards Image Recognition with Extremely Low FLOPs
• Fluid Batching: Exit-Aware Preemptive Serving of Early-Exit Neural Networks on Edge NPUs
• A Review on Edge Large Language Models: Design, Execution, and Applications
• Data Quality in Edge Machine Learning: A State-of-the-Art Survey

Cited by in corpus (4)

• Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review
• Comparative Evaluation of ChatGPT and DeepSeek Across Key NLP Tasks: Strengths, Weaknesses, and Domain-Specific Performance
• Mixture of Experts for Decentralized Generative AI and Reinforcement Learning in Wireless Networks: A Comprehensive Survey
• Hybrid Learning for Cold-Start-Aware Microservice Scheduling in Dynamic Edge Environments