Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design

Abstract

Artificial intelligence (AI) technologies have dramatically advanced in recent years, resulting in revolutionary changes in people's lives. Empowered by edge computing, AI workloads are migrating from centralized cloud architectures to distributed edge systems, introducing a new paradigm called edge AI. While edge AI has the promise of bringing significant increases in autonomy and intelligence into everyday lives through common edge devices, it also raises new challenges, especially for the development of its algorithms and the deployment of its services, which call for novel design methodologies catered to these unique challenges. In this paper, we provide a comprehensive survey of the latest enabling design methodologies that span the entire edge AI development stack. We suggest that the key methodologies for effective edge AI development are single-layer specialization and cross-layer co-design. We discuss representative methodologies in each category in detail, including on-device training methods, specialized software design, dedicated hardware design, benchmarking and design automation, software/hardware co-design, software/compiler co-design, and compiler/hardware co-design. Moreover, we attempt to reveal hidden cross-layer design opportunities that can further boost the solution quality of future edge AI and provide insights into future directions and emerging areas that require increased research focus.

arXiv commentAccepted by IEEE Design & Test (D&T)

References in corpus (16)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• cuDNN: Efficient Primitives for Deep Learning
• Towards Federated Learning at Scale: System Design
• HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
• Channel Gating Neural Networks
• Edge Intelligence: Architectures, Challenges, and Applications
• Trading-off Accuracy and Energy of Deep Inference on Embedded Systems: A Co-Design Approach
• FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training
• XSP: Across-Stack Profiling and Analysis of Machine Learning Models on GPUs
• Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers
• Precision Gating: Improving Neural Network Efficiency with Dynamic Dual-Precision Activations
• CoCoPIE: Making Mobile AI Sweet As PIE --Compression-Compilation Co-Design Goes a Long Way
• Automated Design Space Exploration for optimised Deployment of DNN on Arm Cortex-A CPUs
• The Design and Implementation of a Scalable DL Benchmarking Platform