Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey

Abstract

Facial expressions convey human emotions and can be categorized into macro-expressions (MaEs) and micro-expressions (MiEs) based on duration and intensity. While MaEs are voluntary and easily recognized, MiEs are involuntary, rapid, and can reveal concealed emotions. The integration of facial expression analysis with Internet-of-Thing (IoT) systems has significant potential across diverse scenarios. IoT-enhanced MaE analysis enables real-time monitoring of patient emotions, facilitating improved mental health care in smart healthcare. Similarly, IoT-based MiE detection enhances surveillance accuracy and threat detection in smart security. Our work aims to provide a comprehensive overview of research progress in facial expression analysis and explores its potential integration with IoT systems. We discuss the distinctions between our work and existing surveys, elaborate on advancements in MaE and MiE analysis techniques across various learning paradigms, and examine their potential applications in IoT. We highlight challenges and future directions for the convergence of facial expression-based technologies and IoT systems, aiming to foster innovation in this domain. By presenting recent developments and practical applications, our work offers a systematic understanding of the ways of facial expression analysis to enhance IoT systems in healthcare, security, and beyond.

References in corpus (13)

• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Deep Facial Expression Recognition: A Survey
• Less is More: Micro-expression Recognition from Video using Apex Frame
• On Data Augmentation for GAN Training
• Towards Reading Hidden Emotions: A comparative Study of Spontaneous Micro-expression Spotting and Recognition Methods
• Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms
• Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks
• Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition
• LEARNet Dynamic Imaging Network for Micro Expression Recognition
• SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimation
• The Indian Spontaneous Expression Database for Emotion Recognition
• Self-supervised Contrastive Learning of Multi-view Facial Expressions
• MSSTNet: A Multi-Scale Spatio-Temporal CNN-Transformer Network for Dynamic Facial Expression Recognition