Data Leakage and Evaluation Issues in Micro-Expression Analysis
arXiv:2211.11425 · doi:10.1109/TAFFC.2023.3265063
Abstract
Micro-expressions have drawn increasing interest lately due to various potential applications. The task is, however, difficult as it incorporates many challenges from the fields of computer vision, machine learning and emotional sciences. Due to the spontaneous and subtle characteristics of micro-expressions, the available training and testing data are limited, which make evaluation complex. We show that data leakage and fragmented evaluation protocols are issues among the micro-expression literature. We find that fixing data leaks can drastically reduce model performance, in some cases even making the models perform similarly to a random classifier. To this end, we go through common pitfalls, propose a new standardized evaluation protocol using facial action units with over 2000 micro-expression samples, and provide an open source library that implements the evaluation protocols in a standardized manner. Code is publicly available in \url{https://github.com/tvaranka/meb}.
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
- Array Programming with NumPy
- Towards Reading Hidden Emotions: A comparative Study of Spontaneous Micro-expression Spotting and Recognition Methods
- Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms
- Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition
- Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition
- A Survey of Automatic Facial Micro-expression Analysis: Databases, Methods and Challenges
- Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition
- Leakage and the Reproducibility Crisis in ML-based Science