Temporal Convolution Networks with Positional Encoding for Evoked Expression Estimation

Abstract

This paper presents an approach for Evoked Expressions from Videos (EEV) challenge, which aims to predict evoked facial expressions from video. We take advantage of pre-trained models on large-scale datasets in computer vision and audio signals to extract the deep representation of timestamps in the video. A temporal convolution network, rather than an RNN like architecture, is used to explore temporal relationships due to its advantage in memory consumption and parallelism. Furthermore, to address the missing annotations of some timestamps, positional encoding is employed to ensure continuity of input data when discarding these timestamps during training. We achieved state-of-the-art results on the EEV challenge with a Pearson correlation coefficient of 0.05477, the first ranked performance in the EEV 2021 challenge.

Oral presentation at AUVi Workshop - CVPR 2021 (https://sites.google.com/view/auvi-cvpr2021/program). Source code available at https://github.com/th2l/EvokedExpression-tcnpe

References in corpus (6)

• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Robust Real-World Image Super-Resolution against Adversarial Attacks
• Towards Learning a Universal Non-Semantic Representation of Speech
• EEV: A Large-Scale Dataset for Studying Evoked Expressions from Video
• Less is More: Sparse Sampling for Dense Reaction Predictions
• Multi-Granularity Network with Modal Attention for Dense Affective Understanding