244 citations · 1.5k across the 70 of their papers we have counts for
10 papers · 1 filter
Jointly Learning Visual and Auditory Speech Representations from Raw Data
Alexandros Haliassos, Pingchuan Ma, Rodrigo Mira +2
We present RAVEn, a self-supervised multi-modal approach to jointly learn visual and auditory speech representations. Our pre-training objective involves encoding masked inputs, an…
FAN-Trans: Online Knowledge Distillation for Facial Action Unit Detection
Jing Yang, Jie Shen, Yiming Lin +2
Due to its importance in facial behaviour analysis, facial action unit (AU) detection has attracted increasing attention from the research community. Leveraging the online knowledg…
LA-VocE: Low-SNR Audio-visual Speech Enhancement using Neural Vocoders
Rodrigo Mira, Buye Xu, Jacob Donley +4
Audio-visual speech enhancement aims to extract clean speech from a noisy environment by leveraging not only the audio itself but also the target speaker's lip movements. This appr…
Streaming Audio-Visual Speech Recognition with Alignment Regularization
Pingchuan Ma, Niko Moritz, Stavros Petridis +2
In this work, we propose a streaming AV-ASR system based on a hybrid connectionist temporal classification (CTC)/attention neural network architecture. The audio and the visual enc…
SS-VAERR: Self-Supervised Apparent Emotional Reaction Recognition from Video
Marija Jegorova, Stavros Petridis, Maja Pantic
This work focuses on the apparent emotional reaction recognition (AERR) from the video-only input, conducted in a self-supervised fashion. The network is first pre-trained on diffe…
Training Strategies for Improved Lip-reading
Pingchuan Ma, Yujiang Wang, Stavros Petridis +2
Several training strategies and temporal models have been recently proposed for isolated word lip-reading in a series of independent works. However, the potential of combining the…