158 citations · 672 across the 57 of their papers we have counts for
9 papers · 1 filter
Jointly Learning Visual and Auditory Speech Representations from Raw Data
Alexandros Haliassos, Pingchuan Ma, Rodrigo Mira +2
We present RAVEn, a self-supervised multi-modal approach to jointly learn visual and auditory speech representations. Our pre-training objective involves encoding masked inputs, an…
LA-VocE: Low-SNR Audio-visual Speech Enhancement using Neural Vocoders
Rodrigo Mira, Buye Xu, Jacob Donley +4
Audio-visual speech enhancement aims to extract clean speech from a noisy environment by leveraging not only the audio itself but also the target speaker's lip movements. This appr…
Streaming Audio-Visual Speech Recognition with Alignment Regularization
Pingchuan Ma, Niko Moritz, Stavros Petridis +2
In this work, we propose a streaming AV-ASR system based on a hybrid connectionist temporal classification (CTC)/attention neural network architecture. The audio and the visual enc…
SS-VAERR: Self-Supervised Apparent Emotional Reaction Recognition from Video
Marija Jegorova, Stavros Petridis, Maja Pantic
This work focuses on the apparent emotional reaction recognition (AERR) from the video-only input, conducted in a self-supervised fashion. The network is first pre-trained on diffe…
Training Strategies for Improved Lip-reading
Pingchuan Ma, Yujiang Wang, Stavros Petridis +2
Several training strategies and temporal models have been recently proposed for isolated word lip-reading in a series of independent works. However, the potential of combining the…
SVTS: Scalable Video-to-Speech Synthesis
Rodrigo Mira, Alexandros Haliassos, Stavros Petridis +2
Video-to-speech synthesis (also known as lip-to-speech) refers to the translation of silent lip movements into the corresponding audio. This task has received an increasing amount…
Self-supervised Video-centralised Transformer for Video Face Clustering
Yujiang Wang, Mingzhi Dong, Jie Shen +5
This paper presents a novel method for face clustering in videos using a video-centralised transformer. Previous works often employed contrastive learning to learn frame-level repr…
Visual Speech Recognition for Multiple Languages in the Wild
Pingchuan Ma, Stavros Petridis, Maja Pantic
Visual speech recognition (VSR) aims to recognize the content of speech based on lip movements, without relying on the audio stream. Advances in deep learning and the availability…
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
Alexandros Haliassos, Rodrigo Mira, Stavros Petridis +1
One of the most pressing challenges for the detection of face-manipulated videos is generalising to forgery methods not seen during training while remaining effective under common…