2.3k citations · 7.4k across the 34 of their papers we have counts for
46 papers
TalkNCE: Improving Active Speaker Detection with Talk-Aware Contrastive Learning
Chaeyoung Jung, Suyeon Lee, Kihyun Nam +4
The goal of this work is Active Speaker Detection (ASD), a task to determine whether a person is speaking or not in a series of video frames. Previous works have dealt with the tas…
SlowFast Network for Continuous Sign Language Recognition
Junseok Ahn, Youngjoon Jang, Joon Son Chung
The objective of this work is the effective extraction of spatial and dynamic features for Continuous Sign Language Recognition (CSLR). To accomplish this, we utilise a two-pathway…
Sound Source Localization is All about Cross-Modal Alignment
Arda Senocak, Hyeonggon Ryu, Junsik Kim +3
Humans can easily perceive the direction of sound sources in a visual scene, termed sound source localization. Recent studies on learning-based sound source localization have mainl…
Let There Be Sound: Reconstructing High Quality Speech from Silent Videos
Ji-Hoon Kim, Jaehun Kim, Joon Son Chung
The goal of this work is to reconstruct high quality speech from lip motions alone, a task also known as lip-to-speech. A key challenge of lip-to-speech systems is the one-to-many…
FlexiAST: Flexibility is What AST Needs
Jiu Feng, Mehmet Hamza Erol, Joon Son Chung +1
The objective of this work is to give patch-size flexibility to Audio Spectrogram Transformers (AST). Recent advancements in ASTs have shown superior performance in various audio-b…
That's What I Said: Fully-Controllable Talking Face Generation
Youngjoon Jang, Kyeongha Rho, Jong-Bin Woo +5
The goal of this paper is to synthesise talking faces with controllable facial motions. To achieve this goal, we propose two key ideas. The first is to establish a canonical space…
Hindi as a Second Language: Improving Visually Grounded Speech with Semantically Similar Samples
Hyeonggon Ryu, Arda Senocak, In So Kweon +1
The objective of this work is to explore the learning of visually grounded speech models (VGS) from multilingual perspective. Bilingual VGS models are generally trained with an equ…
Imaginary Voice: Face-styled Diffusion Model for Text-to-Speech
Jiyoung Lee, Joon Son Chung, Soo-Whan Chung
The goal of this work is zero-shot text-to-speech synthesis, with speaking styles and voices learnt from facial characteristics. Inspired by the natural fact that people can imagin…
MarginNCE: Robust Sound Localization with a Negative Margin
Sooyoung Park, Arda Senocak, Joon Son Chung
The goal of this work is to localize sound sources in visual scenes with a self-supervised approach. Contrastive learning in the context of sound source localization leverages the…
Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition
Youngjoon Jang, Youngtaek Oh, Jae Won Cho +3
The goal of this work is background-robust continuous sign language recognition. Most existing Continuous Sign Language Recognition (CSLR) benchmarks have fixed backgrounds and are…
VoxSRC 2021: The Third VoxCeleb Speaker Recognition Challenge
Andrew Brown, Jaesung Huh, Joon Son Chung +3
The third instalment of the VoxCeleb Speaker Recognition Challenge was held in conjunction with Interspeech 2021. The aim of this challenge was to assess how well current speaker r…
VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge
Arsha Nagrani, Joon Son Chung, Jaesung Huh +6
We held the second installment of the VoxCeleb Speaker Recognition Challenge in conjunction with Interspeech 2020. The goal of this challenge was to assess how well current speaker…
Supervised attention for speaker recognition
Seong Min Kye, Joon Son Chung, Hoirin Kim
The recently proposed self-attentive pooling (SAP) has shown good performance in several speaker recognition systems. In SAP systems, the context vector is trained end-to-end toget…
Look who's not talking
Youngki Kwon, Hee Soo Heo, Jaesung Huh +2
The objective of this work is speaker diarisation of speech recordings 'in the wild'. The ability to determine speech segments is a crucial part of diarisation systems, accounting…
The ins and outs of speaker recognition: lessons from VoxSRC 2020
Yoohwan Kwon, Hee-Soo Heo, Bong-Jin Lee +1
The VoxCeleb Speaker Recognition Challenge (VoxSRC) at Interspeech 2020 offers a challenging evaluation for speaker recognition systems, which includes celebrities playing differen…
Playing a Part: Speaker Verification at the Movies
Andrew Brown, Jaesung Huh, Arsha Nagrani +2
The goal of this work is to investigate the performance of popular speaker recognition models on speech segments from movies, where often actors intentionally disguise their voice…
Graph Attention Networks for Speaker Verification
Jee-weon Jung, Hee-Soo Heo, Ha-Jin Yu +1
This work presents a novel back-end framework for speaker verification using graph attention networks. Segment-wise speaker embeddings extracted from multiple crops within an utter…
Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020
Hee Soo Heo, Bong-Jin Lee, Jaesung Huh +1
This report describes our submission to the VoxCeleb Speaker Recognition Challenge (VoxSRC) at Interspeech 2020. We perform a careful analysis of speaker recognition models based o…
Self-Supervised Learning of Audio-Visual Objects from Video
Triantafyllos Afouras, Andrew Owens, Joon Son Chung +1
Our objective is to transform a video into a set of discrete audio-visual objects using self-supervised learning. To this end, we introduce a model that uses attention to localize…
Cross attentive pooling for speaker verification
Seong Min Kye, Yoohwan Kwon, Joon Son Chung
The goal of this paper is text-independent speaker verification where utterances come from 'in the wild' videos and may contain irrelevant signal. While speaker verification is nat…
BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues
Samuel Albanie, Gül Varol, Liliane Momeni +4
Recent progress in fine-grained gesture and action classification, and machine translation, point to the possibility of automated sign language recognition becoming a reality. A ke…
Augmentation adversarial training for self-supervised speaker recognition
Jaesung Huh, Hee Soo Heo, Jingu Kang +2
The goal of this work is to train robust speaker recognition models without speaker labels. Recent works on unsupervised speaker representations are based on contrastive learning i…
Spot the conversation: speaker diarisation in the wild
Joon Son Chung, Jaesung Huh, Arsha Nagrani +2
The goal of this paper is speaker diarisation of videos collected 'in the wild'. We make three key contributions. First, we propose an automatic audio-visual diarisation method for…
Metric Learning for Keyword Spotting
Jaesung Huh, Minjae Lee, Heesoo Heo +2
The goal of this work is to train effective representations for keyword spotting via metric learning. Most existing works address keyword spotting as a closed-set classification pr…