132 citations · 191 across the 26 of their papers we have counts for
26 papers
Target Speech Diarization with Multimodal Prompts
Yidi Jiang, Ruijie Tao, Zhengyang Chen +2
Traditional speaker diarization seeks to detect ``who spoke when'' according to speaker characteristics. Extending to target speech diarization, we detect ``when target event occur…
How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?
Tianchi Liu, Lin Zhang, Rohan Kumar Das +3
Partially manipulating a sentence can greatly change its meaning. Recent work shows that countermeasures (CMs) trained on partially spoofed audio can effectively detect such spoofi…
Audio-Visual Target Speaker Extraction with Reverse Selective Auditory Attention
Ruijie Tao, Xinyuan Qian, Yidi Jiang +3
Audio-visual target speaker extraction (AV-TSE) aims to extract the specific person's speech from the audio mixture given auxiliary visual cues. Previous methods usually search for…
Voice Conversion Augmentation for Speaker Recognition on Defective Datasets
Ruijie Tao, Zhan Shi, Yidi Jiang +2
Modern speaker recognition system relies on abundant and balanced datasets for classification training. However, diverse defective datasets, such as partially-labelled, small-scale…
Enhancing Real-World Active Speaker Detection with Multi-Modal Extraction Pre-Training
Ruijie Tao, Xinyuan Qian, Rohan Kumar Das +3
Audio-visual active speaker detection (AV-ASD) aims to identify which visible face is speaking in a scene with one or more persons. Most existing AV-ASD methods prioritize capturin…
Prompt-driven Target Speech Diarization
Yidi Jiang, Zhengyang Chen, Ruijie Tao +3
We introduce a novel task named `target speech diarization', which seeks to determine `when target event occurred' within an audio signal. We devise a neural architecture called Pr…
Audio-Visual Active Speaker Extraction for Sparsely Overlapped Multi-talker Speech
Junjie Li, Ruijie Tao, Zexu Pan +3
Target speaker extraction aims to extract the speech of a specific speaker from a multi-talker mixture as specified by an auxiliary reference. Most studies focus on the scenario wh…
Emphasized Non-Target Speaker Knowledge in Knowledge Distillation for Automatic Speaker Verification
Duc-Tuan Truong, Ruijie Tao, Jia Qi Yip +2
Knowledge distillation (KD) is used to enhance automatic speaker verification performance by ensuring consistency between large teacher networks and lightweight student networks at…
USED: Universal Speaker Extraction and Diarization
Junyi Ao, Mehmet Sinan Yıldırım, Ruijie Tao +4
Speaker extraction and diarization are two enabling techniques for real-world speech applications. Speaker extraction aims to extract a target speaker's voice from a speech mixture…
Audio Generation with Multiple Conditional Diffusion Model
Zhifang Guo, Jianguo Mao, Rui Tao +4
Text-based audio generation models have limitations as they cannot encompass all the information in audio, leading to restricted controllability when relying solely on text. To add…
Target Active Speaker Detection with Audio-visual Cues
Yidi Jiang, Ruijie Tao, Zexu Pan +1
In active speaker detection (ASD), we would like to detect whether an on-screen person is speaking based on audio-visual cues. Previous studies have primarily focused on modeling a…
I4U System Description for NIST SRE'20 CTS Challenge
Kong Aik Lee, Tomi Kinnunen, Daniele Colibro +23
This manuscript describes the I4U submission to the 2020 NIST Speaker Recognition Evaluation (SRE'20) Conversational Telephone Speech (CTS) Challenge. The I4U's submission was resu…
Speaker recognition with two-step multi-modal deep cleansing
Ruijie Tao, Kong Aik Lee, Zhan Shi +1
Neural network-based speaker recognition has achieved significant improvement in recent years. A robust speaker representation learns meaningful knowledge from both hard and easy s…
Self-Supervised Training of Speaker Encoder with Multi-Modal Diverse Positive Pairs
Ruijie Tao, Kong Aik Lee, Rohan Kumar Das +2
We study a novel neural architecture and its training strategies of speaker encoder for speaker recognition without using any identity labels. The speaker encoder is trained to ext…
A Hybrid System of Sound Event Detection Transformer and Frame-wise Model for DCASE 2022 Task 4
Yiming Li, Zhifang Guo, Zhirong Ye +6
In this paper, we describe in detail our system for DCASE 2022 Task4. The system combines two considerably different models: an end-to-end Sound Event Detection Transformer (SEDT)…
HLT-NUS SUBMISSION FOR 2020 NIST Conversational Telephone Speech SRE
Rohan Kumar Das, Ruijie Tao, Haizhou Li
This work provides a brief description of Human Language Technology (HLT) Laboratory, National University of Singapore (NUS) system submission for 2020 NIST conversational telephon…
Sound Event Detection Transformer: An Event-based End-to-End Model for Sound Event Detection
Zhirong Ye, Xiangdong Wang, Hong Liu +4
Sound event detection (SED) has gained increasing attention with its wide application in surveillance, video indexing, etc. Existing models in SED mainly generate frame-level predi…
SP-SEDT: Self-supervised Pre-training for Sound Event Detection Transformer
Zhirong Ye, Xiangdong Wang, Hong Liu +4
Recently, an event-based end-to-end model (SEDT) has been proposed for sound event detection (SED) and achieves competitive performance. However, compared with the frame-based mode…
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Kristen Grauman, Andrew Westbury, Eugene Byrne +82
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of daily-life activity video spanning hundreds of scenarios (household, outd…
Couple Learning for semi-supervised sound event detection
Rui Tao, Long Yan, Kazushige Ouchi +1
The recently proposed Mean Teacher method, which exploits large-scale unlabeled data in a self-ensembling manner, has achieved state-of-the-art results in several semi-supervised l…
Self-supervised Speaker Recognition with Loss-gated Learning
Ruijie Tao, Kong Aik Lee, Rohan Kumar Das +2
In self-supervised learning for speaker recognition, pseudo labels are useful as the supervision signals. It is a known fact that a speaker recognition model doesn't always benefit…
Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection
Ruijie Tao, Zexu Pan, Rohan Kumar Das +3
Active speaker detection (ASD) seeks to detect who is speaking in a visual scene of one or more speakers. The successful ASD depends on accurate interpretation of short-term and lo…
Selective Listening by Synchronizing Speech with Lips
Zexu Pan, Ruijie Tao, Chenglin Xu +1
A speaker extraction algorithm seeks to extract the speech of a target speaker from a multi-talker speech mixture when given a cue that represents the target speaker, such as a pre…
HLT-NUS Submission for NIST 2019 Multimedia Speaker Recognition Evaluation
Rohan Kumar Das, Ruijie Tao, Jichen Yang +3
This work describes the speaker verification system developed by Human Language Technology Laboratory, National University of Singapore (HLT-NUS) for 2019 NIST Multimedia Speaker R…