47 citations · 121 across the 17 of their papers we have counts for
17 papers
The USTC-NERCSLIP Systems for the CHiME-9 MCoRec Challenge
Ya Jiang, Ruoyu Wang, Jingxuan Zhang +19
This report details our submission to the CHiME-9 MCoRec Challenge on recognizing and clustering multiple concurrent natural conversations within indoor social settings. Unlike con…
Exploring Speaker Diarization with Mixture of Experts
Gaobin Yang, Maokui He, Shutong Niu +3
In this paper, we propose a novel neural speaker diarization system using memory-aware multi-speaker embedding with sequence-to-sequence architecture (NSD-MS2S), which integrates a…
DCF-DS: Deep Cascade Fusion of Diarization and Separation for Speech Recognition under Realistic Single-Channel Conditions
Shu-Tong Niu, Jun Du, Ruo-Yu Wang +4
We propose a single-channel Deep Cascade Fusion of Diarization and Separation (DCF-DS) framework for back-end automatic speech recognition (ASR), combining neural speaker diarizati…
EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion
Haotian Wang, Yuzhe Weng, Yueyan Li +10
Diffusion models have revolutionized the field of talking head generation, yet still face challenges in expressiveness, controllability, and stability in long-time generation. In t…
The USTC-NERCSLIP Systems for the CHiME-8 NOTSOFAR-1 Challenge
Shutong Niu, Ruoyu Wang, Jun Du +17
This technical report outlines our submission system for the CHiME-8 NOTSOFAR-1 Challenge. The primary difficulty of this challenge is the dataset recorded across various conferenc…
Quality-Aware End-to-End Audio-Visual Neural Speaker Diarization
Mao-Kui He, Jun Du, Shu-Tong Niu +2
In this paper, we propose a quality-aware end-to-end audio-visual neural speaker diarization framework, which comprises three key techniques. First, our audio-visual model takes bo…
The USTC-NERCSLIP Systems for the CHiME-8 MMCSG Challenge
Ya Jiang, Hongbo Lan, Jun Du +2
In the two-person conversation scenario with one wearing smart glasses, transcribing and displaying the speaker's content in real-time is an intriguing application, providing a pri…
Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention
Yuzhe Weng, Haotian Wang, Tian Gao +3
In multimodal sentiment analysis, collecting text data is often more challenging than video or audio due to higher annotation costs and inconsistent automatic speech recognition (A…
Incorporating Spatial Cues in Modular Speaker Diarization for Multi-channel Multi-party Meetings
Ruoyu Wang, Shutong Niu, Gaobin Yang +4
Although fully end-to-end speaker diarization systems have made significant progress in recent years, modular systems often achieve superior results in real-world scenarios due to…
Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding with Sequence-to-Sequence Architecture
Gaobin Yang, Maokui He, Shutong Niu +6
We propose a novel neural speaker diarization system using memory-aware multi-speaker embedding with sequence-to-sequence architecture (NSD-MS2S), which integrates the strengths of…
The USTC-NERCSLIP Systems for the CHiME-7 DASR Challenge
Ruoyu Wang, Maokui He, Jun Du +16
This technical report details our submission system to the CHiME-7 DASR Challenge, which focuses on speaker diarization and speech recognition under complex multi-speaker scenarios…
Semi-supervised multi-channel speaker diarization with cross-channel attention
Shilong Wu, Jun Du, Maokui He +4
Most neural speaker diarization systems rely on sufficient manual training data labels, which are hard to collect under real-world scenarios. This paper proposes a semi-supervised…
The USTC-Ximalaya system for the ICASSP 2022 multi-channel multi-party meeting transcription (M2MeT) challenge
Maokui He, Xiang Lv, Weilin Zhou +8
We propose two improvements to target-speaker voice activity detection (TS-VAD), the core component in our proposed speaker diarization system that was submitted to the 2022 Multi-…
Separation Guided Speaker Diarization in Realistic Mismatched Conditions
Shu-Tong Niu, Jun Du, Lei Sun +1
We propose a separation guided speaker diarization (SGSD) approach by fully utilizing a complementarity of speech separation and speaker clustering. Since the conventional clusteri…
USTC-NELSLIP System Description for DIHARD-III Challenge
Yuxuan Wang, Maokui He, Shutong Niu +6
This system description describes our submission system to the Third DIHARD Speech Diarization Challenge. Besides the traditional clustering based system, the innovation of our sys…
A Two-Stage Approach to Device-Robust Acoustic Scene Classification
Hu Hu, Chao-Han Huck Yang, Xianjun Xia +13
To improve device robustness, a highly desirable key feature of a competitive data-driven acoustic scene classification (ASC) system, a novel two-stage system based on fully convol…
Device-Robust Acoustic Scene Classification Based on Two-Stage Categorization and Data Augmentation
Hu Hu, Chao-Han Huck Yang, Xianjun Xia +13
In this technical report, we present a joint effort of four groups, namely GT, USTC, Tencent, and UKE, to tackle Task 1 - Acoustic Scene Classification (ASC) in the DCASE 2020 Chal…