41 citations · 104 across the 27 of their papers we have counts for
27 papers
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu +1
With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross-modal associatio…
Training Text-to-Speech Model with Purely Synthetic Data: Feasibility, Sensitivity, and Generalization Capability
Tingxiao Zhou, Leying Zhang, Zhengyang Chen +1
The potential of synthetic data in text-to-speech (TTS) model training has gained increasing attention, yet its rationality and effectiveness require systematic validation. In this…
Advanced Zero-Shot Text-to-Speech for Background Removal and Preservation with Controllable Masked Speech Prediction
Leying Zhang, Wangyou Zhang, Zhengyang Chen +1
The acoustic background plays a crucial role in natural conversation. It provides context and helps listeners understand the environment, but a strong background makes it difficult…
Data-Efficient Low-Complexity Acoustic Scene Classification via Distilling and Progressive Pruning
Bing Han, Wen Huang, Zhengyang Chen +7
The goal of the acoustic scene classification (ASC) task is to classify recordings into one of the predefined acoustic scene classes. However, in real-world scenarios, ASC systems…
Prototype and Instance Contrastive Learning for Unsupervised Domain Adaptation in Speaker Verification
Wen Huang, Bing Han, Zhengyang Chen +2
Speaker verification system trained on one domain usually suffers performance degradation when applied to another domain. To address this challenge, researchers commonly use featur…
Flow-TSVAD: Target-Speaker Voice Activity Detection via Latent Flow Matching
Zhengyang Chen, Bing Han, Shuai Wang +2
Speaker diarization is typically considered a discriminative task, using discriminative approaches to produce fixed diarization results. In this paper, we explore the use of neural…
Disentangling the Prosody and Semantic Information with Pre-trained Model for In-Context Learning based Zero-Shot Voice Conversion
Zhengyang Chen, Shuai Wang, Mingyang Zhang +3
Voice conversion (VC) aims to modify the speaker's timbre while retaining speech content. Previous approaches have tokenized the outputs from self-supervised into semantic tokens,…
Generating Speakers by Prompting Listener Impressions for Pre-trained Multi-Speaker Text-to-Speech Systems
Zhengyang Chen, Xuechen Liu, Erica Cooper +2
This paper proposes a speech synthesis system that allows users to specify and control the acoustic characteristics of a speaker by means of prompts describing the speaker's traits…
Target Speech Diarization with Multimodal Prompts
Yidi Jiang, Ruijie Tao, Zhengyang Chen +2
Traditional speaker diarization seeks to detect ``who spoke when'' according to speaker characteristics. Extending to target speech diarization, we detect ``when target event occur…
Prompt-driven Target Speech Diarization
Yidi Jiang, Zhengyang Chen, Ruijie Tao +3
We introduce a novel task named `target speech diarization', which seeks to determine `when target event occurred' within an audio signal. We devise a neural architecture called Pr…
Leveraging In-the-Wild Data for Effective Self-Supervised Pretraining in Speaker Recognition
Shuai Wang, Qibing Bai, Qi Liu +5
Current speaker recognition systems primarily rely on supervised approaches, constrained by the scale of labeled datasets. To boost the system performance, researchers leverage lar…
Attention-based Encoder-Decoder End-to-End Neural Diarization with Embedding Enhancer
Zhengyang Chen, Bing Han, Shuai Wang +1
Deep neural network-based systems have significantly improved the performance of speaker diarization tasks. However, end-to-end neural diarization (EEND) systems often struggle to…
Exploring Binary Classification Loss For Speaker Verification
Bing Han, Zhengyang Chen, Yanmin Qian
The mismatch between close-set training and open-set testing usually leads to significant performance degradation for speaker verification task. For existing loss functions, metric…
Wespeaker baselines for VoxSRC2023
Shuai Wang, Chengdong Liang, Xu Xiang +4
This report showcases the results achieved using the wespeaker toolkit for the VoxSRC2023 Challenge. Our aim is to provide participants, especially those with limited experience, w…
Attention-based Encoder-Decoder Network for End-to-End Neural Speaker Diarization with Target Speaker Attractor
Zhengyang Chen, Bing Han, Shuai Wang +1
This paper proposes a novel Attention-based Encoder-Decoder network for End-to-End Neural speaker Diarization (AED-EEND). In AED-EEND system, we incorporate the target speaker enro…
Self-Supervised Learning with Cluster-Aware-DINO for High-Performance Robust Speaker Verification
Bing Han, Zhengyang Chen, Yanmin Qian
Automatic speaker verification task has made great achievements using deep learning approaches with the large-scale manually annotated dataset. However, it's very difficult and exp…
A comprehensive study on self-supervised distillation for speaker representation learning
Zhengyang Chen, Yao Qian, Bing Han +2
In real application scenarios, it is often challenging to obtain a large amount of labeled data for speaker representation learning due to speaker privacy concerns. Self-supervised…
Build a SRE Challenge System: Lessons from VoxSRC 2022 and CNSRC 2022
Zhengyang Chen, Bing Han, Xu Xiang +3
Many speaker recognition challenges have been held to assess the speaker verification system in the wild and probe the performance limit. Voxceleb Speaker Recognition Challenge (Vo…
Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit
Hongji Wang, Chengdong Liang, Shuai Wang +5
Speaker modeling is essential for many related tasks, such as speaker recognition and speaker diarization. The dominant modeling approach is fixed-dimensional vector representation…
SJTU-AISPEECH System for VoxCeleb Speaker Recognition Challenge 2022
Zhengyang Chen, Bing Han, Xu Xiang +3
This report describes the SJTU-AISPEECH system for the Voxceleb Speaker Recognition Challenge 2022. For track1, we implemented two kinds of systems, the online system and the offli…
The SJTU System for Short-duration Speaker Verification Challenge 2021
Bing Han, Zhengyang Chen, Zhikai Zhou +1
This paper presents the SJTU system for both text-dependent and text-independent tasks in short-duration speaker verification (SdSV) challenge 2021. In this challenge, we explored…
Self-Supervised Speaker Verification Using Dynamic Loss-Gate and Label Correction
Bing Han, Zhengyang Chen, Yanmin Qian
For self-supervised speaker verification, the quality of pseudo labels decides the upper bound of the system due to the massive unreliable labels. In this work, we propose dynamic…
Contrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognition
Zhan Chen, Hong Liu, Tianyu Guo +3
Self-supervised skeleton-based action recognition with contrastive learning has attracted much attention. Recent literature shows that data augmentation and large sets of contrasti…
The SJTU X-LANCE Lab System for CNSRC 2022
Zhengyang Chen, Bei Liu, Bing Han +2
This technical report describes the SJTU X-LANCE Lab system for the three tracks in CNSRC 2022. In this challenge, we explored the speaker embedding modeling ability of deep ResNet…