1.9k citations · 1.9k across the 15 of their papers we have counts for
15 papers
TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation
Chenyang Le, Yao Qian, Dongmei Wang +8
There is a rising interest and trend in research towards directly translating speech from one language to another, known as end-to-end speech-to-speech translation. However, most e…
DDTSE: Discriminative Diffusion Model for Target Speech Extraction
Leying Zhang, Yao Qian, Linfeng Yu +5
Diffusion models have gained attention in speech enhancement tasks, providing an alternative to conventional discriminative methods. However, research on target speech extraction u…
Adapting Large Language Model with Speech for Fully Formatted End-to-End Speech Recognition
Shaoshi Ling, Yuxuan Hu, Shuangbei Qian +5
Most end-to-end (E2E) speech recognition models are composed of encoder and decoder blocks that perform acoustic and language modeling functions. Pretrained large language models (…
Adapting Multi-Lingual ASR Models for Handling Multiple Talkers
Chenda Li, Yao Qian, Zhuo Chen +5
State-of-the-art large-scale universal speech models (USMs) show a decent automatic speech recognition (ASR) performance across multiple domains and languages. However, it remains…
i-Code Studio: A Configurable and Composable Framework for Integrative AI
Yuwei Fang, Mahmoud Khademi, Chenguang Zhu +8
Artificial General Intelligence (AGI) requires comprehensive understanding and generation capabilities for a variety of tasks spanning different modalities and functionalities. Int…
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data
Ziyi Yang, Mahmoud Khademi, Yichong Xu +16
The convergence of text, visual, and audio data is a key step towards human-like artificial intelligence, however the current Vision-Language-Speech landscape is dominated by encod…
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation
Chenyang Le, Yao Qian, Long Zhou +4
Joint speech-language training is challenging due to the large demand for training data and GPU consumption, as well as the modality gap between speech and language. We present Com…
Code-Switching Text Generation and Injection in Mandarin-English ASR
Haibin Yu, Yuxuan Hu, Yao Qian +7
Code-switching speech refers to a means of expression by mixing two or more languages within a single utterance. Automatic Speech Recognition (ASR) with End-to-End (E2E) modeling f…
Target Sound Extraction with Variable Cross-modality Clues
Chenda Li, Yao Qian, Zhuo Chen +5
Automatic target sound extraction (TSE) is a machine learning approach to mimic the human auditory perception capability of attending to a sound source of interest from a mixture o…
A comprehensive study on self-supervised distillation for speaker representation learning
Zhengyang Chen, Yao Qian, Bing Han +2
In real application scenarios, it is often challenging to obtain a large amount of labeled data for speaker representation learning due to speaker privacy concerns. Self-supervised…
The Microsoft System for VoxCeleb Speaker Recognition Challenge 2022
Gang Liu, Tianyan Zhou, Yong Zhao +4
In this report, we describe our submitted system for track 2 of the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC-22). We fuse a variety of good-performing models ranging fro…
i-Code: An Integrative and Composable Multimodal Learning Framework
Ziyi Yang, Yuwei Fang, Chenguang Zhu +17
Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview. Most current pretraining methods, however, are limited to…
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
Sanyuan Chen, Yu Wu, Chengyi Wang +8
Self-supervised learning (SSL) is a long-standing goal for speech processing, since it utilizes large-scale unlabeled data and avoids extensive human labeling. Recent years witness…
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Sanyuan Chen, Chengyi Wang, Zhengyang Chen +15
Self-supervised learning (SSL) achieves great success in speech recognition, while limited exploration has been attempted for other speech processing tasks. As speech signal contai…
To Trust, or Not to Trust? A Study of Human Bias in Automated Video Interview Assessments
Chee Wee Leong, Katrina Roohr, Vikram Ramanarayanan +6
Supervised systems require human labels for training. But, are humans themselves always impartial during the annotation process? We examine this question in the context of automate…