216 citations · 418 across the 22 of their papers we have counts for
22 papers
LLaDA-TTS: Unifying Speech Synthesis and Zero-Shot Editing via Masked Diffusion Modeling
Xiaoyu Fan, Huizhi Xie, Wei Zou +1
Large language model (LLM)-based text-to-speech (TTS) systems achieve remarkable naturalness via autoregressive (AR) decoding, but require N sequential steps to generate N speech t…
Understanding the Modality Gap: An Empirical Study on the Speech-Text Alignment Mechanism of Large Speech Language Models
Bajian Xiang, Shuaijiang Zhao, Tingwei Guo +1
End-to-end Large Speech Language Models (LSLMs) have demonstrated impressive conversational generation abilities, yet consistently fall short of traditional pipeline systems on sem…
SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learning
Cheng Wen, Tingwei Guo, Shuaijiang Zhao +2
Recent work shows that reinforcement learning(RL) can markedly sharpen the reasoning ability of large language models (LLMs) by prompting them to "think before answering." Yet whet…
Advancing Speech Language Models by Scaling Supervised Fine-Tuning with Over 60,000 Hours of Synthetic Speech Dialogue Data
Shuaijiang Zhao, Tingwei Guo, Bajian Xiang +4
The GPT-4o represents a significant milestone in enabling real-time interaction with large language models (LLMs) through speech, its remarkable low latency and high fluency not on…
ChatHome: Development and Evaluation of a Domain-Specific Language Model for Home Renovation
Cheng Wen, Xianghui Sun, Shuaijiang Zhao +3
This paper presents the development and evaluation of ChatHome, a domain-specific language model (DSLM) designed for the intricate field of home renovation. Considering the proven…
Analyzing Robustness of End-to-End Neural Models for Automatic Speech Recognition
Goutham Rajendran, Wei Zou
We investigate robustness properties of pre-trained neural models for automatic speech recognition. Real life data in machine learning is usually very noisy and almost never clean,…
Audio Deep Fake Detection System with Neural Stitching for ADD 2022
Rui Yan, Cheng Wen, Shuran Zhou +3
This paper describes our best system and methodology for ADD 2022: The First Audio Deep Synthesis Detection Challenge\cite{Yi2022ADD}. The very same system was used for both two ro…
Time Domain Adversarial Voice Conversion for ADD 2022
Cheng Wen, Tingwei Guo, Xingjun Tan +5
In this paper, we describe our speech generation system for the first Audio Deep Synthesis Detection Challenge (ADD 2022). Firstly, we build an any-to-many voice conversion (VC) sy…
Audio-Visual Wake Word Spotting System For MISP Challenge 2021
Yanguang Xu, Jianwei Sun, Yang Han +7
This paper presents the details of our system designed for the Task 1 of Multimodal Information Based Speech Processing (MISP) Challenge 2021. The purpose of Task 1 is to leverage…
GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio
Guoguo Chen, Shuzhou Chai, Guanbo Wang +18
This paper introduces GigaSpeech, an evolving, multi-domain English speech recognition corpus with 10,000 hours of high quality labeled audio suitable for supervised training, and…
Semantic Data Augmentation for End-to-End Mandarin Speech Recognition
Jianwei Sun, Zhiyuan Tang, Hengxin Yin +6
End-to-end models have gradually become the preferred option for automatic speech recognition (ASR) applications. During the training of end-to-end ASR, data augmentation is a quit…
TMT: A Transformer-based Modal Translator for Improving Multimodal Sequence Representations in Audio Visual Scene-aware Dialog
Wubo Li, Dongwei Jiang, Wei Zou +1
Audio Visual Scene-aware Dialog (AVSD) is a task to generate responses when discussing about a given video. The previous state-of-the-art model shows superior performance for this…
Speech SIMCLR: Combining Contrastive and Reconstruction Objective for Self-supervised Speech Representation Learning
Dongwei Jiang, Wubo Li, Miao Cao +2
Self-supervised visual pretraining has shown significant progress recently. Among those methods, SimCLR greatly advanced the state of the art in self-supervised and semi-supervised…
DiDiSpeech: A Large Scale Mandarin Speech Corpus
Tingwei Guo, Cheng Wen, Dongwei Jiang +8
This paper introduces a new open-sourced Mandarin speech corpus, called DiDiSpeech. It consists of about 800 hours of speech data at 48kHz sampling rate from 6000 speakers and the…
Transformer based unsupervised pre-training for acoustic representation learning
Ruixiong Zhang, Haiwei Wu, Wubo Li +3
Recently, a variety of acoustic tasks and related applications arised. For many acoustic tasks, the labeled data size may be limited. To handle this problem, we propose an unsuperv…
A Further Study of Unsupervised Pre-training for Transformer Based Speech Recognition
Dongwei Jiang, Wubo Li, Ruixiong Zhang +5
Building a good speech recognition system usually requires large amounts of transcribed data, which is expensive to collect. To tackle this problem, many unsupervised pre-training…
Improving Transformer-based Speech Recognition Using Unsupervised Pre-training
Dongwei Jiang, Xiaoning Lei, Wubo Li +4
Speech recognition technologies are gaining enormous popularity in various industrial applications. However, building a good speech recognition system usually requires large amount…
TCT: A Cross-supervised Learning Method for Multimodal Sequence Representation
Wubo Li, Wei Zou, Xiangang Li
Multimodalities provide promising performance than unimodality in most tasks. However, learning the semantic of the representations from multimodalities efficiently is extremely ch…
Cross-task pre-training for on-device acoustic scene classification
Ruixiong Zhang, Wei Zou, Xiangang Li
Acoustic scene classification (ASC) and acoustic event detection (AED) are different but related tasks. Acoustic events can provide useful information for recognizing acoustic scen…
DELTA: A DEep learning based Language Technology plAtform
Kun Han, Junwen Chen, Hui Zhang +20
In this paper we present DELTA, a deep learning based language technology platform. DELTA is an end-to-end platform designed to solve industry level natural language and speech pro…
Towards End-to-End Code-Switching Speech Recognition
Ne Luo, Dongwei Jiang, Shuaijiang Zhao +3
Code-switching speech recognition has attracted an increasing interest recently, but the need for expert linguistic knowledge has always been a big issue. End-to-end automatic spee…
A comparable study of modeling units for end-to-end Mandarin speech recognition
Wei Zou, Dongwei Jiang, Shuaijiang Zhao +1
End-To-End speech recognition have become increasingly popular in mandarin speech recognition and achieved delightful performance. Mandarin is a tonal language which is different f…