38 citations · 88 across the 46 of their papers we have counts for
28 papers · 1 filter
Augmenting Dysarthric Speech Severity Assessment with MOS Supervision
Kaimeng Jia, Minzhu Tu, Zengrui Jin +2
Dysarthria is a speech disorder marked by reduced intelligibility and communicative effectiveness. Automatic utterance-level assessment of dysarthric speech can support scalable sp…
UNISON: A Unified Sound Generation and Editing Framework via Deep LLM Fusion
Zhaoqing Li, Haoning Xu, Jingran Su +9
We present UNISON, a latent diffusion framework that unifies speech generation, sound generation, and audio editing within a single model. A single model handles text-to-audio, tex…
FSA-GRPO: Teaching Auditory LLMs to Use Few-Shot Demonstrations
Haolong Zheng, Siyin Wang, Xulin Fan +2
Few-shot prompting provides an effective way to adapt auditory large language models to low-resource tasks such as children's speech recognition. However, most auditory large langu…
Towards Fine-Grained and Multi-Granular Contrastive Language-Speech Pre-training
Yifan Yang, Bing Han, Hui Wang +8
Modeling fine-grained speaking styles remains challenging for language-speech representation pre-training, as existing speech-text models are typically trained with coarse captions…
Towards Effective and Efficient Non-autoregressive decoders for Conformer and LLM-based ASR using Block-based Attention Mask
Tianzi Wang, Xurong Xie, Zengrui Jin +9
Automatic speech recognition (ASR) systems often rely on autoregressive (AR) Transformer decoder architectures, which limit efficient inference parallelization due to their sequent…
Augmenting Open-Vocabulary Dysarthric Speech Assessment with Human Perceptual Supervision
Kaimeng Jia, Minzhu Tu, Zengrui Jin +2
Dysarthria is a speech disorder characterized by impaired intelligibility and reduced communicative effectiveness. Automatic dysarthria assessment provides a scalable, cost-effecti…
Towards General Auditory Intelligence: Large Multimodal Models for Machine Listening and Speaking
Siyin Wang, Zengrui Jin, Changli Tang +26
In the era of large language models (LLMs) and artificial general intelligence (AGI), computer audition must evolve beyond traditional paradigms to fully leverage the capabilities…
Multiplexing Neural Audio Watermarks with Adaptive Routing
Zheqi Yuan, Yucheng Huang, Guangzhi Sun +2
Audio watermarking supports speech authenticity verification. We study whether combining heterogeneous watermarks can retain at least one detectable provenance signal when their fa…
SPEAR: A Unified SSL Framework for Learning Speech and Audio Representations
Xiaoyu Yang, Yifan Yang, Zengrui Jin +5
Self-supervised learning (SSL) has significantly advanced acoustic representation learning. However, most existing models are optimised for either speech or audio event understandi…
Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing
Mengqi Wang, Zhan Liu, Zengrui Jin +3
Diffusion-based large language models (DLLMs) have recently attracted growing interest as an alternative to autoregressive decoders. In this work, we present an empirical study on…
Structured Speaker-Deficiency Adaptation of Foundation Models for Dysarthric and Elderly Speech Recognition
Shujie Hu, Xurong Xie, Mengzhe Geng +8
Data-intensive fine-tuning of speech foundation models (SFMs) to scarce and diverse dysarthric and elderly speech leads to data bias and poor generalization to unseen speakers. Thi…
k2SSL: A Faster and Better Framework for Self-Supervised Speech Representation Learning
Yifan Yang, Jianheng Zhuo, Zengrui Jin +9
Self-supervised learning (SSL) has achieved great success in speech-related tasks. While Transformer and Conformer architectures have dominated SSL backbones, encoders like Zipform…
CR-CTC: Consistency regularization on CTC for improved speech recognition
Zengwei Yao, Wei Kang, Xiaoyu Yang +7
Connectionist Temporal Classification (CTC) is a widely used method for automatic speech recognition (ASR), renowned for its simplicity and computational efficiency. However, it of…
Advancing Multi-talker ASR Performance with Large Language Models
Mohan Shi, Zengrui Jin, Yaoxun Xu +6
Recognizing overlapping speech from multiple speakers in conversational scenarios is one of the most challenging problem for automatic speech recognition (ASR). Serialized output t…
Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition
Shujie Hu, Xurong Xie, Mengzhe Geng +8
Self-supervised learning (SSL) based speech foundation models have been applied to a wide range of ASR tasks. However, their application to dysarthric and elderly speech via data-i…
Towards Automatic Data Augmentation for Disordered Speech Recognition
Zengrui Jin, Xurong Xie, Tianzi Wang +5
Automatic recognition of disordered speech remains a highly challenging task to date due to data scarcity. This paper presents a reinforcement learning (RL) based on-the-fly data a…
Zipformer: A faster and better encoder for automatic speech recognition
Zengwei Yao, Liyong Guo, Xiaoyu Yang +6
The Conformer has become the most popular encoder model for automatic speech recognition (ASR). It adds convolution modules to a transformer to learn both local and global dependen…
Audio-visual End-to-end Multi-channel Speech Separation, Dereverberation and Recognition
Guinan Li, Jiajun Deng, Mengzhe Geng +6
Accurate recognition of cocktail party speech containing overlapping speakers, noise and reverberation remains a highly challenging task to date. Motivated by the invariance of vis…
Hyper-parameter Adaptation of Conformer ASR Systems for Elderly and Dysarthric Speech Recognition
Tianzi Wang, Shoukang Hu, Jiajun Deng +5
Automatic recognition of disordered and elderly speech remains highly challenging tasks to date due to data scarcity. Parameter fine-tuning is often used to exploit the large quant…
Factorised Speaker-environment Adaptive Training of Conformer Speech Recognition Systems
Jiajun Deng, Guinan Li, Xurong Xie +6
Rich sources of variability in natural speech present significant challenges to current data intensive speech recognition technologies. To model both speaker and environment level…
Use of Speech Impairment Severity for Dysarthric Speech Recognition
Mengzhe Geng, Zengrui Jin, Tianzi Wang +7
A key challenge in dysarthric speech recognition is the speaker-level diversity attributed to both speaker-identity associated factors such as gender, and speech impairment severit…
Confidence Score Based Speaker Adaptation of Conformer Speech Recognition Systems
Jiajun Deng, Xurong Xie, Tianzi Wang +6
Speaker adaptation techniques provide a powerful solution to customise automatic speech recognition (ASR) systems for individual users. Practical application of unsupervised model-…
Adversarial Data Augmentation Using VAE-GAN for Disordered Speech Recognition
Zengrui Jin, Xurong Xie, Mengzhe Geng +5
Automatic recognition of disordered speech remains a highly challenging task to date. The underlying neuro-motor conditions, often compounded with co-occurring physical disabilitie…
Confidence Score Based Conformer Speaker Adaptation for Speech Recognition
Jiajun Deng, Xurong Xie, Tianzi Wang +7
A key challenge for automatic speech recognition (ASR) systems is to model the speaker level variability. In this paper, compact speaker dependent learning hidden unit contribution…