Mengzhe Geng

Auditing generative audio calls for known-task audio-llm evaluation

Mengzhe Geng

Speech and audio LLMs are evaluated by comparing waveform predictions with predictions from an automatic speech recognition (ASR) transcript. For fixed closed-set tasks, this confl…

AudioNoisePrints: Model-free audio watermarking using spatial correlation in flow matching TTS

Timothy Tin-Long, Jian Zhu, Aidan Pine +1

We present AudioNoisePrints, a training-free watermarking pipeline for flow matching and diffusion TTS models, which requires minimal extra computation during inference and does no…

Explainable and Trustworthy Speech Emotion Recognition Using Confidence Score and Reinforcement Learning Rectified Speech Emotion Descriptors

Youjun Chen, Xurong Xie, Mengzhe Geng +9

Explainable and trustworthy speech emotion recognition (SER) remains a challenging task to date, largely due to the scarcity of SER data with reliable speech emotion descriptor (SE…

Towards Personalized Federated Learning for Dysarthric Speech Recognition

Tao Zhong, Mengzhe Geng, Jiajun Deng +2

Speech recognition is challenging for dysarthric speakers. While federated learning (FL)-based ASR can be an effective tool for protecting privacy, it suffers from heterogeneity is…

Towards Data-free and Training-free Compression for Speech Foundation Models Using Parameter Clustering

Haoning Xu, Zhaoqing Li, Huimeng Wang +4

This paper presents a novel data-free and training-free compression approach for speech foundation models using channelwise clustering via k-means. More fine-grained, mixed sparsit…

Multi-Channel Speech Enhancement for Cocktail Party Speech Emotion Recognition

Youjun Chen, Guinan Li, Mengzhe Geng +9

This paper highlights the critical importance of multi-channel speech enhancement (MCSE) for speech emotion recognition (ER) in cocktail party scenarios. A multi-channel speech der…

Supporting SENCOTEN Language Documentation Efforts with Automatic Speech Recognition

Mengzhe Geng, Patrick Littell, Aidan Pine +3

The SENCOTEN language, spoken on the Saanich peninsula of southern Vancouver Island, is in the midst of vigorous language revitalization efforts to turn the tide of language loss a…

Towards LLM-Empowered Fine-Grained Speech Descriptors for Explainable Emotion Recognition

Youjun Chen, Xurong Xie, Haoning Xu +6

This paper presents a novel end-to-end LLM-empowered explainable speech emotion recognition (SER) approach. Fine-grained speech emotion descriptor (SED) features, e.g., pitch, tone…

Effective and Efficient One-pass Compression of Speech Foundation Models Using Sparsity-aware Self-pinching Gates

Haoning Xu, Zhaoqing Li, Youjun Chen +5

This paper presents a novel approach for speech foundation models compression that tightly integrates model pruning and parameter update into a single stage. Highly compact layer-l…

On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognition

Shujie HU, Xurong Xie, Mengzhe Geng +8

This paper proposes a novel MoE-based speaker adaptation framework for foundation models based dysarthric speech recognition. This approach enables zero-shot adaptation and real-ti…

Effective and Efficient Mixed Precision Quantization of Speech Foundation Models

Haoning Xu, Zhaoqing Li, Zengrui Jin +7

This paper presents a novel mixed-precision quantization approach for speech foundation models that tightly integrates mixed-precision learning and quantized model parameter estima…

Phone-purity Guided Discrete Tokens for Dysarthric Speech Recognition

Huimeng Wang, Xurong Xie, Mengzhe Geng +6

Discrete tokens extracted provide efficient and domain adaptable speech features. Their application to disordered speech that exhibits articulation imprecision and large mismatch a…

Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation

Mengzhe Geng, Xurong Xie, Jiajun Deng +7

The application of data-intensive automatic speech recognition (ASR) technologies to dysarthric and elderly adult speech is confronted by their mismatch against healthy and nonaged…

One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model

Zhaoqing Li, Haoning Xu, Tianzi Wang +7

We propose a novel one-pass multiple ASR systems joint compression and quantization approach using an all-in-one neural model. A single compression cycle allows multiple nested sys…

Joint Speaker Features Learning for Audio-visual Multichannel Speech Separation and Recognition

Guinan Li, Jiajun Deng, Youjun Chen +8

This paper proposes joint speaker feature learning methods for zero-shot adaptation of audio-visual multichannel speech separation and recognition systems. xVector and ECAPA-TDNN s…