6 citations · 7 across the 11 of their papers we have counts for
10 papers · 1 filter
The WER Trap: Shattering the Illusion of Unified Tokens in Speech Language Models
Xiangyu Zhang, Yuxin Li, Haoyang Zhang +5
The pursuit of a "unified" discrete token for both speech understanding and generation has led the Speech Language Model (SLM) community to heavily rely on Word Error Rate (WER) --…
Why Your Tokenizer Fails in Information Fusion: A Timing-Aware Pre-Quantization Fusion for Video-Enhanced Audio Tokenization
Xiangyu Zhang, Benjamin John Southwell, Siqi Pan +3
Audio tokenization has emerged as a critical component in end-to-end audio language models, enabling efficient discrete representation learning for both audio understanding and gen…
Distinctive Feature Codec: An Adaptive Efficient Speech Representation for Depression Detection
Xiangyu Zhang, Fuming Fang, Peng Gao +3
Large Language Models (LLMs) have demonstrated remarkable success across diverse fields, establishing a powerful paradigm for complex information processing. This has inspired the…
Why Pre-trained Models Fail: Feature Entanglement in Multi-modal Depression Detection
Xiangyu Zhang, Beena Ahmed, Julien Epps
Depression remains a pressing global mental health issue, driving considerable research into AI-driven detection approaches. While pre-trained models, particularly speech self-supe…
SpeechT-RAG: Reliable Depression Detection in LLMs with Retrieval-Augmented Generation Using Speech Timing Information
Xiangyu Zhang, Hexin Liu, Qiquan Zhang +2
Large Language Models (LLMs) have been increasingly adopted for health-related tasks, yet their performance in depression detection remains limited when relying solely on text inpu…
Rethinking Mamba in Speech Processing by Self-Supervised Models
Xiangyu Zhang, Jianbo Ma, Mostafa Shahin +2
The Mamba-based model has demonstrated outstanding performance across tasks in computer vision, natural language processing, and speech processing. However, in the realm of speech…
Auto-Landmark: Acoustic Landmark Dataset and Open-Source Toolkit for Landmark Extraction
Xiangyu Zhang, Daijiao Liu, Tianyi Xiao +5
In the speech signal, acoustic landmarks identify times when the acoustic manifestations of the linguistically motivated distinctive features are most salient. Acoustic landmarks h…
Mamba in Speech: Towards an Alternative to Self-Attention
Xiangyu Zhang, Qiquan Zhang, Hexin Liu +6
Transformer and its derivatives have achieved success in diverse tasks across computer vision, natural language processing, and speech processing. To reduce the complexity of compu…
When LLMs Meets Acoustic Landmarks: An Efficient Approach to Integrate Speech into Large Language Models for Depression Detection
Xiangyu Zhang, Hexin Liu, Kaishuai Xu +4
Depression is a critical concern in global mental health, prompting extensive research into AI-based detection methods. Among various AI technologies, Large Language Models (LLMs)…
Phonological Level wav2vec2-based Mispronunciation Detection and Diagnosis Method
Mostafa Shahin, Julien Epps, Beena Ahmed
The automatic identification and analysis of pronunciation errors, known as Mispronunciation Detection and Diagnosis (MDD) plays a crucial role in Computer Aided Pronunciation Lear…