9 papers
EgoPro-Bench: Benchmarking Personalized Proactive Interaction in Egocentric Video Streams
Dongchuan Ran, Linyu Ou, Xueheng Li +5
Existing Multimodal Large Language Models (MLLMs) remain primarily reactive, failing to continuously perceive environments or proactively assist users. While emerging benchmarks ad…
HASS: Hierarchical Simulation of Logopenic Aphasic Speech for Scalable PPA Detection
Harrison Li, Kevin Wang, Cheol Jun Cho +13
Building a diagnosis model for primary progressive aphasia (PPA) has been challenging due to the data scarcity. Collecting clinical data at scale is limited by the high vulnerabili…
HuPER: A Human-Inspired Framework for Phonetic Perception
Chenxu Guo, Jiachen Lian, Yisi Liu +9
We propose HuPER, a human-inspired framework that models phonetic perception as adaptive inference over acoustic-phonetics evidence and linguistic knowledge. With only 100 hours of…
Teaching Machines to Speak Using Articulatory Control
Akshay Anand, Chenxu Guo, Cheol Jun Cho +2
Current speech production systems predominantly rely on large transformer models that operate as black boxes, providing little interpretability or grounding in the physical mechani…
InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue
Wenwen Tong, Hewei Guo, Dongchuan Ran +23
We introduce InteractiveOmni, a unified and open-source omni-modal large language model for audio-visual multi-turn interaction, ranging from 4B to 8B parameters, designed to lead…
LCS-CTC: Leveraging Soft Alignments to Enhance Phonetic Transcription Robustness
Zongli Ye, Jiachen Lian, Akshaj Gupta +18
Phonetic speech transcription is crucial for fine-grained linguistic analysis and downstream speech applications. While Connectionist Temporal Classification (CTC) is a widely used…
K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Function
Shuhe Li, Chenxu Guo, Jiachen Lian +18
Evaluating young children's language is challenging for automatic speech recognizers due to high-pitched voices, prolonged sounds, and limited data. We introduce K-Function, a fram…
Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis
Zongli Ye, Jiachen Lian, Xuanru Zhou +14
Accurate alignment of dysfluent speech with intended text is crucial for automating the diagnosis of neurodegenerative speech disorders. Traditional methods often fail to model pho…
Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detection
Chenxu Guo, Jiachen Lian, Xuanru Zhou +13
Automatic detection of speech dysfluency aids speech-language pathologists in efficient transcription of disordered speech, enhancing diagnostics and treatment planning. Traditiona…