8 citations · 15 across the 33 of their papers we have counts for
37 papers
KIRA: A Full-Stack Reading Screening System with Dual-Pathway Analysis for Children
Maggie Dong, Rohit Mekkoth, Zixuan Wang +3
Early-grade reading screening is critical for identifying children at risk of reading difficulty. Existing AI-based screeners are often limited in the scope of their analyses and e…
HASS: Hierarchical Simulation of Logopenic Aphasic Speech for Scalable PPA Detection
Harrison Li, Kevin Wang, Cheol Jun Cho +13
Building a diagnosis model for primary progressive aphasia (PPA) has been challenging due to the data scarcity. Collecting clinical data at scale is limited by the high vulnerabili…
HuPER: A Human-Inspired Framework for Phonetic Perception
Chenxu Guo, Jiachen Lian, Yisi Liu +9
We propose HuPER, a human-inspired framework that models phonetic perception as adaptive inference over acoustic-phonetics evidence and linguistic knowledge. With only 100 hours of…
S-MARC: Causal Streaming Reasoning for Full-Duplex Conversational Behavior Modeling
Dingkun Zhou, Shuchang Pan, Jiachen Lian +11
Human conversation is organized by an implicit chain of thought and manifests as temporally structured conversational behaviors. Capturing this perceptual pathway is critical for b…
Asymmetric Hierarchical Anchoring for Robust Audio-Visual Cross-Modal Generalization
Bixing Wu, Yuhong Zhao, Zongli Ye +3
Audio-visual joint representation learning under Cross-Modal Generalization (CMG) aims to transfer knowledge from a labeled source modality to an unlabeled target modality through…
Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speech
Shuchang Pan, Siddharth Banerjee, Dhruv Hebbar +9
Human conversation is organized by an implicit chain of thoughts that manifests as timed speech acts. Capturing this causal pathway is key to building natural full-duplex interacti…
Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech
Xuanru Zhou, Jiachen Lian, Henry Hong +2
Current speech-language models (SLMs) typically use a cascade of speech encoder and large language model, treating speech understanding as a single black box. They analyze the cont…
Teaching Machines to Speak Using Articulatory Control
Akshay Anand, Chenxu Guo, Cheol Jun Cho +2
Current speech production systems predominantly rely on large transformer models that operate as black boxes, providing little interpretability or grounding in the physical mechani…
TART: A Comprehensive Tool for Technique-Aware Audio-to-Tab Guitar Transcription
Akshaj Gupta, Andrea Guzman, Anagha Badriprasad +5
Automatic Music Transcription (AMT) has advanced significantly for the piano, but transcription for the guitar remains limited due to several key challenges. Existing systems fail…
AV-EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Omni-modal LLMS with Audio-visual Cues
Dingkun Zhou, Krish Patel, Ajay Kankipati +14
Emotions conveyed through voice and face shape engagement and context in human AI interaction. Despite rapid progress in omni modal large language models, the holistic evaluation o…
EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Spoken Dialogue Systems
Jingwen Liu, Kan Jen Cheng, Jiachen Lian +8
Speech emotions play a crucial role in human-computer interaction, shaping engagement and context-aware communication. Despite recent advances in spoken dialogue systems, a holisti…
LCS-CTC: Leveraging Soft Alignments to Enhance Phonetic Transcription Robustness
Zongli Ye, Jiachen Lian, Akshaj Gupta +18
Phonetic speech transcription is crucial for fine-grained linguistic analysis and downstream speech applications. While Connectionist Temporal Classification (CTC) is a widely used…
Towards Accurate Phonetic Error Detection Through Phoneme Similarity Modeling
Xuanru Zhou, Jiachen Lian, Cheol Jun Cho +12
Phonetic error detection, a core subtask of automatic pronunciation assessment, identifies pronunciation deviations at the phoneme level. Speech variability from accents and dysflu…
Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models
Guan-Ting Lin, Shih-Yun Shan Kuan, Qirui Wang +4
Full-duplex spoken dialogue systems promise to transform human-machine interaction from a rigid, turn-based protocol into a fluid, natural conversation. However, the central challe…
K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Function
Shuhe Li, Chenxu Guo, Jiachen Lian +18
Evaluating young children's language is challenging for automatic speech recognizers due to high-pitched voices, prolonged sounds, and limited data. We introduce K-Function, a fram…
Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detection
Jinming Zhang, Xuanru Zhou, Jiachen Lian +10
Speech dysfluency detection is crucial for clinical diagnosis and language assessment, but existing methods are limited by the scarcity of high-quality annotated data. Although rec…
Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis
Zongli Ye, Jiachen Lian, Xuanru Zhou +14
Accurate alignment of dysfluent speech with intended text is crucial for automating the diagnosis of neurodegenerative speech disorders. Traditional methods often fail to model pho…
Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detection
Chenxu Guo, Jiachen Lian, Xuanru Zhou +13
Automatic detection of speech dysfluency aids speech-language pathologists in efficient transcription of disordered speech, enhancing diagnostics and treatment planning. Traditiona…
Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities
Guan-Ting Lin, Jiachen Lian, Tingle Li +4
Spoken dialogue modeling poses challenges beyond text-based language modeling, requiring real-time interaction, turn-taking, and backchanneling. While most Spoken Dialogue Models (…
SSDM 2.0: Time-Accurate Speech Rich Transcription with Non-Fluencies
Jiachen Lian, Xuanru Zhou, Zoe Ezzes +6
Speech is a hierarchical collection of text, prosody, emotions, dysfluencies, etc. Automatic transcription of speech that goes beyond text (words) is an underexplored problem. We f…
Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection
Xuanru Zhou, Jiachen Lian, Cheol Jun Cho +10
Speech dysfluency modeling is a task to detect dysfluencies in speech, such as repetition, block, insertion, replacement, and deletion. Most recent advancements treat this problem…
Stutter-Solver: End-to-end Multi-lingual Dysfluency Detection
Xuanru Zhou, Cheol Jun Cho, Ayati Sharma +9
Current de-facto dysfluency modeling methods utilize template matching algorithms which are not generalizable to out-of-domain real-world dysfluencies across languages, and are not…
YOLO-Stutter: End-to-end Region-Wise Speech Dysfluency Detection
Xuanru Zhou, Anshul Kashyap, Steve Li +9
Dysfluent speech detection is the bottleneck for disordered speech analysis and spoken language learning. Current state-of-the-art models are governed by rule-based systems which l…
SSDM: Scalable Speech Dysfluency Modeling
Jiachen Lian, Xuanru Zhou, Zoe Ezzes +6
Speech dysfluency modeling is the core module for spoken language learning, and speech therapy. However, there are three challenges. First, current state-of-the-art solutions\cite{…