16 citations · 60 across the 78 of their papers we have counts for
31 papers · 1 filter
HASS: Hierarchical Simulation of Logopenic Aphasic Speech for Scalable PPA Detection
Harrison Li, Kevin Wang, Cheol Jun Cho +13
Building a diagnosis model for primary progressive aphasia (PPA) has been challenging due to the data scarcity. Collecting clinical data at scale is limited by the high vulnerabili…
HuPER: A Human-Inspired Framework for Phonetic Perception
Chenxu Guo, Jiachen Lian, Yisi Liu +9
We propose HuPER, a human-inspired framework that models phonetic perception as adaptive inference over acoustic-phonetics evidence and linguistic knowledge. With only 100 hours of…
Sylber 2.0: A Universal Syllable Embedding
Cheol Jun Cho, Nicholas Lee, Alan W Black +1
Scaling spoken language modeling requires speech tokens that are both efficient and universal. Recent work has proposed syllables as promising speech tokens at low temporal resolut…
Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech
Xuanru Zhou, Jiachen Lian, Henry Hong +2
Current speech-language models (SLMs) typically use a cascade of speech encoder and large language model, treating speech understanding as a single black box. They analyze the cont…
Teaching Machines to Speak Using Articulatory Control
Akshay Anand, Chenxu Guo, Cheol Jun Cho +2
Current speech production systems predominantly rely on large transformer models that operate as black boxes, providing little interpretability or grounding in the physical mechani…
LCS-CTC: Leveraging Soft Alignments to Enhance Phonetic Transcription Robustness
Zongli Ye, Jiachen Lian, Akshaj Gupta +18
Phonetic speech transcription is crucial for fine-grained linguistic analysis and downstream speech applications. While Connectionist Temporal Classification (CTC) is a widely used…
Towards Accurate Phonetic Error Detection Through Phoneme Similarity Modeling
Xuanru Zhou, Jiachen Lian, Cheol Jun Cho +12
Phonetic error detection, a core subtask of automatic pronunciation assessment, identifies pronunciation deviations at the phoneme level. Speech variability from accents and dysflu…
Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detection
Jinming Zhang, Xuanru Zhou, Jiachen Lian +10
Speech dysfluency detection is crucial for clinical diagnosis and language assessment, but existing methods are limited by the scarcity of high-quality annotated data. Although rec…
RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Coding
Yisi Liu, Chenyang Wang, Hanjo Kim +2
Voice conversion has emerged as a pivotal technology in numerous applications ranging from assistive communication to entertainment. In this paper, we present RT-VC, a zero-shot re…
Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis
Zongli Ye, Jiachen Lian, Xuanru Zhou +14
Accurate alignment of dysfluent speech with intended text is crucial for automating the diagnosis of neurodegenerative speech disorders. Traditional methods often fail to model pho…
Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detection
Chenxu Guo, Jiachen Lian, Xuanru Zhou +13
Automatic detection of speech dysfluency aids speech-language pathologists in efficient transcription of disordered speech, enhancing diagnostics and treatment planning. Traditiona…
Deep Speech Synthesis from Multimodal Articulatory Representations
Peter Wu, Bohan Yu, Kevin Scheck +6
The amount of articulatory data available for training deep learning models is much less compared to acoustic speech data. In order to improve articulatory-to-acoustic synthesis pe…
SSDM 2.0: Time-Accurate Speech Rich Transcription with Non-Fluencies
Jiachen Lian, Xuanru Zhou, Zoe Ezzes +6
Speech is a hierarchical collection of text, prosody, emotions, dysfluencies, etc. Automatic transcription of speech that goes beyond text (words) is an underexplored problem. We f…
Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection
Xuanru Zhou, Jiachen Lian, Cheol Jun Cho +10
Speech dysfluency modeling is a task to detect dysfluencies in speech, such as repetition, block, insertion, replacement, and deletion. Most recent advancements treat this problem…
Stutter-Solver: End-to-end Multi-lingual Dysfluency Detection
Xuanru Zhou, Cheol Jun Cho, Ayati Sharma +9
Current de-facto dysfluency modeling methods utilize template matching algorithms which are not generalizable to out-of-domain real-world dysfluencies across languages, and are not…
YOLO-Stutter: End-to-end Region-Wise Speech Dysfluency Detection
Xuanru Zhou, Anshul Kashyap, Steve Li +9
Dysfluent speech detection is the bottleneck for disordered speech analysis and spoken language learning. Current state-of-the-art models are governed by rule-based systems which l…
Fast, High-Quality and Parameter-Efficient Articulatory Synthesis using Differentiable DSP
Yisi Liu, Bohan Yu, Drake Lin +3
Articulatory trajectories like electromagnetic articulography (EMA) provide a low-dimensional representation of the vocal tract filter and have been used as natural, grounded featu…
SSDM: Scalable Speech Dysfluency Modeling
Jiachen Lian, Xuanru Zhou, Zoe Ezzes +6
Speech dysfluency modeling is the core module for spoken language learning, and speech therapy. However, there are three challenges. First, current state-of-the-art solutions\cite{…
Towards EMG-to-Speech with a Necklace Form Factor
Peter Wu, Ryan Kaveh, Raghav Nautiyal +8
Electrodes for decoding speech from electromyography (EMG) are typically placed on the face, requiring adhesives that are inconvenient and skin-irritating if used regularly. We exp…
Coding Speech through Vocal Tract Kinematics
Cheol Jun Cho, Peter Wu, Tejas S. Prabhune +2
Vocal tract articulation is a natural, grounded control space of speech production. The spatiotemporal coordination of articulators combined with the vocal source shapes intelligib…
Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detection
Jiachen Lian, Carly Feng, Naasir Farooqi +7
Dysfluent speech modeling requires time-accurate and silence-aware transcription at both the word-level and phonetic-level. However, current research in dysfluency modeling primari…
Self-Supervised Models of Speech Infer Universal Articulatory Kinematics
Cheol Jun Cho, Abdelrahman Mohamed, Alan W Black +1
Self-Supervised Learning (SSL) based models of speech have shown remarkable performance on a range of downstream tasks. These state-of-the-art models have remained blackboxes, but…
Deep Speech Synthesis from MRI-Based Articulatory Representations
Peter Wu, Tingle Li, Yijing Lu +6
In this paper, we study articulatory synthesis, a speech synthesis method using human vocal tract information that offers a way to develop efficient, generalizable and interpretabl…
Speaker-Independent Acoustic-to-Articulatory Speech Inversion
Peter Wu, Li-Wei Chen, Cheol Jun Cho +4
To build speech processing methods that can handle speech as naturally as humans, researchers have explored multiple ways of building an invertible mapping from speech to an interp…