16 citations · 60 across the 78 of their papers we have counts for
83 papers
StyleStream 2.0: Fast and Controllable Streaming Voice Style Conversion
Yisi Liu, Nicholas Lee, Gopala K. Anumanchipalli
Voice style conversion (VSC) aims to transform a source utterance to match the timbre, accent, and emotion of a target voice while preserving linguistic content. StyleStream 1.0 in…
KIRA: A Full-Stack Reading Screening System with Dual-Pathway Analysis for Children
Maggie Dong, Rohit Mekkoth, Zixuan Wang +3
Early-grade reading screening is critical for identifying children at risk of reading difficulty. Existing AI-based screeners are often limited in the scope of their analyses and e…
AVIO: Learning to Add and Remove Sounding Objects in Audiovisual Scenes
Weihan Xu, Kan Jen Cheng, Koichi Saito +8
Adding or removing a sounding object requires coordinated changes to visual content and sound while preserving the surrounding scene. Yet paired supervision for localized non-speec…
TART: A Modular Tool for Technique-Aware Audio-to-Tablature Guitar Transcription
Akshaj Gupta, Hwi Joo Park, Andrea Guzman +5
Automatic Music Transcription (AMT) for guitar remains limited by three challenges: existing systems often fail to capture expressive techniques such as slides, bends, and percussi…
Automated Discovery Has No Universally Superior Harness
Akshat Gupta, Jermaine Lei, Alexander Lu +2
Autonomous discovery systems such as OpenEvolve and TTT-Discover are often used as general-purpose harnesses. However, in practice these are composite systems combining several des…
Benchmarking Single-Factor Physical Video-to-Audio Generation
Tingle Li, Siddharth Gururani, Kevin J. Shih +6
Generative video-to-audio (V2A) models produce highly plausible soundtracks, but it remains unclear whether they capture the underlying physical processes. Existing evaluations emp…
HASS: Hierarchical Simulation of Logopenic Aphasic Speech for Scalable PPA Detection
Harrison Li, Kevin Wang, Cheol Jun Cho +13
Building a diagnosis model for primary progressive aphasia (PPA) has been challenging due to the data scarcity. Collecting clinical data at scale is limited by the high vulnerabili…
StyleStream: Real-Time Zero-Shot Voice Style Conversion
Yisi Liu, Nicholas Lee, Gopala Anumanchipalli
Voice style conversion aims to transform an input utterance to match a target speaker's timbre, accent, and emotion, with a central challenge being the disentanglement of linguisti…
HuPER: A Human-Inspired Framework for Phonetic Perception
Chenxu Guo, Jiachen Lian, Yisi Liu +9
We propose HuPER, a human-inspired framework that models phonetic perception as adaptive inference over acoustic-phonetics evidence and linguistic knowledge. With only 100 hours of…
S-MARC: Causal Streaming Reasoning for Full-Duplex Conversational Behavior Modeling
Dingkun Zhou, Shuchang Pan, Jiachen Lian +11
Human conversation is organized by an implicit chain of thought and manifests as temporally structured conversational behaviors. Capturing this perceptual pathway is critical for b…
Asymmetric Hierarchical Anchoring for Robust Audio-Visual Cross-Modal Generalization
Bixing Wu, Yuhong Zhao, Zongli Ye +3
Audio-visual joint representation learning under Cross-Modal Generalization (CMG) aims to transfer knowledge from a labeled source modality to an unlabeled target modality through…
Sylber 2.0: A Universal Syllable Embedding
Cheol Jun Cho, Nicholas Lee, Alan W Black +1
Scaling spoken language modeling requires speech tokens that are both efficient and universal. Recent work has proposed syllables as promising speech tokens at low temporal resolut…
Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
Immanuel Abdi, Akshat Gupta, Micah Mok +3
One of the biggest missing capabilities in current AI systems is the ability to learn continuously after deployment. Implementing such continually learning systems have several cha…
Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speech
Shuchang Pan, Siddharth Banerjee, Dhruv Hebbar +9
Human conversation is organized by an implicit chain of thoughts that manifests as timed speech acts. Capturing this causal pathway is key to building natural full-duplex interacti…
Schrodinger Audio-Visual Editor: Object-Level Audiovisual Removal
Weihan Xu, Kan Jen Cheng, Koichi Saito +10
Joint editing of audio and visual content is crucial for precise and controllable content creation. This new task poses challenges due to the limitations of paired audio-visual dat…
Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech
Xuanru Zhou, Jiachen Lian, Henry Hong +2
Current speech-language models (SLMs) typically use a cascade of speech encoder and large language model, treating speech understanding as a single black box. They analyze the cont…
Teaching Machines to Speak Using Articulatory Control
Akshay Anand, Chenxu Guo, Cheol Jun Cho +2
Current speech production systems predominantly rely on large transformer models that operate as black boxes, providing little interpretability or grounding in the physical mechani…
TART: A Comprehensive Tool for Technique-Aware Audio-to-Tab Guitar Transcription
Akshaj Gupta, Andrea Guzman, Anagha Badriprasad +5
Automatic Music Transcription (AMT) has advanced significantly for the piano, but transcription for the guitar remains limited due to several key challenges. Existing systems fail…
How Do LLMs Use Their Depth?
Akshat Gupta, Jay Yeung, Gopala Anumanchipalli +1
Growing evidence suggests that large language models do not use their depth uniformly, yet we still lack a fine-grained understanding of their layer-wise prediction dynamics. In th…
AV-EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Omni-modal LLMS with Audio-visual Cues
Dingkun Zhou, Krish Patel, Ajay Kankipati +14
Emotions conveyed through voice and face shape engagement and context in human AI interaction. Despite rapid progress in omni modal large language models, the holistic evaluation o…
The Oracle Has Spoken: A Multi-Aspect Evaluation of Dialogue in Pythia
Zixun Chen, Petr Babkin, Akshat Gupta +2
Dialogue is one of the landmark abilities of large language models (LLMs). Despite its ubiquity, few studies actually distinguish specific ingredients underpinning dialogue behavio…
Scaling Spoken Language Models with Syllabic Speech Tokenization
Nicholas Lee, Cheol Jun Cho, Alan W Black +1
Spoken language models (SLMs) typically discretize speech into high-frame-rate tokens extracted from SSL speech models. As the most successful LMs are based on the Transformer arch…
EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Spoken Dialogue Systems
Jingwen Liu, Kan Jen Cheng, Jiachen Lian +8
Speech emotions play a crucial role in human-computer interaction, shaping engagement and context-aware communication. Despite recent advances in spoken dialogue systems, a holisti…
LCS-CTC: Leveraging Soft Alignments to Enhance Phonetic Transcription Robustness
Zongli Ye, Jiachen Lian, Akshaj Gupta +18
Phonetic speech transcription is crucial for fine-grained linguistic analysis and downstream speech applications. While Connectionist Temporal Classification (CTC) is a widely used…