94 citations · 255 across the 63 of their papers we have counts for
73 papers
Uni-Stream: A Unified Streaming Speech Language Model for Speech Synthesis and Recognition
Xuan Shi, Chang Zeng, Tiantian Feng +5
Unified streaming speech language models aim to support speech recognition and synthesis within a single model, enabling real-time human-computer interaction. However, jointly supp…
Probing Neural Audio Codecs with Speaker Traits and Speech Characteristics
Jihwan Lee, Tiantian Feng, Minu Kim +6
Neural audio codecs are increasingly used for low-bitrate speech coding and serve as tokenizers for speech language models, yet it remains unclear how their codebook hierarchies en…
PEDBench: Can Speech Models Detect Phonetic Errors?
Zhonghao Shi, Zackary Rackauckas, Jasmine Yip +9
Phonetic error detection (PED) aims to identify phoneme-level substitutions, deletions, and insertions and is essential for educational and clinical speech assessment and intervent…
Towards Automated Clinical Behavioral Coding with Large Language Models: A Case study Using BOSCC recordings of Children
Jordan Prescott, Aditya Kommineni, Tiantian Feng +8
Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by differences in social communication and by restricted interests and repetitive behaviors. Treatmen…
MyoCodec: A Streaming Neural Codec for Electromyography
Jihwan Lee, Kleanthis Avramidis, Junhyeok Lee +3
Neural codecs encode continuous signals into compact sequences of discrete tokens, providing an interface for efficient transmission, storage, and token-based sequence modeling. Th…
Towards Interpretable Framework for Neural Audio Codecs via Sparse Autoencoders: Exploration toward Age, Gender, and Accent Steering
Shih-Heng Wang, Tiantian Feng, Aditya Kommineni +4
Neural audio codecs (NACs) are widely used in speech generation and audio-language modeling, yet how they encode speaker-trait information remains poorly understood. Prior work app…
Speaker Role and Language Diarization for Analyzing Multilingual Interviews for Language Proficiency of Older Adults
Anfeng Xu, Tiantian Feng, Kevin Huang +8
Automatic language proficiency assessment in the context of multilingual interview-based settings remains underexplored. In this work, we develop Whisper-based speaker-role and lan…
RRP-Voice: A Longitudinal Dataset and Benchmark for Recurrent Respiratory Papillomatosis Detection
Wenze Ren, Ke-Han Lu, Kai-Wei Chang +8
Deep learning has advanced pathological voice detection rapidly, yet rare laryngeal diseases remain underexplored due to data scarcity. Recurrent Respiratory Papillomatosis (RRP) e…
ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood
Tiantian Feng, Anfeng Xu, Xuan Shi +10
We present ChildVox, a novel benchmark for characterizing the diverse acoustic signals through which children communicate. Specifically, ChildVox follows the full developmental tra…
A Multi-dimensional Framework for Evaluating Generalization in EEG Foundation Models
Aditya Kommineni, Emily Zhou, Kleanthis Avramidis +2
Evaluating foundation models under appropriate adaptation settings is essential for understanding the quality and transferability of the learned representations. Recent EEG foundat…
Aperiodic and Low-Frequency Spectral Bias in Reconstruction based EEG Foundation Models
Aditya Kommineni, Emily Zhou, Kleanthis Avramidis +7
EEG foundation models, pre-trained on large-scale unlabelled EEG data, have emerged as a promising direction towards learning generalizable EEG representations. Despite showing pos…
Exploring Speech Foundation Models for Speaker Diarization Across Lifespan
Anfeng Xu, Tiantian Feng, Shrikanth Narayanan
Speech foundation models have shown strong transferability across a wide range of speech applications. However, their robustness to age-related domain shift in speaker diarization…
Smiling Regulates Emotion During Traumatic Recollection
Marcus Ma, Emily Zhou, Leonard Ludwig +7
We study when, where, and why 978 Holocaust survivors smile in video testimonies. We create an automatic smile detection model from facial features with an F1 of 85% and annotate d…
Affect Decoding in Phonated and Silent Speech Production from Surface EMG
Simon Pistrosch, Kleanthis Avramidis, Zhao Ren +7
The expression of affect is integral to spoken communication, yet, its link to underlying articulatory execution remains unclear. Measures of articulatory muscle activity such as E…
VoxCare: Studying Natural Communication Behaviors of Hospital Caregivers through Wearable Sensing of Egocentric Audio
Tiantian Feng, Kleanthis Avramidis, Anfeng Xu +3
Healthcare professionals work in complex, high-stakes environments where effective communication is critical for care delivery, team coordination, and individual well-being. Howeve…
Learning-free L2-Accented Speech Generation using Phonological Rules
Thanathai Lertpetchpun, Yoonjeong Lee, Jihwan Lee +3
Accent plays a crucial role in speaker identity and inclusivity in speech technologies. Existing accented text-to-speech (TTS) systems either require large-scale accented datasets…
Accent Vector: Controllable Accent Manipulation for Multilingual TTS Without Accented Data
Thanathai Lertpetchpun, Thanapat Trachu, Jihwan Lee +3
Accent is an integral part of society, reflecting multiculturalism and shaping how individuals express identity. The majority of English speakers are non-native (L2) speakers, yet…
Rethinking Visual Privacy: A Compositional Privacy Risk Framework for Severity Assessment with VLMs
Efthymios Tsaprazlis, Tiantian Feng, Anil Ramakrishna +3
Existing visual privacy benchmarks largely treat privacy as a binary property, labeling images as private or non-private based on visible sensitive content. We argue that privacy i…
Towards Interpretable Framework for Neural Audio Codecs via Sparse Autoencoders: A Case Study on Accent Information
Shih-Heng Wang, Tiantian Feng, Aditya Kommineni +4
Neural Audio Codecs (NACs) are widely adopted in modern speech systems, yet how they encode linguistic and paralinguistic information remains unclear. Improving the interpretabilit…
Speech Codec Probing from Semantic and Phonetic Perspectives
Xuan Shi, Chang Zeng, Tiantian Feng +3
Speech tokenizers are essential for connecting speech to large language models (LLMs) in multimodal systems. Speech tokenizers are expected to preserve both semantic and acoustic i…
An Approach to Simultaneous Acquisition of Real-Time MRI Video, EEG, and Surface EMG for Articulatory, Brain, and Muscle Activity During Speech Production
Jihwan Lee, Parsa Razmara, Kevin Huang +16
Speech production is a complex process spanning neural planning, motor control, muscle activation, and articulatory kinematics. While the acoustic speech signal is the most accessi…
Quantifying Speaker Embedding Phonological Rule Interactions in Accented Speech Synthesis
Thanathai Lertpetchpun, Yoonjeong Lee, Thanapat Trachu +4
Many spoken languages, including English, exhibit wide variation in dialects and accents, making accent control an important capability for flexible text-to-speech (TTS) models. Cu…
Encoding Emotion Through Self-Supervised Eye Movement Reconstruction
Marcus Ma, Jordan Prescott, Emily Zhou +4
The relationship between emotional expression and eye movement is well-documented, with literature establishing gaze patterns are reliable indicators of emotion. However, most stud…
VoxCog: Towards End-to-End Multilingual Cognitive Impairment Classification through Dialectal Knowledge
Tiantian Feng, Anfeng Xu, Jinkook Lee +1
In this work, we present a novel perspective on cognitive impairment classification from speech by integrating speech foundation models that explicitly recognize speech dialects. O…