8 papers
ParaAnya: Accelerating Parallel Diffusion Sampling with Plug-and-Play Output Caching
Chee-En Yu, Xiao-Xi Tan, Yi-Cheng Lin +3
Diffusion models have achieved remarkable success in generative tasks, but their inherently sequential sampling process introduces a severe computational bottleneck. Recent Paralle…
TASTE2: Text-Aligned Speech Modeling and Deployment toward Full-Duplex Voice Interaction
Yi-Chang Chen, Chun Wei Chen, Dien-Ruei Wu +7
Full-duplex voice interaction requires more than utterance-level conversion. It must process streaming speech, manage turn-taking and interruptions, while preserving pretrained lin…
Hearing Like Humans? Sound Symbolism and Perceptual Alignment in Speech Language Models
Yun-Shao Tsai, Chun-Wei Chen, Chee-En Yu +2
Sound symbolism, the human tendency to map speech sounds to perceptual qualities such as roundness or sharpness, arises primarily from the acoustics of speech rather than spelling.…
Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models
Chun-Yi Kuan, Wei-Ping Huang, Hung-yi Lee
Recent audio-aware large language models (ALLMs) have demonstrated strong capabilities across diverse audio understanding and reasoning tasks, but they still frequently produce hal…
The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluation
Yun-Shao Tsai, Yi-Cheng Lin, Huang-Cheng Chou +5
Objective metrics for emotional expressiveness are vital for speech generation, particularly in expressive synthesis and voice conversion requiring emotional prosody transfer. To q…
TASTE-Streaming: Towards Streamable Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
Liang-Hsuan Tseng, Hung-yi Lee
Text-speech joint spoken language modeling (SLM) aims at natural and intelligent speech-based interactions, but developing such a system may suffer from modality mismatch: speech u…
On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation
Chan-Jan Hsu, Liang-Hsuan Tseng, Yi-Cheng Lin +5
Generative spoken language models pretrained on large-scale raw audio can continue a speech prompt with appropriate content while preserving attributes like speaker and emotion, se…
ParaS2S: Benchmarking and Aligning Spoken Language Models for Paralinguistic-aware Speech-to-Speech Interaction
Shu-wen Yang, Ming Tu, Andy T. Liu +5
Speech-to-Speech (S2S) models have shown promising dialogue capabilities, but their ability to handle paralinguistic cues - such as emotion, tone, and speaker attributes - and to r…