5 papers
Listen, Critique, and Refine: RL-Based Self-Refinement for Instruction-Following Speech Synthesis
Chee-En Yu, Yi-Cheng Lin, Sung-Feng Huang +4
Large Audio Language Models (LALMs) can follow diverse instructions to synthesize speech in specified styles. However, complex instructions that require simultaneous control over p…
Hearing Like Humans? Sound Symbolism and Perceptual Alignment in Speech Language Models
Yun-Shao Tsai, Chun-Wei Chen, Chee-En Yu +2
Sound symbolism, the human tendency to map speech sounds to perceptual qualities such as roundness or sharpness, arises primarily from the acoustics of speech rather than spelling.…
Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Models
Wei-Ping Huang, Chee-En Yu, Guan-Ting Lin +1
Test-Time Adaptation (TTA) via entropy minimization (EM) has proven effective for classification tasks, yet its application to generative autoregressive models remains theoreticall…
DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment
Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu +25
We introduce DeSTA2.5-Audio, a general-purpose Large Audio Language Model (LALM) designed for robust auditory perception and instruction-following. Recent LALMs augment Large Langu…
Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks
Chien-yu Huang, Wei-Chih Chen, Shu-wen Yang +77
Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spo…