14 papers
Invariant Measures for Interacting Continuous-State Population Models
Lina Ji, Jie Xiong, Wen Xu +1
We study invariant probabilities for nonlinear two-population branching systems with competition, cooperation, or predator--prey interaction, driven by Brownian noise and spectrall…
Acoustic-to-Text KV Compression for Full-Duplex Speech Models
Yejin Lee, Seungbeom Kim, Yongha Lee +1
Full-duplex speech language models continuously accumulate acoustic key-value (KV) states, making long-running interactions memory-intensive. During listening, the model can finish…
DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech
Junwon Moon, Yejin Lee, Seungbeom Kim +4
Autoregressive (AR) text-to-speech (TTS) models generate discrete speech tokens sequentially, which makes inference slow and can degrade robustness, since local errors propagate to…
Beyond Binary Instrument QA: Probing Instrument Grounding in Music Audio-Language Models
Yujun Lee, Joonhyeok Shin, Hyoeun Kim +1
Recent music audio-language models achieve high accuracy on instrument question-answering benchmarks, but it remains unclear whether this reflects robust audio grounding or benchma…
TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech
Yejin Lee, Junwon Moon, Hyoeun Kim +3
Codec-based autoregressive (AR) speech language models have achieved strong text-to-speech (TTS) quality by modeling speech as sequences of discrete audio tokens with large pretrai…
GLASS: GRPO-Trained LoRA for Acoustic Style Steering in Zero-Shot Text-to-Speech
Jaehoon Kang, Yejin Lee, Kyuhong Shim
We propose GLASS, a framework for composable acoustic style control in zero-shot autoregressive text-to-speech (TTS) that learns controls from post-generation rewards rather than s…
Feedback Adaptation for Retrieval-Augmented Generation
Jihwan Bang, Seunghan Yang, Kyuhong Shim +3
Retrieval-Augmented Generation (RAG) systems are typically evaluated under static assumptions, despite being frequently corrected through user or expert feedback in deployment. Exi…
Unlocking Fine-Grained and Within-Utterance Speaking Style Control in Prompt-Based Text-to-Speech Models
Jaehoon Kang, Yejin Lee, Yoonji Park +1
While prompt-based text-to-speech (TTS) models enable natural language-driven speaking style control, they often provide limited fine-grained control and apply a single global styl…
DialBGM: A Benchmark for Background Music Recommendation from Everyday Multi-Turn Dialogues
Joonhyeok Shin, Jaehoon Kang, Yujun Lee +4
Selecting an appropriate background music (BGM) that supports natural human conversation is a common production step in media and interactive systems. In this paper, we introduce d…
WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Models
Hanna Lee, Tan Dat Nguyen, Jaehoon Kang +1
Recent decoder-only autoregressive text-to-speech (AR-TTS) models produce high-fidelity speech, but their memory and compute costs scale quadratically with sequence length due to f…
Mask2Flow-TSE: Two-Stage Target Speaker Extraction with Masking and Flow Matching
Junwon Moon, Seungbeom Kim, Hansol Park +4
Target speaker extraction (TSE) extracts the target speaker's voice from overlapping speech given a reference utterance. Existing masking-based approaches are lightweight and effec…
Whisper-CD: Accurate Long-Form Speech Recognition using Multi-Negative Contrastive Decoding
Hoseong Ahn, Jeongyun Chae, Yoonji Park +1
Long-form speech recognition with large encoder-decoder models such as Whisper often exhibit hallucinations, repetition loops, and content omissions. These errors can accumulate an…
Evaluating Hallucinations in Audio-Visual Multimodal LLMs with Spoken Queries under Diverse Acoustic Conditions
Hansol Park, Hoseong Ahn, Junwon Moon +2
Hallucinations in multimodal models have been extensively studied using benchmarks that probe reliability in image-text query settings. However, the effect of spoken queries on mul…
P2VA: Converting Persona Descriptions into Voice Attributes for Fair and Controllable Text-to-Speech
Yejin Lee, Jaehoon Kang, Kyuhong Shim
While persona-driven large language models (LLMs) and prompt-based text-to-speech (TTS) systems have advanced significantly, a usability gap arises when users attempt to generate v…