10 papers
Toward Elastic Speech Inference: Training-Free Wake-Word Detection from Pretrained ASR
Hwayeon Kim, Youngwon Choi, Hyeonyu Kim
Recent ASR development has placed growing emphasis on generalization across diverse domains and acoustic conditions. Existing approaches typically adapt pretrained ASR models to fr…
Do Spoken Language Models Hear Speech as They Read Text? Bridging Structural Gaps Between Speech and Text
Hyeonyu Kim, Hwayeon Kim, Youngwon Choi +2
Spoken Language Models (SLMs) generate textual responses directly from speech, offering an alternative to cascaded systems. Despite recent advances, existing SLMs still exhibit wea…
Not All Redundant Tokens Are Alike: Analyzing Visual Token Pruning through Token Roles
Hyeonyu Kim, Sehwan Lim, Youngwon Choi +2
Vision-language models (VLMs) process an image as a sequence of visual tokens, which creates a substantial computational bottleneck during inference. Recent visual token pruning me…
CORTIS: Text-Only Adaptation of Spoken Language Models for Task-Oriented Voice Agents
Youngwon Choi, Hyeonyu Kim, Taeyoun Kwon +2
Task-oriented voice agents need to map spoken user requests to structured outputs such as semantic frames, executable actions, and function calls. A common approach is to cascade A…
Mine-JEPA: In-Domain Self-Supervised Learning for Mine-Like Object Classification in Side-Scan Sonar
Taeyoun Kwon, Youngwon Choi, Hyeonyu Kim +3
Side-scan sonar (SSS) mine classification is a challenging maritime vision problem characterized by extreme data scarcity and a large domain gap from natural images. While self-sup…
ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesis
Youngwon Choi, Jinwoo Oh, Hwayeon Kim +1
We investigate the use of zero-shot text-to-speech (ZS-TTS) as a data augmentation source for low-resource personalized speech synthesis. While synthetic augmentation can provide l…
Exploring Fine-Tuning of Large Audio Language Models for Spoken Language Understanding under Limited Speech Data
Youngwon Choi, Jaeyoon Jung, Hyeonyu Kim +2
Large Audio Language Models (LALMs) have emerged as powerful tools for speech-related tasks but remain underexplored for fine-tuning, especially with limited speech data. To bridge…
Audio-Guided Visual Editing with Complex Multi-Modal Prompts
Hyeonyu Kim, Seokhoon Jeong, Seonghee Han +2
Visual editing with diffusion models has made significant progress but often struggles with complex scenarios that textual guidance alone could not adequately describe, highlightin…
Generating Realistic Images from In-the-wild Sounds
Taegyeong Lee, Jeonghun Kang, Hyeonyu Kim +1
Representing wild sounds as images is an important but challenging task due to the lack of paired datasets between sound and images and the significant differences in the character…
Technical Report for CVPR 2022 LOVEU AQTC Challenge
Hyeonyu Kim, Jongeun Kim, Jeonghun Kang +3
This technical report presents the 2nd winning model for AQTC, a task newly introduced in CVPR 2022 LOng-form VidEo Understanding (LOVEU) challenges. This challenge faces difficult…