23 citations · 84 across the 24 of their papers we have counts for
28 papers
Speaking Longer Than Learned: Training-Free Long-Form Generation for Full-Stream Text-to-Speech
Jeseung Lim, Jiseong Woo, Heeseung Kim
Recent full-stream text-to-speech (TTS) models enable low-latency personalized synthesis as input text arrives. However, sustaining generation beyond the lengths seen during traini…
Beyond Fixed Targets: On-Policy One-Shot Speaker Personalization for Autoregressive Text-To-Speech Models
Sangkwon Park, Che Hyun Lee, Heeseung Kim +1
One-shot personalization of autoregressive (AR) text-to-speech (TTS) models remains challenging because supervised fine-tuning (SFT) on a single reference pair rapidly degrades gen…
SSEmb: Training a Generalizable Speech Retrieval Model Using Only Automatic Speech Recognition Data
Sangkwon Park, Dongwook Lee, Youngho Cho +3
End-to-end speech-to-text retrieval is becoming an important capability for voice agents, allowing spoken queries to retrieve relevant documents without intermediate transcription.…
Phrase-Localized Language-Contrastive Guidance: Training-Free Localized Accent Control for Code-Switching Text-to-Speech
Che Hyun Lee, Sangkwon Park, Donghun Kang +4
Current speech synthesis struggles with code-switching, which mixes a foreign language phrase into a primary language utterance, causing the phrase to be spoken with the primary la…
HEAR Who Said What: Unlocking Speaker-Attributed Reasoning via Counterfactual Voice Grounding
Dongwook Lee, Sangkwon Park, Eunwoo Song +6
Speech Language Models (SLMs) are increasingly deployed in multi-speaker environments, yet their ability to attribute speech to the correct speaker and reason over speaker identiti…
DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech
Junwon Moon, Yejin Lee, Seungbeom Kim +4
Autoregressive (AR) text-to-speech (TTS) models generate discrete speech tokens sequentially, which makes inference slow and can degrade robustness, since local errors propagate to…
From Awareness to Adherence: Bridging the Context Gap in Spoken Dialogue Systems via Context-Aware Decoding
Che Hyun Lee, Heeseung Kim, Sungroh Yoon
Despite the success of end-to-end (E2E) spoken dialogue systems, maintaining strict context adherence in multi-round conversations remains a challenge. While prior works attribute…
NaturalFlow: Reducing Disruptive Pauses for Natural Speech Flow in Simultaneous Speech-to-Speech Translation
Dongwook Lee, Youngho Cho, Sangkwon Park +2
Simultaneous speech-to-speech translation aims to enable near-real-time communication by minimizing latency, offering a compelling, real-time alternative to the high latency of con…
TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech
Yejin Lee, Junwon Moon, Hyoeun Kim +3
Codec-based autoregressive (AR) speech language models have achieved strong text-to-speech (TTS) quality by modeling speech as sequences of discrete audio tokens with large pretrai…
Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalization
Yeongtak Oh, Dongwook Lee, Sangkwon Park +2
While multimodal large language models have advanced across text, image, and audio, personalization research has remained primarily vision-language, with unified omnimodal benchmar…
Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions
Dongwook Lee, Eunwoo Song, Che Hyun Lee +2
While recent Spoken Language Models (SLMs) have been actively deployed in real-world scenarios, they lack the capability to discern Third-Party Interruptions (TPI) from the primary…
Mask2Flow-TSE: Two-Stage Target Speaker Extraction with Masking and Flow Matching
Junwon Moon, Seungbeom Kim, Hansol Park +4
Target speaker extraction (TSE) extracts the target speaker's voice from overlapping speech given a reference utterance. Existing masking-based approaches are lightweight and effec…
EdiText: Controllable Coarse-to-Fine Text Editing with Diffusion Language Models
Che Hyun Lee, Heeseung Kim, Jiheum Yeom +1
We propose EdiText, a controllable text editing method that modifies the reference text to desired attributes at various scales. We integrate an SDEdit-based editing technique that…
Does Your Voice Assistant Remember? Analyzing Conversational Context Recall and Utilization in Voice Interaction Models
Heeseung Kim, Che Hyun Lee, Sangkwon Park +4
Recent advancements in multi-turn voice interaction models have improved user-model communication. However, while closed-source models effectively retain and recall past utterances…
Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generator
Chaehun Shin, Jooyoung Choi, Heeseung Kim +1
Subject-driven text-to-image generation aims to produce images of a new subject within a desired context by accurately capturing both the visual characteristics of the subject and…
Style-Friendly SNR Sampler for Style-Driven Generation
Jooyoung Choi, Chaehun Shin, Yeongtak Oh +3
Recent text-to-image diffusion models generate high-quality images but struggle to learn new, personalized styles, which limits the creation of unique style templates. In style-dri…
NanoVoice: Efficient Speaker-Adaptive Text-to-Speech for Multiple Speakers
Nohil Park, Heeseung Kim, Che Hyun Lee +3
We present NanoVoice, a personalized text-to-speech model that efficiently constructs voice adapters for multiple speakers simultaneously. NanoVoice introduces a batch-wise speaker…
VoiceGuider: Enhancing Out-of-Domain Performance in Parameter-Efficient Speaker-Adaptive Text-to-Speech via Autoguidance
Jiheum Yeom, Heeseung Kim, Jooyoung Choi +3
When applying parameter-efficient finetuning via LoRA onto speaker adaptive text-to-speech models, adaptation performance may decline compared to full-finetuned counterparts, espec…
VoiceTailor: Lightweight Plug-In Adapter for Diffusion-Based Personalized Text-to-Speech
Heeseung Kim, Sang-gil Lee, Jiheum Yeom +3
We propose VoiceTailor, a parameter-efficient speaker-adaptive text-to-speech (TTS) system, by equipping a pre-trained diffusion-based TTS model with a personalized adapter. VoiceT…
HyperCLOVA X Technical Report
Kang Min Yoo, Jaegeun Han, Sookyo In +393
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. H…
Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversation
Heeseung Kim, Soonshin Seo, Kyeongseok Jeong +9
Recent work shows promising results in expanding the capabilities of large language models (LLM) to directly understand and synthesize speech. However, an LLM-based strategy for mo…
UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Data
Heeseung Kim, Sungwon Kim, Jiheum Yeom +1
We propose UnitSpeech, a speaker-adaptive speech synthesis method that fine-tunes a diffusion-based text-to-speech (TTS) model using minimal untranscribed data. To achieve this, we…
Edit-A-Video: Single Video Editing with Object-Aware Consistency
Chaehun Shin, Heeseung Kim, Che Hyun Lee +2
Despite the fact that text-to-video (TTV) model has recently achieved remarkable success, there have been few approaches on TTV for its extension to video editing. Motivated by app…
Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Data
Sungwon Kim, Heeseung Kim, Sungroh Yoon
We propose Guided-TTS 2, a diffusion-based generative model for high-quality adaptive TTS using untranscribed data. Guided-TTS 2 combines a speaker-conditional diffusion model with…