1 citations · 1 across the 22 of their papers we have counts for
22 papers
ProsoCodec: Prosody-Oriented Speech Codec for Voice Conversion
Jeongsoo Choi, Ji-Hoon Kim, Shujie Hu +1
Neural speech codecs efficiently compress speech and have become a foundation for speech generation, but they are typically learned as holistic representations that intertwine ling…
Probing Cross-modal Information Hubs in Audio-Visual LLMs
Jihoo Jung, Chaeyoung Jung, Ji-Hoon Kim +1
Audio-visual large language models (AVLLMs) have recently emerged as a powerful architecture capable of jointly reasoning over audio, visual, and textual modalities. In AVLLMs, the…
Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models
Kyudan Jung, Jihwan Kim, Soyoon Kim +3
As the paradigm of AI shifts from text-based LLMs to Speech Language Models (SLMs), there is a growing demand for full-duplex systems capable of real-time, natural human-computer i…
SNAP: Speaker Nulling for Artifact Projection in Speech Deepfake Detection
Kyudan Jung, Jihwan Kim, Minwoo Lee +4
Recent advancements in text-to-speech technologies enable generating high-fidelity synthetic speech nearly indistinguishable from real human voices. While recent studies show the e…
MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesis
Tan Dat Nguyen, Sangmin Bae, Joon Son Chung +1
Despite the remarkable quality of LLM-based text-to-speech systems, their reliance on autoregressive Transformers leads to quadratic computational complexity, which severely limits…
UNMIXX: Untangling Highly Correlated Singing Voices Mixtures
Jihoo Jung, Ji-Hoon Kim, Doyeop Kwak +3
We introduce UNMIXX, a novel framework for multiple singing voices separation (MSVS). While related to speech separation, MSVS faces unique challenges: data scarcity and the highly…
TAVID: Text-Driven Audio-Visual Interactive Dialogue Generation
Ji-Hoon Kim, Junseok Ahn, Doyeop Kwak +2
The objective of this paper is to jointly synthesize interactive videos and conversational speech from text and reference images. With the ultimate goal of building human-like conv…
SPADE: Structured Pruning and Adaptive Distillation for Efficient LLM-TTS
Tan Dat Nguyen, Jaehun Kim, Ji-Hoon Kim +3
The goal of this paper is to introduce SPADE, a framework for Structured Pruning and Adaptive Distillation for Efficient Large Language Model-based text-to-speech (LLM-TTS). Recent…
InfiniteAudio: Infinite-Length Audio Generation with Consistency
Chaeyoung Jung, Hojoon Ki, Ji-Hoon Kim +2
This paper presents InfiniteAudio, a simple yet effective strategy for generating infinite-length audio using diffusion-based text-to-audio methods. Current approaches face memory…
Accelerating Diffusion-based Text-to-Speech Model Training with Dual Modality Alignment
Jeongsoo Choi, Zhikang Niu, Ji-Hoon Kim +3
The goal of this paper is to optimize the training process of diffusion-based text-to-speech models. While recent studies have achieved remarkable advancements, their training dema…
AlignDiT: Multimodal Aligned Diffusion Transformer for Synchronized Speech Generation
Jeongsoo Choi, Ji-Hoon Kim, Kim Sung-Bin +2
In this paper, we address the task of multimodal-to-speech generation, which aims to synthesize high-quality speech from multiple input modalities: text, video, and reference audio…
From Faces to Voices: Learning Hierarchical Representations for High-quality Video-to-Speech
Ji-Hoon Kim, Jeongsoo Choi, Jaehun Kim +2
The objective of this study is to generate high-quality speech from silent talking face videos, a task also known as video-to-speech synthesis. A significant challenge in video-to-…
AdaptVC: High Quality Voice Conversion with Adaptive Learning
Jaehun Kim, Ji-Hoon Kim, Yeunju Choi +3
The goal of voice conversion is to transform the speech of a source speaker to sound like that of a reference speaker while preserving the original content. A key challenge is to e…
CrossSpeech++: Cross-lingual Speech Synthesis with Decoupled Language and Speaker Generation
Ji-Hoon Kim, Hong-Sun Yang, Yoon-Cheol Ju +3
The goal of this work is to generate natural speech in multiple languages while maintaining the same speaker identity, a task known as cross-lingual speech synthesis. A key challen…
V2SFlow: Video-to-Speech Generation with Speech Decomposition and Rectified Flow
Jeongsoo Choi, Ji-Hoon Kim, Jinyu Li +2
In this paper, we introduce V2SFlow, a novel Video-to-Speech (V2S) framework designed to generate natural and intelligible speech directly from silent talking face videos. While re…
Accelerating Codec-based Speech Synthesis with Multi-Token Prediction and Speculative Decoding
Tan Dat Nguyen, Ji-Hoon Kim, Jeongsoo Choi +4
The goal of this paper is to accelerate codec-based speech synthesis systems with minimum sacrifice to speech quality. We propose an enhanced inference method that allows for flexi…
SpoofCeleb: Speech Deepfake Detection and SASV In The Wild
Jee-weon Jung, Yihan Wu, Xin Wang +11
This paper introduces SpoofCeleb, a dataset designed for Speech Deepfake Detection (SDD) and Spoofing-robust Automatic Speaker Verification (SASV), utilizing source data from real-…
Text-To-Speech Synthesis In The Wild
Jee-weon Jung, Wangyou Zhang, Soumi Maiti +11
Traditional Text-to-Speech (TTS) systems rely on studio-quality speech recorded in controlled settings.a Recently, an effort known as noisy-TTS training has emerged, aiming to util…
VoxSim: A perceptual voice similarity dataset
Junseok Ahn, Youkyum Kim, Yeunju Choi +4
This paper introduces VoxSim, a dataset of perceptual voice similarity ratings. Recent efforts to automate the assessment of speech synthesis technologies have primarily focused on…
FlowAVSE: Efficient Audio-Visual Speech Enhancement with Conditional Flow Matching
Chaeyoung Jung, Suyeon Lee, Ji-Hoon Kim +1
This work proposes an efficient method to enhance the quality of corrupted speech signals by leveraging both acoustic and visual cues. While existing diffusion-based approaches hav…
Faces that Speak: Jointly Synthesising Talking Face and Speech from Text
Youngjoon Jang, Ji-Hoon Kim, Junseok Ahn +6
The goal of this work is to simultaneously generate natural talking faces and speech outputs from text. We achieve this by integrating Talking Face Generation (TFG) and Text-to-Spe…
FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocoder
Tan Dat Nguyen, Ji-Hoon Kim, Youngjoon Jang +2
The goal of this paper is to generate realistic audio with a lightweight and fast diffusion-based vocoder, named FreGrad. Our framework consists of the following three key componen…