109 citations · 258 across the 25 of their papers we have counts for
26 papers
MixProLAP: Mixture-Induced Uncertainty Modeling for Probabilistic Language-Audio Pretraining
Yu Nakagome, Jaesong Lee, Soo-Whan Chung
Acoustic environments often contain multiple overlapping sound events, and the same acoustic scene can be described using diverse textual expressions, making audio-text alignment i…
Seeing What You Say: Expressive Image Generation from Speech
Jiyoung Lee, Song Park, Sanghyuk Chun +1
This paper proposes VoxStudio, the first unified and end-to-end speech-to-image model that generates expressive images directly from spoken descriptions by jointly aligning linguis…
Listen through the Sound: Generative Speech Restoration Leveraging Acoustic Context Representation
Soo-Whan Chung, Min-Seok Choi
This paper introduces a novel approach to speech restoration by integrating a context-related conditioning strategy. Specifically, we employ the diffusion-based generative restorat…
S3D: Sketch-Driven 3D Model Generation
Hail Song, Wonsik Shin, Naeun Lee +3
Generating high-quality 3D models from 2D sketches is a challenging task due to the inherent ambiguity and sparsity of sketch data. In this paper, we present S3D, a novel framework…
Speak in the Scene: Diffusion-based Acoustic Scene Transfer toward Immersive Speech Generation
Miseul Kim, Soo-Whan Chung, Youna Ji +2
This paper introduces a novel task in generative speech processing, Acoustic Scene Transfer (AST), which aims to transfer acoustic scenes of speech signals to diverse environments.…
HD-DEMUCS: General Speech Restoration with Heterogeneous Decoders
Doyeon Kim, Soo-Whan Chung, Hyewon Han +2
This paper introduces an end-to-end neural speech restoration model, HD-DEMUCS, demonstrating efficacy across multiple distortion environments. Unlike conventional approaches that…
MF-PAM: Accurate Pitch Estimation through Periodicity Analysis and Multi-level Feature Fusion
Woo-Jin Chung, Doyeon Kim, Soo-Whan Chung +1
We introduce Multi-level feature Fusion-based Periodicity Analysis Model (MF-PAM), a novel deep learning-based pitch estimation model that accurately estimates pitch trajectory in…
An empirical study on speech restoration guided by self supervised speech representation
Jaeuk Byun, Youna Ji, Soo Whan Chung +2
Enhancing speech quality is an indispensable yet difficult task as it is often complicated by a range of degradation factors. In addition to additive noise, reverberation, clipping…
MoLE : Mixture of Language Experts for Multi-Lingual Automatic Speech Recognition
Yoohwan Kwon, Soo-Whan Chung
Multi-lingual speech recognition aims to distinguish linguistic expressions in different languages and integrate acoustic processing simultaneously. In contrast, current multi-ling…
Imaginary Voice: Face-styled Diffusion Model for Text-to-Speech
Jiyoung Lee, Joon Son Chung, Soo-Whan Chung
The goal of this work is zero-shot text-to-speech synthesis, with speaking styles and voices learnt from facial characteristics. Inspired by the natural fact that people can imagin…
Diffusion-based Generative Speech Source Separation
Robin Scheibler, Youna Ji, Soo-Whan Chung +3
We propose DiffSep, a new single channel source separation method based on score-matching of a stochastic differential equation (SDE). We craft a tailored continuous time diffusion…
Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting
Hyeon-Kyeong Shin, Hyewon Han, Doyeon Kim +2
In this paper, we propose a novel end-to-end user-defined keyword spotting method that utilizes linguistically corresponding patterns between speech and text sequences. Unlike prev…
Baseline Systems for the First Spoofing-Aware Speaker Verification Challenge: Score and Embedding Fusion
Hye-jin Shim, Hemlata Tak, Xuechen Liu +12
Deep learning has brought impressive progress in the study of both automatic speaker verification (ASV) and spoofing countermeasures (CM). Although solutions are mutually dependent…
SASV 2022: The First Spoofing-Aware Speaker Verification Challenge
Jee-weon Jung, Hemlata Tak, Hye-jin Shim +6
The first spoofing-aware speaker verification (SASV) challenge aims to integrate research efforts in speaker verification and anti-spoofing. We extend the speaker verification scen…
SASV Challenge 2022: A Spoofing Aware Speaker Verification Challenge Evaluation Plan
Jee-weon Jung, Hemlata Tak, Hye-jin Shim +7
ASV (automatic speaker verification) systems are intrinsically required to reject both non-target (e.g., voice uttered by different speaker) and spoofed (e.g., synthesised or conve…
Phase Continuity: Learning Derivatives of Phase Spectrum for Speech Enhancement
Doyeon Kim, Hyewon Han, Hyeon-Kyeong Shin +2
Modern neural speech enhancement models usually include various forms of phase information in their training loss terms, either explicitly or implicitly. However, these loss terms…
Look Who's Talking: Active Speaker Detection in the Wild
You Jin Kim, Hee-Soo Heo, Soyeon Choe +5
In this work, we present a novel audio-visual dataset for active speaker detection in the wild. A speaker is considered active when his or her face is visible and the voice is audi…
Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separation
Jiyoung Lee, Soo-Whan Chung, Sunok Kim +2
In this paper, we address the problem of separating individual speech signals from videos using audio-visual neural processing. Most conventional approaches utilize frame-wise matc…
Learning in your voice: Non-parallel voice conversion based on speaker consistency loss
Yoohwan Kwon, Soo-Whan Chung, Hee-Soo Heo +1
In this paper, we propose a novel voice conversion strategy to resolve the mismatch between the training and conversion scenarios when parallel speech corpus is unavailable for tra…
MIRNet: Learning multiple identities representations in overlapped speech
Hyewon Han, Soo-Whan Chung, Hong-Goo Kang
Many approaches can derive information about a single speaker's identity from the speech by learning to recognize consistent characteristics of acoustic parameters. However, it is…
Intra-class variation reduction of speaker representation in disentanglement framework
Yoohwan Kwon, Soo-Whan Chung, Hong-Goo Kang
In this paper, we propose an effective training strategy to ex-tract robust speaker representations from a speech signal. Oneof the key challenges in speaker recognition tasks is t…
FaceFilter: Audio-visual speech separation using still images
Soo-Whan Chung, Soyeon Choe, Joon Son Chung +1
The objective of this paper is to separate a target speaker's speech from a mixture of two speakers using a deep audio-visual speech separation network. Unlike previous works that…
Seeing voices and hearing voices: learning discriminative embeddings using cross-modal self-supervision
Soo-Whan Chung, Hong Goo Kang, Joon Son Chung
The goal of this work is to train discriminative cross-modal embeddings without access to manually annotated data. Recent advances in self-supervised learning have shown that effec…
End-to-End Lip Synchronisation Based on Pattern Classification
You Jin Kim, Hee Soo Heo, Soo-Whan Chung +1
The goal of this work is to synchronise audio and video of a talking face using deep neural network models. Existing works have trained networks on proxy tasks such as cross-modal…