114 citations · 476 across the 43 of their papers we have counts for
43 papers
VoiceLDM: Text-to-Speech with Environmental Context
Yeonghyeon Lee, Inmo Yeon, Juhan Nam +1
This paper presents VoiceLDM, a model designed to produce audio that accurately follows two distinct natural language text prompts: the description prompt and the content prompt. T…
A Computational Evaluation Framework for Singable Lyric Translation
Haven Kim, Kento Watanabe, Masataka Goto +1
Lyric translation plays a pivotal role in amplifying the global resonance of music, bridging cultural divides, and fostering universal connections. Translating lyrics, unlike conve…
All-In-One Metrical And Functional Structure Analysis With Neighborhood Attentions on Demixed Audio
Taejun Kim, Juhan Nam
Music is characterized by complex hierarchical structures. Developing a comprehensive model to capture these structures has been a significant challenge in the field of Music Infor…
LP-MusicCaps: LLM-Based Pseudo Music Captioning
SeungHeon Doh, Keunwoo Choi, Jongpil Lee +1
Automatic music captioning, which generates natural language descriptions for given music tracks, holds significant potential for enhancing the understanding and organization of la…
PrimaDNN': A Characteristics-aware DNN Customization for Singing Technique Detection
Yuya Yamamoto, Juhan Nam, Hiroko Terasawa
Professional vocalists modulate their voice timbre or pitch to make their vocal performance more expressive. Such fluctuations are called singing techniques. Automatic detection of…
A study of audio mixing methods for piano transcription in violin-piano ensembles
Hyemi Kim, Jiyun Park, Taegyun Kwon +2
While piano music transcription models have shown high performance for solo piano recordings, their performance degrades when applied to ensemble recordings. This study aims to ana…
A Phoneme-Informed Neural Network Model for Note-Level Singing Transcription
Sangeon Yong, Li Su, Juhan Nam
Note-level automatic music transcription is one of the most representative music information retrieval (MIR) tasks and has been studied for various instruments to understand music.…
Textless Speech-to-Music Retrieval Using Emotion Similarity
SeungHeon Doh, Minz Won, Keunwoo Choi +1
We introduce a framework that recommends music based on the emotions of speech. In content creation and daily life, speech contains information about human emotions, which can be e…
Music Playlist Title Generation Using Artist Information
Haven Kim, SeungHeon Doh, Junwon Lee +1
Automatically generating or captioning music playlist titles given a set of tracks is of significant interest in music streaming services as customized playlists are widely used in…
Multimodal Deep Learning
Cem Akkus, Luyang Chu, Vladana Djakovic +14
This book is the result of a seminar in which we reviewed multimodal approaches and attempted to create a solid overview of the field, starting with the current state-of-the-art ap…
Neural Vocoder Feature Estimation for Dry Singing Voice Separation
Jaekwon Im, Soonbeom Choi, Sangeon Yong +1
Singing voice separation (SVS) is a task that separates singing voice audio from its mixture with instrumental audio. Previous SVS studies have mainly employed the spectrogram mask…
Toward Universal Text-to-Music Retrieval
SeungHeon Doh, Minz Won, Keunwoo Choi +1
This paper introduces effective design choices for text-to-music retrieval systems. An ideal text-based retrieval system would support various input queries such as pre-defined tag…
Analysis and Detection of Singing Techniques in Repertoires of J-POP Solo Singers
Yuya Yamamoto, Juhan Nam, Hiroko Terasawa
In this paper, we focus on singing techniques within the scope of music information retrieval research. We investigate how singers use singing techniques using real-world recording…
YM2413-MDB: A Multi-Instrumental FM Video Game Music Dataset with Emotion Annotations
Eunjin Choi, Yoonjin Chung, Seolhee Lee +3
Existing multi-instrumental datasets tend to be biased toward pop and classical music. In addition, they generally lack high-level annotations such as emotion tags. In this paper,…
Hi,KIA: A Speech Emotion Recognition Dataset for Wake-Up Words
Taesu Kim, SeungHeon Doh, Gyunpyo Lee +3
Wake-up words (WUW) is a short sentence used to activate a speech recognition system to receive the user's speech input. WUW utterances include not only the lexical information for…
Deformable CNN and Imbalance-Aware Feature Learning for Singing Technique Classification
Yuya Yamamoto, Juhan Nam, Hiroko Terasawa
Singing techniques are used for expressive vocal performances by employing temporal fluctuations of the timbre, the pitch, and other components of the voice. Their classification i…
Pseudo-Label Transfer from Frame-Level to Note-Level in a Teacher-Student Framework for Singing Transcription from Polyphonic Music
Sangeun Kum, Jongpil Lee, Keunhyoung Luke Kim +2
Lack of large-scale note-level labeled data is the major obstacle to singing transcription from polyphonic music. We address the issue by using pseudo labels from vocal pitch estim…
Music Playlist Title Generation: A Machine-Translation Approach
SeungHeon Doh, Junwon Lee, Juhan Nam
We propose a machine-translation approach to automatically generate a playlist title from a set of music tracks. We take a sequence of track IDs as input and a sequence of words in…
A Melody-Unsupervision Model for Singing Voice Synthesis
Soonbeom Choi, Juhan Nam
Recent studies in singing voice synthesis have achieved high-quality results leveraging advances in text-to-speech models based on deep neural networks. One of the main issues in t…
EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation
Hsiao-Tzu Hung, Joann Ching, Seungheon Doh +3
While there are many music datasets with emotion labels in the literature, they cannot be used for research on symbolic-domain music analysis or generation, as there are usually au…
PocketVAE: A Two-step Model for Groove Generation and Control
Kyungyun Lee, Wonil Kim, Juhan Nam
Creating a good drum track to imitate a skilled performer in digital audio workstations (DAWs) can be a time-consuming process, especially for those unfamiliar with drums. In this…
Polyphonic Piano Transcription Using Autoregressive Multi-State Note Model
Taegyun Kwon, Dasaem Jeong, Juhan Nam
Recent advances in polyphonic piano transcription have been made primarily by a deliberate design of neural network architectures that detect different note states such as onset or…
A Computational Analysis of Real-World DJ Mixes using Mix-To-Track Subsequence Alignment
Taejun Kim, Minsuk Choi, Evan Sacks +2
A DJ mix is a sequence of music tracks concatenated seamlessly, typically rendered for audiences in a live setting by a DJ on stage. As a DJ mix is produced in a studio or the live…
Semi-supervised learning using teacher-student models for vocal melody extraction
Sangeun Kum, Jing-Hua Lin, Li Su +1
The lack of labeled data is a major obstacle in many music information retrieval tasks such as melody extraction, where labeling is extremely laborious or costly. Semi-supervised l…