13 papers
PitchIn: Natural-Language Vocal Coaching from Unstructured Community Feedback
Hayeon Bang, Hounsu Kim, Joonyong Park +2
Effective vocal coaching connects performance issues with corrective advice, but recordings paired with expert diagnoses and advice are scarce. Online singing communities provide a…
VocalCoachBench: Benchmarking Audio-Language Models on Expert Feedback for Singing
Hayeon Bang, Hounsu Kim, Wonil Kim +1
Recent audio-language models are increasingly evaluated on recognizing, describing, and reasoning about audio, but expert-facing applications require a different capability: produc…
Multi-Task Multi-Frame Visual Piano Transcription
Yonghyun Kim, Hoyeol Sohn, Juhan Nam +1
Audio-based piano transcription performs well on onset, pitch, and velocity, but the sustain pedal lets sound persist long after key release, so audio systems predict pedal-extende…
SKY-Piano: A Multimodal Piano Performance Dataset
Joonhyung Bae, Dawon Park, Taegyun Kwon +10
Music information retrieval research on piano performance increasingly involves diverse modalities of data and annotations beyond audio and MIDI. We present SKY-Piano, a multimodal…
PiAnnotate: A Web Annotation Tool for Piano Fingering, with a Diagnostic Probe
Joonhyung Bae, Kirak Kim, Hyeyoon Cho +8
Piano fingering shapes how a passage can be played, yet it is difficult to label after a performance. An annotator must decide which finger produced each note while reconciling the…
Tipiano: Cascaded Piano Hand Motion Synthesis via Fingertip Priors
Joonhyung Bae, Kirak Kim, Hyeyoon Cho +9
Synthesizing realistic piano hand motions requires both precision and naturalness. Physics-based methods achieve precision but produce stiff motions; data-driven models learn natur…
D3PIA: A Discrete Denoising Diffusion Model for Piano Accompaniment Generation From Lead sheet
Eunjin Choi, Hounsu Kim, Hayeon Bang +2
Generating piano accompaniments in the symbolic music domain is a challenging task that requires producing a complete piece of piano music from given melody and chord constraints,…
Designing a Multimodal Viewer for Piano Performance Analysis -- a Pedagogy-First Approach
Joonhyung Bae, Hyeyoon Cho, Kirak Kim +11
Abstract instructions in piano education, such as "raise your wrist" and "relax your tension," lead to varying interpretations among learners, preventing instructors from effective…
Two Web Toolkits for Multimodal Piano Performance Dataset Acquisition and Fingering Annotation
Junhyung Park, Yonghyun Kim, Joonhyung Bae +4
Piano performance is a multimodal activity that intrinsically combines physical actions with the acoustic rendition. Despite growing research interest in analyzing the multimodal n…
PianoVAM: A Multimodal Piano Performance Dataset
Yonghyun Kim, Junhyung Park, Joonhyung Bae +4
The multimodal nature of music performance has driven increasing interest in data beyond the audio domain within the music information retrieval (MIR) community. This paper introdu…
Dialogue in Resonance: An Interactive Music Piece for Piano and Real-Time Automatic Transcription System
Hayeon Bang, Taegyun Kwon, Juhan Nam
This paper presents <Dialogue in Resonance>, an interactive music piece for a human pianist and a computer-controlled piano that integrates real-time automatic music transcription…
D3RM: A Discrete Denoising Diffusion Refinement Model for Piano Transcription
Hounsu Kim, Taegyun Kwon, Juhan Nam
Diffusion models have been widely used in the generative domain due to their convincing performance in modeling complex data distributions. Moreover, they have shown competitive re…
A Real-Time Lyrics Alignment System Using Chroma And Phonetic Features For Classical Vocal Performance
Jiyun Park, Sangeon Yong, Taegyun Kwon +1
The goal of real-time lyrics alignment is to take live singing audio as input and to pinpoint the exact position within given lyrics on the fly. The task can benefit real-world app…