5 papers
Local Multimodal Music Alignment from Global Supervision
Irmak Bukey, Zachary Novack, Jongmin Jung +2
Understanding music requires understanding localized relationships across data modalities, e.g., how time in performance audio maps onto position in a score image. Yet supervision…
Nested Music Transformer: Sequentially Decoding Compound Tokens in Symbolic Music and Audio Generation
HaeJun Yoo, Hao-Wen Dong, Jongmin Jung +1
Representing symbolic music with compound tokens, where each token consists of several different sub-tokens representing a distinct musical feature or attribute, offers the advanta…
From Generation to Attribution: Music AI Agent Architectures for the Post-Streaming Era
Wonil Kim, Hyeongseok Wi, Seungsoon Park +12
Generative AI is reshaping music creation, but its rapid growth exposes structural gaps in attribution, rights management, and economic models. Unlike past media shifts, from live…
Unified Cross-modal Translation of Score Images, Symbolic Music, and Performance Audio
Jongmin Jung, Dongmin Kim, Sihun Lee +5
Music exists in various modalities, such as score images, symbolic scores, MIDI, and audio. Translations between each modality are established as core tasks of music information re…
LAV: Audio-Driven Dynamic Visual Generation with Neural Compression and StyleGAN2
Jongmin Jung, Dasaem Jeong
This paper introduces LAV (Latent Audio-Visual), a system that integrates EnCodec's neural audio compression with StyleGAN2's generative capabilities to produce visually dynamic ou…