8 citations · 32 across the 20 of their papers we have counts for
22 papers
YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality
Ruibin Yuan, Jiahao Pan, Junyan Jiang +32
Symbolic models make melody, harmony, rhythm, and form explicit but typically stop before a finished recording; audio models produce complete songs while leaving composition implic…
MUUNRiver-Bench: Diagnosing Relation-Dependent Music Retrieval with Multimodal Instructions
Zhancheng Guo, Congren Dai, Shangda Wu +4
Music retrieval is relation-dependent: given a reference track, a listener may seek its style with a new theme, a cover, or a comparable voice, and these intents demand contradicto…
TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data
Jianhuai Hu, Yashan Wang, Shangda Wu +7
Generalizable Audio-to-Score (A2S) transcription is fundamentally constrained by the severe scarcity of high-quality, real-world paired data. Relying solely on existing human-annot…
UniVerse: Benchmarking and Enhancing LALMs on Culturally Inclusive Low-Resource Music Understanding
Ziya Zhou, Shangda Wu, Shenyang Xu +16
Recent advances in large audio-language models (LALMs) have significantly improved performance in tasks such as music captioning, genre classification, and sound event detection. H…
Voices of Civilizations: A Multilingual QA Benchmark for Global Music Understanding
Shangda Wu, Ziya Zhou, Yongyi Zang +4
We introduce Voices of Civilizations, the first multilingual QA benchmark for evaluating audio LLMs' cultural comprehension on full-length music recordings. Covering 380 tracks acr…
NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms
Yashan Wang, Shangda Wu, Jianhuai Hu +7
We introduce NotaGen, a symbolic music generation model aiming to explore the potential of producing high-quality classical sheet music. Inspired by the success of Large Language M…
YuE: Scaling Open Foundation Models for Long-Form Music Generation
Ruibin Yuan, Hanfeng Lin, Shuyue Guo +55
We tackle the task of long-form music generation--particularly the challenging \textbf{lyrics-to-song} problem--by introducing YuE, a family of open foundation models based on the…
CLaMP 3: Universal Music Information Retrieval Across Unaligned Modalities and Unseen Languages
Shangda Wu, Zhancheng Guo, Ruibin Yuan +7
CLaMP 3 is a unified framework developed to address challenges of cross-modal and cross-lingual generalization in music information retrieval. Using contrastive learning, it aligns…
Exploring Tokenization Methods for Multitrack Sheet Music Generation
Yashan Wang, Shangda Wu, Xingjian Du +1
This study explores the tokenization of multitrack sheet music in ABC notation, introducing two methods--bar-stream and line-stream patching. We compare these methods against exist…
CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models
Shangda Wu, Yashan Wang, Ruibin Yuan +12
Challenges in managing linguistic diversity and integrating various musical modalities are faced by current music information retrieval systems. These limitations reduce their effe…
Foundation Models for Music: A Survey
Yinghao Ma, Anders Øland, Anton Ragni +39
In recent years, foundation models (FMs) such as large language models (LLMs) and latent diffusion models (LDMs) have profoundly impacted diverse sectors, including music. This com…
MelodyT5: A Unified Score-to-Score Transformer for Symbolic Music Processing
Shangda Wu, Yashan Wang, Xiaobing Li +2
In the domain of symbolic music research, the progress of developing scalable systems has been notably hindered by the scarcity of available training data and the demand for models…
MuPT: A Generative Symbolic Music Pretrained Transformer
Xingwei Qu, Yuelin Bai, Yinghao Ma +25
In this paper, we explore the application of Large Language Models (LLMs) to the pre-training of music. While the prevalent use of MIDI in music modeling is well-established, our f…
Beyond Language Models: Byte Models are Digital World Simulators
Shangda Wu, Xu Tan, Zili Wang +3
Traditional deep learning often overlooks bytes, the basic units of the digital world, where all forms of information and operations are encoded and manipulated in binary format. I…
ChatMusician: Understanding and Generating Music Intrinsically with LLM
Ruibin Yuan, Hanfeng Lin, Yi Wang +32
While Large Language Models (LLMs) demonstrate impressive capabilities in text generation, we find that their ability has yet to be generalized to music, humanity's creative langua…
A Holistic Evaluation of Piano Sound Quality
Monan Zhou, Shangda Wu, Shaohua Ji +2
This paper aims to develop a holistic evaluation method for piano sound quality to assist in purchasing decisions. Unlike previous studies that focused on the effect of piano perfo…
EMelodyGen: Emotion-Conditioned Melody Generation in ABC Notation with the Musical Feature Template
Monan Zhou, Xiaobing Li, Feng Yu +1
The EMelodyGen system focuses on emotional melody generation in ABC notation controlled by the musical feature template. Owing to the scarcity of well-structured and emotionally la…
CLaMP: Contrastive Language-Music Pre-training for Cross-Modal Symbolic Music Information Retrieval
Shangda Wu, Dingyao Yu, Xu Tan +1
We introduce CLaMP: Contrastive Language-Music Pre-training, which learns cross-modal representations between natural language and symbolic music using a music encoder and a text e…
TunesFormer: Forming Irish Tunes with Control Codes by Bar Patching
Shangda Wu, Xiaobing Li, Feng Yu +1
This paper introduces TunesFormer, an efficient Transformer-based dual-decoder model specifically designed for the generation of melodies that adhere to user-defined musical forms.…
Exploring the Efficacy of Pre-trained Checkpoints in Text-to-Music Generation Task
Shangda Wu, Maosong Sun
Benefiting from large-scale datasets and pre-trained models, the field of generative models has recently gained significant momentum. However, most datasets for symbolic music are…
Efficient and Training-Free Control of Language Generation
Shangda Wu, Maosong Sun
In recent years, there has been a growing interest in the development of language models capable of generating text with controllable attributes. While several approaches have been…
Generating Chord Progression from Melody with Flexible Harmonic Rhythm and Controllable Harmonic Density
Shangda Wu, Yue Yang, Zhaowen Wang +2
Melody harmonization, which involves generating a chord progression that complements a user-provided melody, continues to pose a significant challenge. A chord progression must not…