8 citations · 18 across the 25 of their papers we have counts for
32 papers
High-Fidelity Song Generation with Joint Discrete and Continuous Modeling
Zheqi Dai, Zhiyue Wu, Ruibin Yuan +5
Generating a full song from lyrics and a style description requires long-range musical planning without sacrificing vocal and instrumental detail. Low-rate discrete semantic tokens…
HMU-Bench: Hierarchical Music Understanding Benchmark for Omni Models
Zheqi Dai, Boyang Wang, Zhiyue Wu +6
Music-understanding benchmarks should identify both the evidence available to a model and the operations a correct answer requires. We introduce HMU-Bench, a bilingual benchmark wi…
SheetSage2: Coherent Lead-Sheet Transcription with Synthetic Supervision
Junyan Jiang, Ruibin Yuan, Jiahao Pan +4
Transcribing music into a human-readable score requires a coherent understanding of rhythm, harmony, melody, and form. Two obstacles limit this goal: annotated recordings are scarc…
HelixWorld: A Real-time Interactive Audio-Visual World Model
Lei Ke, Jiahao Pan, Zeyue Tian +13
World simulation is inherently multisensory, demanding synchronized visual and acoustic dynamics in real time. Yet prevailing interactive world models remain strictly silent, focus…
YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality
Ruibin Yuan, Jiahao Pan, Junyan Jiang +32
Symbolic models make melody, harmony, rhythm, and form explicit but typically stop before a finished recording; audio models produce complete songs while leaving composition implic…
StepAudio 3 Music Technical Report
Chengli Feng, Zhiyue Wu, Jiahao Song +10
We introduce StepAudio 3 Music, a large-scale, long-form music generation model that supports explicit musical planning and open-domain text-controlled generation. The StepAudio Mu…
Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence
Zhiqin Yang, Jingwen Fu, Yuhan Liu +16
Recent advances in large reasoning models (LRMs) have shown that reinforcement learning with verifiable rewards (RLVR) can substantially improve reasoning in mathematics and code,…
UniVerse: Benchmarking and Enhancing LALMs on Culturally Inclusive Low-Resource Music Understanding
Ziya Zhou, Shangda Wu, Shenyang Xu +16
Recent advances in large audio-language models (LALMs) have significantly improved performance in tasks such as music captioning, genre classification, and sound event detection. H…
Qwen-Music Technical Report
Jin Xu, Kangdi Wang, Ruibin Yuan +24
We introduce Qwen-Music, a music generation model that produces high-fidelity songs with complete vocals. It supports text-to-music generation from descriptions, lyrics, and musica…
Hierarchical Denoising For Multi-Step Visual Reasoning
Zezhong Qian, Xiaowei Chi, Chak-Wing Mak +9
Video models are evolving into vision foundation models, yet they still lack human-like multi-step reasoning. Streaming autoregressive diffusion models are efficient but limited in…
ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships
Xinyu Liu, Shihao Li, Weihong Lin +10
Recent diffusion-based video generation models have made significant progress in multi-reference image-conditioned video editing. However, existing methods still struggle to coordi…
AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation
Zeyue Tian, Lei Ke, Zhaoyang Liu +8
Audio and music generation based on flexible multimodal control signals is a widely applicable topic, with the following key challenges: 1) a unified multimodal modeling framework,…
Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing
Zeyue Tian, Binxin Yang, Zhaoyang Liu +8
Recent progress in multimodal models has spurred rapid advances in audio understanding, generation, and editing. However, these capabilities are typically addressed by specialized…
CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction
Yinghao Ma, Haiwen Xia, Hewei Gao +9
While music generation models have evolved to handle complex multimodal inputs mixing text, lyrics, and reference audio, evaluation mechanisms have lagged behind. In this paper, we…
Voices of Civilizations: A Multilingual QA Benchmark for Global Music Understanding
Shangda Wu, Ziya Zhou, Yongyi Zang +4
We introduce Voices of Civilizations, the first multilingual QA benchmark for evaluating audio LLMs' cultural comprehension on full-length music recordings. Covering 380 tracks acr…
SongFormer: Scaling Music Structure Analysis with Heterogeneous Supervision
Chunbo Hao, Ruibin Yuan, Jixun Yao +5
Music structure analysis (MSA) underpins music understanding and controllable generation, yet progress has been limited by small, inconsistent corpora. We present SongFormer, a sca…
UniSS: Unified Expressive Speech-to-Speech Translation with Your Voice
Sitong Cheng, Weizhen Bian, Xinsheng Wang +5
The ultimate goal of expressive speech-to-speech translation (S2ST) is to accurately translate spoken content while preserving the speaker identity and emotional style. However, pr…
WenetSpeech-Yue: A Large-scale Cantonese Speech Corpus with Multi-dimensional Annotation
Longhao Li, Zhao Guo, Hongjie Chen +15
The development of speech understanding and generation has been significantly accelerated by the availability of large-scale, high-quality speech datasets. Among these, ASR and TTS…
MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix
Ziyang Ma, Yinghao Ma, Yanqiao Zhu +31
We introduce MMAR, a new benchmark designed to evaluate the deep reasoning capabilities of Audio-Language Models (ALMs) across massive multi-disciplinary tasks. MMAR comprises 1,00…
SongEval: A Benchmark Dataset for Song Aesthetics Evaluation
Jixun Yao, Guobin Ma, Huixin Xue +9
Aesthetics serve as an implicit and important criterion in song generation tasks that reflect human perception beyond objective metrics. However, evaluating the aesthetics of gener…
Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens
Xinsheng Wang, Mingqi Jiang, Ziyang Ma +22
Recent advancements in large language models (LLMs) have driven significant progress in zero-shot text-to-speech (TTS) synthesis. However, existing foundation models rely on multi-…
AudioX: A Unified Framework for Anything-to-Audio Generation
Zeyue Tian, Zhaoyang Liu, Yizhu Jin +6
Audio and music generation based on flexible multimodal control signals is a widely applicable topic, with the following key challenges: 1) a unified multimodal modeling framework,…
YuE: Scaling Open Foundation Models for Long-Form Music Generation
Ruibin Yuan, Hanfeng Lin, Shuyue Guo +55
We tackle the task of long-form music generation--particularly the challenging \textbf{lyrics-to-song} problem--by introducing YuE, a family of open foundation models based on the…
Audio-FLAN: An Instruction-Following Dataset for Unified Audio Understanding and Generation of Speech, Music, and Sound
Liumeng Xue, Ziya Zhou, Jiahao Pan +20
Recent advancements in audio tokenization have significantly enhanced the integration of audio capabilities into large language models (LLMs). However, audio understanding and gene…