13 citations · 24 across the 8 of their papers we have counts for
9 papers
Diff-V2M: A Hierarchical Conditional Diffusion Model with Explicit Rhythmic Modeling for Video-to-Music Generation
Shulei Ji, Zihao Wang, Jiaxing Yu +4
Video-to-music (V2M) generation aims to create music that aligns with visual content. However, two main challenges persist in existing methods: (1) the lack of explicit rhythm mode…
A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives
Shuyu Li, Shulei Ji, Zihao Wang +3
Multi-modal music generation, using multiple modalities like text, images, and video alongside musical scores and audio as guidance, is an emerging research area with broad applica…
A Comprehensive Survey on Generative AI for Video-to-Music Generation
Shulei Ji, Songruoyao Wu, Zihao Wang +2
The burgeoning growth of video-to-music generation can be attributed to the ascendancy of multimodal generative models. However, there is a lack of literature that comprehensively…
SongGLM: Lyric-to-Melody Generation with 2D Alignment Encoding and Multi-Task Pre-Training
Jiaxing Yu, Xinda Wu, Yunfei Xu +4
Lyric-to-melody generation aims to automatically create melodies based on given lyrics, requiring the capture of complex and subtle correlations between them. However, previous wor…
ArchiTone: A LEGO-Inspired Gamified System for Visualized Music Education
Jiaxing Yu, Tieyao Zhang, Songruoyao Wu +4
Participation in music activities has many benefits, but often requires music theory knowledge and aural skills, which can be challenging for beginners. To help them engage more ea…
SoundScape: A Human-AI Co-Creation System Making Your Memories Heard
Chongjun Zhong, Jiaxing Yu, Yingping Cao +3
Sound plays a significant role in human memory, yet it is often overlooked by mainstream life-recording methods. Most current UGC (User-Generated Content) creation tools emphasize…
WuYun: Exploring hierarchical skeleton-guided melody generation using knowledge-enhanced deep learning
Kejun Zhang, Xinda Wu, Tieyao Zhang +5
Although deep learning has revolutionized music generation, existing methods for structured melody generation follow an end-to-end left-to-right note-by-note generative paradigm an…
ReLyMe: Improving Lyric-to-Melody Generation by Incorporating Lyric-Melody Relationships
Chen Zhang, Luchin Chang, Songruoyao Wu +4
Lyric-to-melody generation, which generates melody according to given lyrics, is one of the most important automatic music composition tasks. With the rapid development of deep lea…
TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method
Zeqian Ju, Peiling Lu, Xu Tan +7
Lyric-to-melody generation is an important task in automatic songwriting. Previous lyric-to-melody generation systems usually adopt end-to-end models that directly generate melodie…