4 papers
Learning to Hear by Seeing: It's Time for Vision Language Models to Understand Artistic Emotion from Sight and Sound
Dengming Zhang, Weitao You, Jingxiong Li +6
Emotion understanding is critical for making Large Language Models (LLMs) more general, reliable, and aligned with humans. Art conveys emotion through the joint design of visual an…
Controllable Video-to-Music Generation with Multiple Time-Varying Conditions
Junxian Wu, Weitao You, Heda Zuo +3
Music enhances video narratives and emotions, driving demand for automatic video-to-music (V2M) generation. However, existing V2M methods relying solely on visual features or suppl…
GVMGen: A General Video-to-Music Generation Model with Hierarchical Attentions
Heda Zuo, Weitao You, Junxian Wu +5
Composing music for video is essential yet challenging, leading to a growing interest in automating music generation for video applications. Existing approaches often struggle to a…
Personalized Dynamic Music Emotion Recognition with Dual-Scale Attention-Based Meta-Learning
Dengming Zhang, Weitao You, Ziheng Liu +2
Dynamic Music Emotion Recognition (DMER) aims to predict the emotion of different moments in music, playing a crucial role in music information retrieval. The existing DMER methods…