From the 1 of 6 linked papers with an AI index.
6 papers
AVCap: Reinforcing Audio-Video Joint Caption with Detail-Aware Reward
Mingyang Wu, Kaituo Feng, Bohao Li +3
Detailed audio-video joint captioning is essential for multimodal video understanding and generation. However, prior works are constrained by three main limitations: (1) the scarci…
Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models
Hao Wang, Hao Gu, Hongming Piao +6
The paper introduces CurioSFT, an entropy-preserving supervised fine-tuning approach that uses adaptive self-distillation to keep exploration abilities in large reasoning models, l…
Matrix-game 2.0: An open-source real-time and streaming interactive world model
Xianglong He, Chunli Peng, Zexiang Liu +16
Recent advances in interactive video generations have demonstrated diffusion model's potential as world models by capturing complex physical dynamics and interactive behaviors. How…
Video-R1: Reinforcing Video Reasoning in MLLMs
Kaituo Feng, Kaixiong Gong, Bohao Li +7
Inspired by DeepSeek-R1's success in eliciting reasoning abilities through rule-based reinforcement learning (RL), we introduce Video-R1 as the first attempt to systematically expl…
AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?
Kaixiong Gong, Kaituo Feng, Bohao Li +8
Recently, multimodal large language models (MLLMs), such as GPT-4o, Gemini 1.5 Pro, and Reka Core, have expanded their capabilities to include vision and audio modalities. While th…
BIFRÃST: 3D-Aware Image compositing with Language Instructions
Lingxiao Li, Kaixiong Gong, Weihong Li +4
This paper introduces Bifröst, a novel 3D-aware framework that is built upon diffusion models to perform instruction-based image composition. Previous methods concentrate on image…