4 papers
Semore: VLM-guided Enhanced Semantic Motion Representations for Visual Reinforcement Learning
Wentao Wang, Chunyang Liu, Kehua Sheng +2
The growing exploration of Large Language Models (LLM) and Vision-Language Models (VLM) has opened avenues for enhancing the effectiveness of reinforcement learning (RL). However,…
Video-STR: Reinforcing MLLMs in Video Spatio-Temporal Reasoning with Relation Graph
Wentao Wang, Heqing Zou, Tianze Luo +8
Recent progress in Multimodal Large Language Models (MLLMs) has demonstrated strong semantic understanding capabilities, but struggles to perform precise spatio-temporal understand…
EgoTwin: Dreaming Body and View in First Person
Jingqiao Xiu, Fangzhou Hong, Yicong Li +5
While exocentric video synthesis has achieved great progress, egocentric video generation remains largely underexplored, which requires modeling first-person view content along wit…
FoAM: Foresight-Augmented Multi-Task Imitation Policy for Robotic Manipulation
Litao Liu, Wentao Wang, Yifan Han +5
Multi-task imitation learning (MTIL) has shown significant potential in robotic manipulation by enabling agents to perform various tasks using a single policy. This simplifies the…