10 citations · 20 across the 8 of their papers we have counts for
10 papers
OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understanding
Xuyang Liu, Shihao Wang, Xinyu Wei +5
With the rapid development of multimodal large language models (MLLMs), evaluating their video understanding capabilities has become increasingly important. Existing video understa…
PIVOT: Pivot-Aware On Policy Self Distillation for Multi-Turn VLM Agents
Jiazhou Zhou, Hu Zhou, Yucheng Chen +3
Reinforcement learning with verifiable rewards (RLVR) via Group-Relative Policy Optimization (GRPO) is widely used for multi-turn VLM agent training, yet it suffers from zero-gradi…
DRIFT: Difficulty Routing Self-DIstillation with Rhythm-Gated Exploration and Success BuFfer Training
Haisen Luo, Yiwei Liu, Haoning Wang +13
Enabling large language models to achieve stable self-improvement without external expert supervision remains a central challenge in complex reasoning tasks. Existing self-distilla…
ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas
Xiaoyu Tian, Haotian Wang, Shuaiting Chen +12
Large language models (LLMs) are increasingly used as tool-augmented agents for multi-step decision making, yet training robust tool-using agents remains challenging. Existing meth…
SIMA 2: A Generalist Embodied Agent for Virtual Worlds
SIMA team, Adrian Bolton, Alexander Lerchner +63
We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a signifi…
Adaptive Tool Generation with Models as Tools and Reinforcement Learning
Chenpeng Wang, Xiaojie Cheng, Chunye Wang +2
Tool-augmented language models have demonstrated strong capabilities, but their reliance on live API access creates scalability and reliability challenges during training and deplo…
VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding
Shihao Wang, Guo Chen, De-an Huang +6
While Video Large Language Models (Video-LLMs) have shown significant potential in multimodal understanding and reasoning tasks, how to efficiently select the most informative fram…
Training Language Models to Self-Correct via Reinforcement Learning
Aviral Kumar, Vincent Zhuang, Rishabh Agarwal +15
Self-correction is a highly desirable capability of large language models (LLMs), yet it has consistently been found to be largely ineffective in modern LLMs. Current methods for t…
Many-Shot In-Context Learning
Rishabh Agarwal, Avi Singh, Lei M. Zhang +12
Large language models (LLMs) excel at few-shot in-context learning (ICL) -- learning from a few examples provided in context at inference, without any weight updates. Newly expande…
Vision-Language Models as a Source of Rewards
Kate Baumli, Satinder Baveja, Feryal Behbahani +24
Building generalist agents that can accomplish many goals in rich open-ended environments is one of the research frontiers for reinforcement learning. A key limiting factor for bui…