15 papers
Learning to Steer LLM Agents and Multi-Agent Systems
Leyang Shen, Yaxin Gong, Yang Zhang +7
Large language model (LLM)-based agents and multi-agent systems increasingly tackle complex tasks through long-horizon tool use and inter-agent collaboration. However, these system…
Benchmarking Behavioral Steerability in Behavior Foundation Models
Minghe Gao, Zhanxi Yan, Jiahui Liu +9
Behavior Foundation Models (BFMs) are emerging as a paradigm for translating human intentions into executable humanoid behaviors. As these models evolve beyond behavior generation…
CARE: Competence-Aware Reward Shaping for Adaptive Reasoning Length in Video-MLLMs
Chengwen Liu, Hao Peng, Jisheng Dang +3
In multimodal video reasoning, reinforcement learning-based methods typically rely on simplistic and inflexible reasoning-length control strategies that fail to adapt to the model'…
Enhancing Decision-Making with Large Language Models through Multi-Agent Fictitious Play
Leyang Shen, Yang Zhang, Xiaoyan Zhao +2
Large language model (LLM)-based multi-agent systems (MAS) have demonstrated great potential in solving tasks with execution complexity, by distributing subtasks across cooperative…
Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs
Chengwen Liu, Zhe Huang, Jisheng Dang +3
Reinforcement learning has improved the reasoning ability of large language models, but applying outcome-only rewards to video multimodal large language models (Video-MLLMs) provid…
DanceOPD: On-Policy Generative Field Distillation
Wei Zhou, Xiongwei Zhu, Zelin Xu +8
Modern image generation demands a single model that unifies diverse capabilities, including text-to-image (T2I), local editing, and global editing. However, these capabilities are…
DeepTaxon: An Interpretable Retrieval-Augmented Multimodal Framework for Unified Species Identification and Discovery
Jiawei Wang, Ming Lei, Yaning Yang +8
Identifying species in biology among tens of thousands of visually similar taxa while discovering unknown species in open-world environments remains a fundamental challenge in biod…
Modeling Cross-vision Synergy for Unified Large Vision Model
Shengqiong Wu, Lanhu Wu, Mingyang Bao +5
Recent advances in large vision models (LVMs) have shifted from modality-specific designs toward unified architectures that jointly process images, videos, and 3D data. However, ex…
Can Large Language Models Derive New Knowledge? A Dynamic Benchmark for Biological Knowledge Discovery
Chaoqun Yang, Xinyu Lin, Shulin Li +4
Recent advancements in Large Language Model (LLM) agents have demonstrated remarkable potential in automatic knowledge discovery. However, rigorously evaluating an AI's capacity fo…
CARL: Criticality-Aware Agentic Reinforcement Learning
Leyang Shen, Yang Zhang, Chun Kai Ling +2
Agents capable of accomplishing complex tasks through multiple interactions with the environment have emerged as a popular research direction. However, in such multi-step settings,…
WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation
Wei Chow, Jiachun Pan, Yongyuan Liang +10
Recent advances in unified multimodal models (UMMs) have enabled impressive progress in visual comprehension and generation. However, existing datasets and benchmarks focus primari…
Training-Free Multimodal Large Language Model Orchestration
Tianyu Xie, Yuexiao Ma, Yuhang Wu +5
Building interactive omni-modal assistants often relies on end-to-end multimodal alignment to fuse heterogeneous modalities, which incurs substantial data and compute costs and lim…
On Path to Multimodal Generalist: General-Level and General-Bench
Hao Fei, Yuan Zhou, Juncheng Li +29
The Multimodal Large Language Model (MLLM) is currently experiencing rapid growth, driven by the advanced capabilities of LLMs. Unlike earlier specialists, existing MLLMs are evolv…
Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program
Minghe Gao, Xuqi Liu, Zhongqi Yue +7
Recent advancements in reward signal usage for Large Language Models (LLMs) are remarkable. However, significant challenges exist when transitioning reward signal to the multimodal…
Boosting Virtual Agent Learning and Reasoning: A Step-Wise, Multi-Dimensional, and Generalist Reward Model with Benchmark
Bingchen Miao, Yang Wu, Minghe Gao +7
The development of Generalist Virtual Agents (GVAs) has shown significant promise in autonomous task execution. However, current training paradigms face critical limitations, inclu…