11 papers
StateSkill-Agent: An Effective and Efficient Embodied Agent via State-conditioned Skill
Changxu Wei, Jingyi Zhang, Zhiyu Zheng +6
In this work, we aim to develop an embodied MLLM agent that understands and solves embodied tasks by learning, abstracting, and reusing transferrable and generalizable behavioral p…
SkillRubric: Co-Evolving Actor Guidance and Evaluator Rubrics for Multimodal Agents
Bingqing Jiang, Guoxi Zhang, Jasper Wang +7
Recent work incorporates reusable skills distilled from past interactions into multimodal agent training, providing procedural guidance for long-horizon planning and tool use. Howe…
OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Models
Tianyi Lin, Chuanyu Sun, Jingyi Zhang +6
Equipping Large Language Model (LLM) agents with effective skills is crucial for solving complex tasks in real-world systems like OpenClaw. In this work, we aim to develop a framew…
Efficient Exploration for Iterative Nash Preference Optimization
Tianlong Nan, Xiaopeng Li, Christian Kroer +1
Preference alignment is central to improving large language models (LLMs), but reward-based formulations can be restrictive when human preferences are non-transitive. Nash learning…
DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization
Jian Mu, Tianyi Lin, Chengwei Qin +2
Large language models are increasingly deployed in multi-turn interactive settings where users or environments can iteratively provide lightweight feedback. Unfortunately, optimizi…
How AI Aggregation Affects Knowledge
Daron Acemoglu, Tianyi Lin, Asuman Ozdaglar +1
Artificial intelligence (AI) changes social learning when aggregated outputs become training data for future predictions. To study this, we extend the DeGroot model by introducing…
R1-SyntheticVL: Is Synthetic Data from Generative Models Ready for Multimodal Large Language Model?
Jingyi Zhang, Tianyi Lin, Huanjin Yao +3
In this work, we aim to develop effective data synthesis techniques that autonomously synthesize multimodal training data for enhancing MLLMs in solving complex real-world tasks. T…
Reward-free Alignment for Conflicting Objectives
Peter Chen, Xiaopeng Li, Xi Chen +1
Direct alignment methods are increasingly used to align large language models (LLMs) with human preferences. However, many real-world alignment problems involve multiple conflictin…
Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward
Peter Chen, Xiaopeng Li, Ziniu Li +3
This paper examines the exploration-exploitation trade-off in reinforcement learning with verifiable rewards (RLVR), a framework for improving the reasoning of Large Language Model…
Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO
Peter Chen, Xiaopeng Li, Ziniu Li +2
Reinforcement learning (RL) has proven effective in strengthening the reasoning capabilities of large language models (LLMs). A widely adopted method, Group Relative Policy Optimiz…
ComPO: Preference Alignment via Comparison Oracles
Peter Chen, Xi Chen, Wotao Yin +1
Direct alignment methods are increasingly used for aligning large language models (LLMs) with human preferences. However, these methods suffer from the issues of verbosity and like…