Pratap Tokekar

Stable and Efficient Single-Rollout RL for Multimodal Reasoning

Rui Liu, Dian Yu, Lei Ke +6

Reinforcement Learning with Verifiable Rewards (RLVR) has become a key paradigm to improve the reasoning capabilities of Multimodal Large Language Models (MLLMs). However, prevalen…

Dual-Uncertainty Guided Policy Learning for Multimodal Reasoning

Rui Liu, Dian Yu, Tong Zheng +8

Reinforcement learning with verifiable rewards (RLVR) has advanced reasoning capabilities in multimodal large language models. However, existing methods typically treat visual inpu…

AFFORD2ACT: How Far Can Sparse Keypoints Go for Robot Manipulation?

Anukriti Singh, Kasra Torshizi, Khuzema Habib +3

Vision-based manipulation policies are commonly learned from rich visual observations, typically using dense visual features extracted from full resolution images. It remains uncle…

Contextual Neural Moving Horizon Estimation for Robust Quadrotor Control in Varying Conditions

Kasra Torshizi, Chak Lam Shek, Khuzema Habib +3

Adaptive controllers on quadrotors typically rely on estimation of disturbances to ensure robust trajectory tracking. Estimating disturbances across diverse environmental contexts…

MMCD: Multi-Modal Collaborative Decision-Making for Connected Autonomy with Knowledge Distillation

Rui Liu, Zikang Wang, Peng Gao +3

Autonomous systems have advanced significantly, but challenges persist in accident-prone environments where robust decision-making is crucial. A single vehicle's limited sensor ran…

GenFlowRL: Shaping Rewards with Generative Object-Centric Flow in Visual Reinforcement Learning

Kelin Yu, Sheng Zhang, Harshit Soora +4

Recent advances have shown that video generation models can enhance robot learning by deriving effective robot actions through inverse dynamics. However, these methods heavily depe…

Multi-Agent Trust Region Policy Optimisation: A Joint Constraint Approach

Chak Lam Shek, Guangyao Shi, Pratap Tokekar

Multi-agent reinforcement learning (MARL) requires coordinated and stable policy updates among interacting agents. Heterogeneous-Agent Trust Region Policy Optimization (HATRPO) enf…

Learning Multi-Robot Coordination through Locality-Based Factorized Multi-Agent Actor-Critic Algorithm

Chak Lam Shek, Amrit Singh Bedi, Anjon Basak +5

In this work, we present a novel cooperative multi-agent reinforcement learning method called \textbf{Loc}ality based \textbf{Fac}torized \textbf{M}ulti-Agent \textbf{A}ctor-\textb…

Option Discovery Using LLM-guided Semantic Hierarchical Reinforcement Learning

Chak Lam Shek, Pratap Tokekar

Large Language Models (LLMs) have shown remarkable promise in reasoning and decision-making, yet their integration with Reinforcement Learning (RL) for complex robotic tasks remain…

PEnGUiN: Partially Equivariant Graph NeUral Networks for Sample Efficient MARL

Joshua McClellan, Greyson Brothers, Furong Huang +1

Equivariant Graph Neural Networks (EGNNs) have emerged as a promising approach in Multi-Agent Reinforcement Learning (MARL), leveraging symmetry guarantees to greatly improve sampl…

VARP: Reinforcement Learning from Vision-Language Model Feedback with Agent Regularized Preferences

Anukriti Singh, Amisha Bhaskar, Peihong Yu +4

Designing reward functions for continuous-control robotics often leads to subtle misalignments or reward hacking, especially in complex tasks. Preference-based RL mitigates some of…

Sketch-to-Skill: Bootstrapping Robot Learning with Human Drawn Trajectory Sketches

Peihong Yu, Amisha Bhaskar, Anukriti Singh +2

Training robotic manipulation policies traditionally requires numerous demonstrations and/or environmental rollouts. While recent Imitation Learning (IL) and Reinforcement Learning…

CAML: Collaborative Auxiliary Modality Learning for Multi-Agent Systems

Rui Liu, Yu Shen, Peng Gao +2

Multi-modal learning has emerged as a key technique for improving performance across domains such as autonomous driving, robotics, and reasoning. However, in certain scenarios, par…