Amy Zhang

Position: Beyond Training Time, Test-Time Coordination is Essential for Cooperative MARL

Dongsu Lee, Sooraj Sathish, Joonkyung Kim +2

We argue that cooperative multi-agent reinforcement learning (MARL) should treat test-time coordination as part of the problem definition rather than a deployment artifact. Our foc…

Test-time Multi-agent Coordination by Decomposed Value Gradient Flow

Dongsu Lee, Haoran Xu, Amy Zhang

Offline multi-agent reinforcement learning (MARL) faces a persistent trade-off. Expressive generative policies can represent multi-modal coordination in the data, but cannot distin…

Reinforcement Learning via Value Gradient Flow

Haoran Xu, Kaiwen Hu, Somayeh Sojoudi +1

We study behavior-regularized reinforcement learning (RL), where regularization toward a reference distribution (the dataset in offline RL or the base model in LLM RL finetuning) i…

The PokeAgent Challenge: Competitive and Long-Context Learning at Scale

Seth Karten, Jake Grigsby, Tersoo Upaa +28

We present the PokeAgent Challenge, a large-scale benchmark for decision-making research built on Pokemon's multi-agent battle system and expansive role-playing game (RPG) environm…

A Recipe for Stable Offline Multi-agent Reinforcement Learning

Dongsu Lee, Daehee Lee, Amy Zhang

Despite remarkable achievements in single-agent offline reinforcement learning (RL), multi-agent RL (MARL) has struggled to adopt this paradigm, largely persisting with on-policy t…

Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusion

Dan Haramati, Carl Qi, Tal Daniel +3

We propose a hierarchical entity-centric framework for offline Goal-Conditioned Reinforcement Learning (GCRL) that combines subgoal decomposition with factored structure to solve l…

Factored Latent Action World Models

Zizhao Wang, Chang Shi, Jiaheng Hu +4

Learning latent actions from action-free video has emerged as a powerful paradigm for scaling up controllable world model learning. Latent actions provide a natural interface for u…

Learning Robust Reasoning through Guided Adversarial Self-Play

Shuozhe Li, Vaishnav Tadiparthi, Kwonjoon Lee +6

Reinforcement learning from verifiable rewards (RLVR) produces strong reasoning models, yet they can fail catastrophically when the conditioning context is fallible (e.g., corrupte…

CARE-RFT: Confidence-Anchored Reinforcement Finetuning for Reliable Reasoning in Large Language Models

Shuozhe Li, Jincheng Cao, Bodun Hu +3

Reinforcement finetuning (RFT) has emerged as a powerful paradigm for unlocking reasoning capabilities in large language models. However, we identify a critical trade-off: while un…

Multi-agent Coordination via Flow Matching

Dongsu Lee, Daehee Lee, Amy Zhang

This work presents MAC-Flow, a simple yet expressive framework for multi-agent coordination. We argue that requirements of effective coordination are twofold: (i) a rich representa…

ExPO: Unlocking Hard Reasoning with Self-Explanation-Guided Reinforcement Learning

Ruiyang Zhou, Shuozhe Li, Amy Zhang +1

Self-improvement via RL often fails on complex reasoning tasks because GRPO-style post-training methods rely on the model's initial ability to generate positive samples. Without gu…

Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learning

Caleb Chuck, Fan Feng, Carl Qi +4

Hindsight relabeling is a powerful tool for overcoming sparsity in goal-conditioned reinforcement learning (GCRL), especially in certain domains such as navigation and locomotion.…

An Optimal Discriminator Weighted Imitation Perspective for Reinforcement Learning

Haoran Xu, Shuozhe Li, Harshit Sikchi +2

We introduce Iterative Dual Reinforcement Learning (IDRL), a new method that takes an optimal discriminator-weighted imitation view of solving RL. Our method is motivated by a simp…

Fast Adaptation with Behavioral Foundation Models

Harshit Sikchi, Andrea Tirinzoni, Ahmed Touati +6

Unsupervised zero-shot reinforcement learning (RL) has emerged as a powerful paradigm for pretraining behavioral foundation models (BFMs), enabling agents to solve a wide range of…

Offline Action-Free Learning of Ex-BMDPs by Comparing Diverse Datasets

Alexander Levine, Peter Stone, Amy Zhang

While sequential decision-making environments often involve high-dimensional observations, not all features of these observations are relevant for control. In particular, the obser…

Reevaluating Policy Gradient Methods for Imperfect-Information Games

Max Rudolph, Nathan Lichtle, Sobhan Mohammadpour +6

In the past decade, motivated by the putative failure of naive self-play deep reinforcement learning (DRL) in adversarial imperfect-information games, researchers have developed nu…

cs.LG2025★ 1 cited

Towards General-Purpose Model-Free Reinforcement Learning

Scott Fujimoto, Pierluca D'Oro, Amy Zhang +2

Reinforcement learning (RL) promises a framework for near-universal problem-solving. In practice however, RL algorithms are often tailored to specific benchmarks, relying on carefu…

AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers

Jake Grigsby, Justin Sasek, Samyak Parajuli +3

Language models trained on diverse datasets unlock generalization by in-context learning. Reinforcement Learning (RL) policies can achieve a similar effect by meta-learning within…

Proto Successor Measure: Representing the Behavior Space of an RL Agent

Siddhant Agarwal, Harshit Sikchi, Peter Stone +1

Having explored an environment, intelligent agents should be able to transfer their knowledge to most downstream tasks within that environment without additional interactions. Refe…

SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions

Zizhao Wang, Jiaheng Hu, Caleb Chuck +5

Unsupervised skill discovery carries the promise that an intelligent agent can learn reusable skills through autonomous, reward-free environment interaction. Existing unsupervised…

Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedback

Qinqing Zheng, Mikael Henaff, Amy Zhang +2

Automatically synthesizing dense rewards from natural language descriptions is a promising paradigm in reinforcement learning (RL), with applications to sparse reward problems, ope…

Learning a Fast Mixing Exogenous Block MDP using a Single Trajectory

Alexander Levine, Peter Stone, Amy Zhang

In order to train agents that can quickly adapt to new objectives or reward functions, efficient unsupervised representation learning in sequential decision-making environments can…

TRAM: Test-Time Risk Adaptation with Mixture of Agents

Mohamad Fares El Hajj Chehade, Amrit Singh Bedi, Amy Zhang +1

Deployed reinforcement learning agents often face safety requirements that are specified only after training, such as new hazard maps, revised risk thresholds, or behavioral alignm…