15 citations · 37 across the 28 of their papers we have counts for
31 papers
Evidence-Based Policy Choice in Macroeconomics
Toru Kitagawa, Simone Martinalli
This paper studies data-driven macroeconomic policy choice in a structural VAR framework. We formulate the policymaker's problem of optimizing the future path of a policy instrumen…
FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experience
Zixun Huang, Kishan Panaganti, Haitao Mi +1
A reasoning model can improve from its own on-policy experience, but this inner loop is fragile: terminal verifiers provide reliable yet sparse supervision, while dense same-model…
Stale but Stable: Staleness-Adaptive Trust Regions for Stabilizing Asynchronous Reinforcement Learning
Junyao Yang, Yucheng Shi, Zongxia Li +6
Asynchronous reinforcement learning improves throughput by decoupling rollout generation from optimization, but the resulting staleness is an inevitable byproduct, compounded joint…
Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
Ruhan Wang, Yucheng Shi, Zongxia Li +7
The capability of a modern AI agent depends not only on its foundation model but also on its harness, which constructs prompts, manages state, invokes tools, and coordinates execut…
Reasoning or Memorization? Direction-Aware Diversity Exploration in LLM Reinforcement Learning
Jiangnan Xia, Yucheng Shi, Yu Yang +3
Reinforcement learning has become a key paradigm for eliciting reasoning abilities in large language models, where exploration is crucial for discovering effective solution traject…
Leveraging Machine-Learned Advice in Strategic Interactions with No-Regret Learners
Tinashe Handina, Tongxin Li, Kishan Panaganti +2
We study how an agent in a two-player repeated game can effectively utilize potentially imperfect advice when interacting with a no-regret learner. We characterize the advice lands…
Learning to Build the Environment: Self-Evolving Reasoning RL via Verifiable Environment Synthesis
Yucheng Shi, Zhenwen Liang, Kishan Panaganti +3
We pursue a vision for self-improving language models in which the model does not merely generate problems or traces to imitate, but constructs the environments that train it. In z…
The Single-Multi Evolution Loop for Self-Improving Model Collaboration Systems
Shangbin Feng, Kishan Panaganti, Yulia Tsvetkov +1
Model collaboration -- systems where multiple language models (LMs) collaborate -- combines the strengths of diverse models with cost in loading multiple LMs. We improve efficiency…
Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorization
Chengrui Qu, Christopher Yeh, Kishan Panaganti +2
Cooperative multi-agent reinforcement learning (MARL) commonly adopts centralized training with decentralized execution, where value-factorization methods enforce the individual-gl…
Group Distributionally Robust Optimization-Driven Reinforcement Learning for LLM Reasoning
Kishan Panaganti, Zhenwen Liang, Wenhao Yu +2
Recent progress in Large Language Model (LLM) reasoning is increasingly driven by the refinement of post-training loss functions and alignment strategies. However, standard Reinfor…
Can LLMs Guide Their Own Exploration? Gradient-Guided Reinforcement Learning for LLM Reasoning
Zhenwen Liang, Sidi Lu, Wenhao Yu +4
Reinforcement learning has become essential for strengthening the reasoning abilities of large language models, yet current exploration mechanisms remain fundamentally misaligned w…
Guided Self-Evolving LLMs with Minimal Human Supervision
Wenhao Yu, Zhenwen Liang, Chengsong Huang +4
AI self-evolution has long been envisioned as a path toward superintelligence, where models autonomously acquire, refine, and internalize knowledge from their own learning experien…
Every Question Has Its Own Value: Reinforcement Learning with Explicit Human Values
Dian Yu, Yulai Zhao, Kishan Panaganti +3
We propose Reinforcement Learning with Explicit Human Values (RLEV), a method that aligns Large Language Model (LLM) optimization directly with quantifiable human value signals. Wh…
Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation
Yujun Zhou, Zhenwen Liang, Haolin Liu +7
Large language models (LLMs) can improve through reinforcement learning with verifiable rewards (RLVR), but many settings lack reference answers or external verifiers. Label-free s…
TARDIS STRIDE: A Spatio-Temporal Road Image Dataset and World Model for Autonomy
Héctor Carrión, Yutong Bai, Víctor A. Hernández Castro +6
World models aim to simulate environments and enable effective agent behavior. However, modeling real-world environments presents unique challenges as they dynamically change acros…
Risk-Averse Total-Reward Reinforcement Learning
Xihong Su, Jia Lin Hau, Gersi Doko +2
Risk-averse total-reward Markov Decision Processes (MDPs) offer a promising framework for modeling and solving undiscounted infinite-horizon objectives. Existing model-based algori…
Efficient Policy Optimization in Robust Constrained MDPs with Iteration Complexity Guarantees
Sourav Ganguly, Kishan Panaganti, Arnob Ghosh +1
Constrained decision-making is essential for designing safe policies in real-world control systems, yet simulated environments often fail to capture real-world adversities. We cons…
KL-regularization Itself is Differentially Private in Bandits and RLHF
Yizhou Zhang, Kishan Panaganti, Laixi Shi +2
Differential Privacy (DP) provides a rigorous framework for privacy, ensuring the outputs of data-driven algorithms remain statistically indistinguishable across datasets that diff…
Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
Zaiyan Xu, Sushil Vemuri, Kishan Panaganti +3
A major challenge in aligning large language models (LLMs) with human preferences is the issue of distribution shift. LLM alignment algorithms rely on static preference datasets, a…
Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Data
Chengrui Qu, Laixi Shi, Kishan Panaganti +2
Online Reinforcement learning (RL) typically requires high-stakes online interaction data to learn a policy for a target task. This prompts interest in leveraging historical data t…
Distributionally Robust Constrained Reinforcement Learning under Strong Duality
Zhengfei Zhang, Kishan Panaganti, Laixi Shi +3
We study the problem of Distributionally Robust Constrained RL (DRC-RL), where the goal is to maximize the expected reward subject to environmental distribution shifts and constrai…
Tractable Equilibrium Computation in Markov Games through Risk Aversion
Eric Mazumdar, Kishan Panaganti, Laixi Shi
A significant roadblock to the development of principled multi-agent reinforcement learning is the fact that desired solution concepts like Nash equilibria may be intractable to co…
Model-Free Robust -Divergence Reinforcement Learning Using Both Offline and Online Data
Kishan Panaganti, Adam Wierman, Eric Mazumdar
The robust -regularized Markov Decision Process (RRMDP) framework focuses on designing control policies that are robust against parameter uncertainties due to mismatches between…
Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coverage
Kishan Panaganti, Zaiyan Xu, Dileep Kalathil +1
The goal of an offline reinforcement learning (RL) algorithm is to learn optimal polices using historical (offline) data, without access to the environment for online exploration.…