151 citations · 780 across the 55 of their papers we have counts for
55 papers
Contextual Distributionally Robust Chance-Constrained Programs: Exact Reformulations and Valid Inequalities
Hanbin Yang, Nan Jiang, Guodong Lyu
We study a contextual distributionally robust chance-constrained programming (C-DRCCP) model that controls violation probabilities conditional on a neighborhood of a target context…
Q-MMR: Off-Policy Evaluation via Recursive Reweighting and Moment Matching
Xiang Li, Nan Jiang
We present a novel theoretical framework, Q-MMR, for off-policy evaluation in finite-horizon MDPs. Q-MMR learns a set of scalar weights, one for each data point, such that the rewe…
Tightening CVaR Approximations via Scenario-Wise Scaling for Chance-Constrained Programming
Rui Chen, Nan Jiang
Chance-constrained programs (CCPs) provide a powerful modeling framework for decision-making under uncertainty, but their nonconvex feasible regions make them computationally chall…
The Optimal Approximation Factors in Misspecified Off-Policy Value Function Estimation
Philip Amortila, Nan Jiang, Csaba Szepesvári
Theoretical guarantees in reinforcement learning (RL) are known to suffer multiplicative blow-up factors with respect to the misspecification error of function approximation. Yet,…
Explaining RL Decisions with Trajectories
Shripad Vilasrao Deshmukh, Arpan Dasgupta, Balaji Krishnamurthy +4
Explanation is a key component for the adoption of reinforcement learning (RL) in many real-world decision-making problems. In the literature, the explanation is often provided by…
Offline Learning in Markov Games with General Function Approximation
Yuheng Zhang, Yu Bai, Nan Jiang
We study offline multi-agent reinforcement learning (RL) in Markov games, where the goal is to learn an approximate equilibrium -- such as Nash equilibrium and (Coarse) Correlated…
Reinforcement Learning in Low-Rank MDPs with Density Features
Audrey Huang, Jinglin Chen, Nan Jiang
MDPs with low-rank transitions -- that is, the transition matrix can be factored into the product of two matrices, left and right -- is a highly representative structure that enabl…
ALSO-X#: Better Convex Approximations for Distributionally Robust Chance Constrained Programs
Nan Jiang, Weijun Xie
This paper studies distributionally robust chance constrained programs (DRCCPs), where the uncertain constraints must be satisfied with at least a probability of a prespecified thr…
Adversarial Model for Offline Reinforcement Learning
Mohak Bhardwaj, Tengyang Xie, Byron Boots +2
We propose a novel model-based offline Reinforcement Learning (RL) framework, called Adversarial Model for Offline Reinforcement Learning (ARMOR), which can robustly learn policies…
Learning Markov Random Fields for Combinatorial Structures via Sampling through Lovász Local Lemma
Nan Jiang, Yi Gu, Yexiang Xue
Learning to generate complex combinatorial structures satisfying constraints will have transformative impacts in many application domains. However, it is beyond the capabilities of…
ARMOR: A Model-based Framework for Improving Arbitrary Baseline Policies with Offline Data
Tengyang Xie, Mohak Bhardwaj, Nan Jiang +1
We propose a new model-based offline RL framework, called Adversarial Models for Offline Reinforcement Learning (ARMOR), which can robustly learn policies to improve upon an arbitr…
Beyond the Return: Off-policy Function Estimation under User-specified Error-measuring Distributions
Audrey Huang, Nan Jiang
Off-policy evaluation often refers to two related tasks: estimating the expected return of a policy and estimating its value function (or other functions of interest, such as densi…
The Role of Coverage in Online Reinforcement Learning
Tengyang Xie, Dylan J. Foster, Yu Bai +2
Coverage conditions -- which assert that the data logging distribution adequately covers the state space -- play a fundamental role in determining the sample complexity of offline…
A Few Expert Queries Suffices for Sample-Efficient RL with Resets and Linear Value Approximation
Philip Amortila, Nan Jiang, Dhruv Madeka +1
The current paper studies sample-efficient Reinforcement Learning (RL) in settings where only the optimal value function is assumed to be linearly-realizable. It has recently been…
Future-Dependent Value-Based Off-Policy Evaluation in POMDPs
Masatoshi Uehara, Haruka Kiyohara, Andrew Bennett +5
We study off-policy evaluation (OPE) for partially observable MDPs (POMDPs) with general function approximation. Existing methods such as sequential importance sampling estimators…
On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RL
Jinglin Chen, Aditya Modi, Akshay Krishnamurthy +2
We study reward-free reinforcement learning (RL) under general non-linear function approximation, and establish sample efficiency and hardness results under various standard struct…
Interaction-Grounded Learning with Action-inclusive Feedback
Tengyang Xie, Akanksha Saran, Dylan J. Foster +5
Consider the problem setting of Interaction-Grounded Learning (IGL), in which a learner's goal is to optimally interact with the environment with no explicit reward to ground its p…
Tiered Reinforcement Learning: Pessimism in the Face of Uncertainty and Constant Regret
Jiawei Huang, Li Zhao, Tao Qin +3
We propose a new learning framework that captures the tiered structure of many real-world user-interaction applications, where the users can be divided into two groups based on the…
Offline Reinforcement Learning Under Value and Density-Ratio Realizability: The Power of Gaps
Jinglin Chen, Nan Jiang
We consider a challenging theoretical problem in offline reinforcement learning (RL): obtaining sample-efficiency guarantees with a dataset lacking sufficient coverage, under only…
Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality
Jiawei Huang, Jinglin Chen, Li Zhao +3
Deployment efficiency is an important criterion for many real-world applications of reinforcement learning (RL). Despite the community's increasing interest, there lacks a formal t…
Offline Reinforcement Learning with Realizability and Single-policy Concentrability
Wenhao Zhan, Baihe Huang, Audrey Huang +2
Sample-efficiency guarantees for offline reinforcement learning (RL) often rely on strong assumptions on both the function classes (e.g., Bellman-completeness) and the data coverag…
Adversarially Trained Actor Critic for Offline Reinforcement Learning
Ching-An Cheng, Tengyang Xie, Nan Jiang +1
We propose Adversarially Trained Actor Critic (ATAC), a new model-free algorithm for offline reinforcement learning (RL) under insufficient data coverage, based on the concept of r…
Towards Hyperparameter-free Policy Selection for Offline Reinforcement Learning
Siyuan Zhang, Nan Jiang
How to select between policies and value functions produced by different training algorithms in offline reinforcement learning (RL) -- which is crucial for hyperpa-rameter tuning -…
A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes
Chengchun Shi, Masatoshi Uehara, Jiawei Huang +1
We consider off-policy evaluation (OPE) in Partially Observable Markov Decision Processes (POMDPs), where the evaluation policy depends only on observable variables and the behavio…