36 citations · 136 across the 20 of their papers we have counts for
22 papers
Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
Dylan J. Foster, Adam Block, Dipendra Misra
Imitation learning (IL) aims to mimic the behavior of an expert in a sequential decision making task by learning from demonstrations, and has been widely applied to robotics, auton…
Towards Principled Representation Learning from Videos for Reinforcement Learning
Dipendra Misra, Akanksha Saran, Tengyang Xie +2
We study pre-training representations for decision-making using video data, which is abundantly available for tasks such as game agents and software testing. Even though significan…
Learning to Generate Better Than Your LLM
Jonathan D. Chang, Kiante Brantley, Rajkumar Ramamurthy +2
Reinforcement learning (RL) has emerged as a powerful paradigm for fine-tuning Large Language Models (LLMs) for text generation. In particular, recent LLMs such as ChatGPT and GPT-…
Survival Instinct in Offline Reinforcement Learning
Anqi Li, Dipendra Misra, Andrey Kolobov +1
We present a novel observation about the behavior of offline reinforcement learning (RL) algorithms: on many benchmark datasets, offline RL can produce well-performing and safe pol…
Towards Data-Driven Offline Simulations for Online Reinforcement Learning
Shengpu Tang, Felipe Vieira Frujeri, Dipendra Misra +4
Modern decision-making systems, from robots to web recommendation engines, are expected to adapt: to user preferences, changing circumstances or even new tasks. Yet, it is still un…
Agent-Controller Representations: Principled Offline RL with Rich Exogenous Information
Riashat Islam, Manan Tomar, Alex Lamb +8
Learning to control an agent from data collected offline in a rich pixel-based visual observation space is vital for real-world applications of reinforcement learning (RL). A major…
Provable Safe Reinforcement Learning with Binary Feedback
Andrew Bennett, Dipendra Misra, Nathan Kallus
Safety is a crucial necessity in many applications of reinforcement learning (RL), whether robotic, automotive, or medical. Many existing approaches to safe RL rely on receiving nu…
Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Models
Alex Lamb, Riashat Islam, Yonathan Efroni +7
In many sequential decision-making tasks, the agent is not able to model the full complexity of the world, which consists of multitudes of relevant and irrelevant information. For…
Sample-Efficient Reinforcement Learning in the Presence of Exogenous Information
Yonathan Efroni, Dylan J. Foster, Dipendra Misra +2
In real-world reinforcement learning applications the learner's observation space is ubiquitously high-dimensional with both relevant and irrelevant information about the task at h…
Provably Sample-Efficient RL with Side Information about Latent Dynamics
Yao Liu, Dipendra Misra, Miro Dudík +1
We study reinforcement learning (RL) in settings where observations are high-dimensional, but where an RL agent has access to abstract knowledge about the structure of the state sp…
Understanding Contrastive Learning Requires Incorporating Inductive Biases
Nikunj Saunshi, Jordan Ash, Surbhi Goel +5
Contrastive learning is a popular form of self-supervised learning that encourages augmentations (views) of the same input to have more similar representations compared to augmenta…
Provable RL with Exogenous Distractors via Multistep Inverse Dynamics
Yonathan Efroni, Dipendra Misra, Akshay Krishnamurthy +2
Many real-world applications of reinforcement learning (RL) require the agent to deal with high-dimensional observations such as those generated from a megapixel camera. Prior work…
Investigating the Role of Negatives in Contrastive Representation Learning
Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy +1
Noise contrastive learning is a popular technique for unsupervised representation learning. In this approach, a representation is obtained via reduction to supervised learning, whe…
Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learning
Dipendra Misra, Mikael Henaff, Akshay Krishnamurthy +1
We present an algorithm, HOMER, for exploration and reinforcement learning in rich observation environments that are summarizable by an unknown latent state space. The algorithm in…
Combating the Compounding-Error Problem with a Multi-step Model
Kavosh Asadi, Dipendra Misra, Seungchan Kim +1
Model-based reinforcement learning is an appealing framework for creating agents that learn, plan, and act in sequential environments. Model-based algorithms typically involve lear…
Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Howard Chen, Alane Suhr, Dipendra Misra +2
We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task. We introduce the Touchdown task and dataset, where an agent mus…
Early Fusion for Goal Directed Robotic Vision
Aaron Walsman, Yonatan Bisk, Saadia Gabriel +4
Building perceptual systems for robotics which perform well under tight computational budgets requires novel architectures which rethink the traditional computer vision pipeline. M…
Towards a Simple Approach to Multi-step Model-based Reinforcement Learning
Kavosh Asadi, Evan Cater, Dipendra Misra +1
When environmental interaction is expensive, model-based reinforcement learning offers a solution by planning ahead and avoiding costly mistakes. Model-based agents typically learn…
Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations
Dipendra Misra, Ming-Wei Chang, Xiaodong He +1
Semantic parsing from denotations faces two key challenges in model training: (1) given only the denotations (e.g., answers), search for good candidate semantic parses, and (2) cho…
Equivalence Between Wasserstein and Value-Aware Loss for Model-based Reinforcement Learning
Kavosh Asadi, Evan Cater, Dipendra Misra +1
Learning a generative model is a key component of model-based reinforcement learning. Though learning a good model in the tabular setting is a simple task, learning a useful model…
Lipschitz Continuity in Model-based Reinforcement Learning
Kavosh Asadi, Dipendra Misra, Michael L. Littman
We examine the impact of learning Lipschitz continuous models in the context of model-based reinforcement learning. We provide a novel bound on multi-step prediction error of Lipsc…
Mapping Instructions and Visual Observations to Actions with Reinforcement Learning
Dipendra Misra, John Langford, Yoav Artzi
We propose to directly map raw visual observations and text input to actions for instruction execution. While existing approaches assume access to structured environment representa…