7 papers · 1 filter
Next-Latent Prediction Transformers Learn Compact World Models
Jayden Teoh, Manan Tomar, Kwangjun Ahn +7
Transformers replace recurrence with a memory that grows with sequence length and self-attention that enables ad-hoc lookups over past tokens. Consequently, they lack an inherent i…
The Belief State Transformer
Edward S. Hu, Kwangjun Ahn, Qinghua Liu +9
We introduce the "Belief State Transformer", a next-token predictor that takes both a prefix and suffix as inputs, with a novel objective of predicting both the next token for the…
Reward Centering
Abhishek Naik, Yi Wan, Manan Tomar +1
We show that discounted methods for solving continuing reinforcement learning problems can perform significantly better if they center their rewards by subtracting out the rewards'…
Model-Invariant State Abstractions for Model-Based Reinforcement Learning
Manan Tomar, Amy Zhang, Roberto Calandra +2
Accuracy and generalization of dynamics models is key to the success of model-based reinforcement learning (MBRL). As the complexity of tasks increases, so does the sample ineffici…
Multi-step Greedy Reinforcement Learning Algorithms
Manan Tomar, Yonathan Efroni, Mohammad Ghavamzadeh
Multi-step greedy policies have been extensively used in model-based reinforcement learning (RL), both when a model of the environment is available (e.g.,~in the game of Go) and wh…
MaMiC: Macro and Micro Curriculum for Robotic Reinforcement Learning
Manan Tomar, Akhil Sathuluri, Balaraman Ravindran
Shaping in humans and animals has been shown to be a powerful tool for learning complex tasks as compared to learning in a randomized fashion. This makes the problem less complex a…