339 citations · 1.2k across the 31 of their papers we have counts for
31 papers
Learning to Model the World with Language
Jessy Lin, Yuqing Du, Olivia Watkins +4
To interact with humans and act in the world, agents need to understand the range of language that people use and relate it to the visual world. While current agents can learn to e…
Video Prediction Models as Rewards for Reinforcement Learning
Alejandro Escontrela, Ademi Adeniji, Wilson Yan +6
Specifying reward signals that allow agents to learn complex behaviors is a long-standing challenge in reinforcement learning. A promising approach is to extract preferences for be…
Mastering Diverse Domains through World Models
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba +1
Developing a general algorithm that learns to solve tasks across a wide range of applications has been a fundamental challenge in artificial intelligence. Although current reinforc…
Evaluating Long-Term Memory in 3D Mazes
Jurgis Pasukonis, Timothy Lillicrap, Danijar Hafner
Intelligent agents need to remember salient information to reason in partially-observed environments. For example, agents with a first-person view should remember the positions of…
Learning Robust Dynamics through Variational Sparse Gating
Arnav Kumar Jain, Shivakanth Sujit, Shruti Joshi +3
Learning world models from their sensory inputs enables agents to plan for actions by imagining their future outcomes. World models have previously been shown to improve sample-eff…
Temporally Consistent Transformers for Video Generation
Wilson Yan, Danijar Hafner, Stephen James +1
To generate accurate videos, algorithms have to understand the spatial and temporal dependencies in the world. Current algorithms enable accurate predictions over short horizons bu…