Publications (13)
Representation Learning via Invariant Causal Mechanisms
Jovana Mitrovic, Brian McWilliams, Jacob Walker +2
Self-supervised learning has emerged as a strategy to reduce the reliance on costly supervised signal by pretraining representations only using unlabeled data. These methods combin…
Procedural Generalization by Planning with Self-Supervised World Models
Ankesh Anand, Jacob Walker, Yazhe Li +5
One of the key promises of model-based reinforcement learning is the ability to generalize using an internal model of the world to make predictions in novel environments and tasks.…
Predicting Video with VQVAE
Jacob Walker, Ali Razavi, Aäron van den Oord
In recent years, the task of video prediction-forecasting future video given past video frames-has attracted attention in the research community. In this paper we propose a novel a…
Video as the New Language for Real-World Decision Making
Sherry Yang, Jacob Walker, Jack Parker-Holder +5
Both text and video data are abundant on the internet and support large-scale self-supervised learning through next token or frame prediction. However, they have not been equally l…
Dense Optical Flow Prediction from a Static Image
Jacob Walker, Abhinav Gupta, Martial Hebert
Given a scene, what is going to move, and in what direction will it move? Such a question could be considered a non-semantic form of action prediction. In this work, we present a c…
Investigating the role of model-based learning in exploration and transfer
Jacob Walker, Eszter Vértes, Yazhe Li +4
State of the art reinforcement learning has enabled training agents on tasks of ever increasing complexity. However, the current paradigm tends to favor training agents from scratc…