103 citations · 253 across the 13 of their papers we have counts for
19 papers
An Empirical Study of Autoregressive Pre-training from Videos
Jathushan Rajasegaran, Ilija Radosavovic, Rahul Ravishankar +3
We empirically study autoregressive pre-training from videos. To perform our study, we construct a series of autoregressive video models, called Toto. We treat videos as sequences…
Learning Humanoid Locomotion over Challenging Terrain
Ilija Radosavovic, Sarthak Kamat, Trevor Darrell +1
Humanoid robots can, in principle, use their legs to go almost anywhere. Developing controllers capable of traversing diverse terrains, however, remains a considerable challenge. C…
DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
Alexander Khazatsky, Karl Pertsch, Suraj Nair +98
The creation of large, diverse, high-quality robot manipulation datasets is an important stepping stone on the path toward more capable and robust robotic manipulation policies. Ho…
Humanoid Locomotion as Next Token Prediction
Ilija Radosavovic, Bike Zhang, Baifeng Shi +5
We cast real-world humanoid control as a next token prediction problem, akin to predicting the next word in language. Our model is a causal transformer trained via autoregressive p…
Reconstructing Hands in 3D with Transformers
Georgios Pavlakos, Dandan Shan, Ilija Radosavovic +3
We present an approach that can reconstruct hands in 3D from monocular input. Our approach for Hand Mesh Recovery, HaMeR, follows a fully transformer-based architecture and can ana…
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, thi…
Robot Learning with Sensorimotor Pre-training
Ilija Radosavovic, Baifeng Shi, Letian Fu +3
We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. Given a seq…
Real-World Humanoid Locomotion with Reinforcement Learning
Ilija Radosavovic, Tete Xiao, Bike Zhang +3
Humanoid robots that can autonomously operate in diverse environments have the potential to help address labour shortages in factories, assist elderly at homes, and colonize new pl…
Learning to Imitate Object Interactions from Internet Videos
Austin Patel, Andrew Wang, Ilija Radosavovic +1
We study the problem of imitating object interactions from Internet videos. This requires understanding the hand-object interactions in 4D, spatially in 3D and over time, which is…
Real-World Robot Learning with Masked Visual Pre-training
Ilija Radosavovic, Tete Xiao, Stephen James +3
In this work, we explore self-supervised visual pre-training on images from diverse, in-the-wild videos for real-world robotic tasks. Like prior work, our visual representations ar…
Learning to Learn with Generative Models of Neural Network Checkpoints
William Peebles, Ilija Radosavovic, Tim Brooks +2
We explore a data-driven approach for learning to optimize neural networks. We construct a dataset of neural network checkpoints and train a generative model on the parameters. In…
Masked Visual Pre-training for Motor Control
Tete Xiao, Ilija Radosavovic, Trevor Darrell +1
This paper shows that self-supervised visual pre-training from real-world images is effective for learning motor control tasks from pixels. We first train the visual representation…
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Kristen Grauman, Andrew Westbury, Eugene Byrne +82
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of daily-life activity video spanning hundreds of scenarios (household, outd…
Reconstructing Hand-Object Interactions in the Wild
Zhe Cao, Ilija Radosavovic, Angjoo Kanazawa +1
In this work we explore reconstructing hand-object interactions in the wild. The core challenge of this problem is the lack of appropriate 3D labeled data. To overcome this issue,…
State-Only Imitation Learning for Dexterous Manipulation
Ilija Radosavovic, Xiaolong Wang, Lerrel Pinto +1
Modern model-free reinforcement learning methods have recently demonstrated impressive results on a number of problems. However, complex domains like dexterous manipulation remain…
Designing Network Design Spaces
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick +2
In this work, we present a new network design paradigm. Our goal is to help advance the understanding of network design and discover design principles that generalize across settin…
On Network Design Spaces for Visual Recognition
Ilija Radosavovic, Justin Johnson, Saining Xie +2
Over the past several years progress in designing better neural network architectures for visual recognition has been substantial. To help sustain this rate of progress, in this wo…
Attentive Single-Tasking of Multiple Tasks
Kevis-Kokitsi Maninis, Ilija Radosavovic, Iasonas Kokkinos
In this work we address task interference in universal networks by considering that a network is trained on multiple tasks, but performs one task at a time, an approach we refer to…
Data Distillation: Towards Omni-Supervised Learning
Ilija Radosavovic, Piotr Dollár, Ross Girshick +2
We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabe…