204 citations · 448 across the 11 of their papers we have counts for
7 papers · 1 filter
Prismer: A Vision-Language Model with Multi-Task Experts
Shikun Liu, Linxi Fan, Edward Johns +3
Recent vision-language models have shown impressive multi-modal generation capabilities. However, typically they require training huge models on massive datasets. As a more scalabl…
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Linxi Fan, Guanzhi Wang, Yunfan Jiang +7
Autonomous agents have made great strides in specialist domains like Atari games and Go. However, they typically learn tabula rasa in isolated environments with limited and manuall…
MetaMorph: Learning Universal Controllers with Transformers
Agrim Gupta, Linxi Fan, Surya Ganguli +1
Multiple domains like vision, natural language, and audio are witnessing tremendous progress by leveraging Transformers for large scale pre-training followed by task specific fine…
Pre-Trained Language Models for Interactive Decision-Making
Shuang Li, Xavier Puig, Chris Paxton +11
Language model (LM) pre-training is useful in many language processing tasks. But can pre-trained LMs be further leveraged for more general machine learning problems? We propose an…
SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
Linxi Fan, Guanzhi Wang, De-An Huang +4
Generalization has been a long-standing challenge for reinforcement learning (RL). Visual RL, in particular, can be easily distracted by irrelevant factors in high-dimensional obse…
SURREAL-System: Fully-Integrated Stack for Distributed Deep Reinforcement Learning
Linxi Fan, Yuke Zhu, Jiren Zhu +6
We present an overview of SURREAL-System, a reproducible, flexible, and scalable framework for distributed reinforcement learning (RL). The framework consists of a stack of four la…