Pretrained Encoders are All You Need
arXiv:2106.05139
Abstract
Data-efficiency and generalization are key challenges in deep learning and deep reinforcement learning as many models are trained on large-scale, domain-specific, and expensive-to-label datasets. Self-supervised models trained on large-scale uncurated datasets have shown successful transfer to diverse settings. We investigate using pretrained image representations and spatio-temporal attention for state representation learning in Atari. We also explore fine-tuning pretrained representations with self-supervised techniques, i.e., contrastive predictive coding, spatio-temporal contrastive learning, and augmentations. Our results show that pretrained representations are at par with state-of-the-art self-supervised methods trained on domain-specific data. Pretrained representations, thus, yield data and compute-efficient state representations. https://github.com/PAL-ML/PEARL_v1
References in corpus (25)
- Learning Transferable Visual Models From Natural Language Supervision
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- Bootstrap your own latent: A new approach to self-supervised Learning
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- Learning Representations by Maximizing Mutual Information Across Views
- EfficientDet: Scalable and Efficient Object Detection
- DeepMind Control Suite
- Big Self-Supervised Models are Strong Semi-Supervised Learners
- Model-Based Reinforcement Learning for Atari
- Reinforcement Learning with Augmented Data
- MONet: Unsupervised Scene Decomposition and Representation
- Dream to Control: Learning Behaviors by Latent Imagination
- DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
- Deep Attention Recurrent Q-Network
- Loss is its own Reward: Self-Supervision for Reinforcement Learning
- Dynamics-Aware Unsupervised Discovery of Skills
- Decoupling Representation Learning from Reinforcement Learning
- Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning
- Fast Task Inference with Variational Intrinsic Successor Features
- A Perspective on Objects and Systematic Generalization in Model-Based RL
- An initial attempt of combining visual selective attention with deep reinforcement learning
- Beyond Fine-Tuning: Transferring Behavior in Reinforcement Learning
- Behavior From the Void: Unsupervised Active Pre-Training
- Attention-Privileged Reinforcement Learning
- Personalizing Pre-trained Models