Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
arXiv:1906.02768
Abstract
The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization being present rather than by helping the optimization process (Frankle & Carbin, 2019). Intriguingly, this phenomenon suggests that initialization strategies for DNNs can be improved substantially, but the lottery ticket hypothesis has only previously been tested in the context of supervised learning for natural image tasks. Here, we evaluate whether "winning ticket" initializations exist in two different domains: natural language processing (NLP) and reinforcement learning (RL).For NLP, we examined both recurrent LSTM models and large-scale Transformer models (Vaswani et al., 2017). For RL, we analyzed a number of discrete-action space tasks, including both classic control and pixel control. Consistent with workin supervised image classification, we confirm that winning ticket initializations generally outperform parameter-matched random initializations, even at extreme pruning rates for both NLP and RL. Notably, we are able to find winning ticket initializations for Transformers which enable models one-third the size to achieve nearly equivalent performance. Together, these results suggest that the lottery ticket hypothesis is not restricted to supervised learning of natural images, but rather represents a broader phenomenon in DNNs.
arXiv commentICLR 2020
References in corpus (12)
- Efficient Estimation of Word Representations in Vector Space
- Sequence to Sequence Learning with Neural Networks
- How transferable are features in deep neural networks?
- Pruning Convolutional Neural Networks for Resource Efficient Inference
- Rethinking the Value of Network Pruning
- A Convergence Theory for Deep Learning via Over-Parameterization
- The State of Sparsity in Deep Neural Networks
- Variational Dropout Sparsifies Deep Neural Networks
- Quantifying Generalization in Reinforcement Learning
- Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
- On the Power of Over-parametrization in Neural Networks with Quadratic Activation
- Can Deep Reinforcement Learning Solve Erdos-Selfridge-Spencer Games?
Cited by in corpus (32)
- Linear Mode Connectivity and the Lottery Ticket Hypothesis
- Interpreting Deep Learning-Based Networking Systems
- Pruning neural networks without any data by iteratively conserving synaptic flow
- Breaking Adiabatic Quantum Control with Deep Learning
- The Lottery Ticket Hypothesis for Pre-trained BERT Networks
- When BERT Plays the Lottery, All Tickets Are Winning
- Movement Pruning: Adaptive Sparsity by Fine-Tuning
- The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models
- A Unified Lottery Ticket Hypothesis for Graph Neural Networks
- A Programmable Approach to Neural Network Compression
- Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
- EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
- Robust Lottery Tickets for Pre-trained Language Models
- Learning Sparse Sharing Architectures for Multiple Tasks
- PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
- Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
- Exploring Lottery Ticket Hypothesis in Media Recommender Systems
- Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
- Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
- The Differentially Private Lottery Ticket Mechanism
- Image Captioning with Sparse Recurrent Neural Network
- Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks
- Reinforcement Learning with Chromatic Networks for Compact Architecture Search
- Bespoke vs. Prêt-à-Porter Lottery Tickets: Exploiting Mask Similarity for Trainable Sub-Network Finding
- SuperShaper: Task-Agnostic Super Pre-training of BERT Models with Variable Hidden Dimensions
- Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
- Spending Your Winning Lottery Better After Drawing It
- On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning
- Disentangling Neural Architectures and Weights: A Case Study in Supervised Classification
- Differentiable Architecture Pruning for Transfer Learning
- Membership Inference Attacks on Lottery Ticket Networks
- Data-Informed Global Sparseness in Attention Mechanisms for Deep Neural Networks