Learning What Data to Learn

Abstract

Machine learning is essentially the sciences of playing with data. An adaptive data selection strategy, enabling to dynamically choose different data at various training stages, can reach a more effective model in a more efficient way. In this paper, we propose a deep reinforcement learning framework, which we call \emph{\textbf{N}eural \textbf{D}ata \textbf{F}ilter} (\textbf{NDF}), to explore automatic and adaptive data selection in the training process. In particular, NDF takes advantage of a deep neural network to adaptively select and filter important data instances from a sequential stream of training data, such that the future accumulative reward (e.g., the convergence speed) is maximized. In contrast to previous studies in data selection that is mainly based on heuristic strategies, NDF is quite generic and thus can be widely suitable for many machine learning tasks. Taking neural network training with stochastic gradient descent (SGD) as an example, comprehensive experiments with respect to various neural network modeling (e.g., multi-layer perceptron networks, convolutional neural networks and recurrent neural networks) and several applications (e.g., image classification and text understanding) demonstrate that NDF powered SGD can achieve comparable accuracy with standard SGD process by using less data and fewer iterations.

A preliminary version will appear in ICLR 2017, workshop track. https://openreview.net/forum?id=SyJNmVqgg&noteId=SyJNmVqgg

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• ADADELTA: An Adaptive Learning Rate Method
• Playing Atari with Deep Reinforcement Learning
• Asynchronous Methods for Deep Reinforcement Learning
• Learning to learn by gradient descent by gradient descent
• Online Batch Selection for Faster Training of Neural Networks
• The Optimal Reward Baseline for Gradient-Based Reinforcement Learning
• Learning to Optimize
• Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening

Cited by in corpus (13)

• Attention in Natural Language Processing
• Not All Samples Are Created Equal: Deep Learning with Importance Sampling
• Forward and Reverse Gradient-Based Hyperparameter Optimization
• MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
• Learning to Selectively Transfer: Reinforced Transfer Learning for Deep Text Matching
• Online Deep Reinforcement Learning for Autonomous UAV Navigation and Exploration of Outdoor Environments
• Accelerating Minibatch Stochastic Gradient Descent using Typicality Sampling
• Suggestive Annotation of Brain Tumour Images with Gradient-guided Sampling
• Investigating the Compositional Structure Of Deep Neural Networks
• Skeleton-Based Action Recognition with Synchronous Local and Non-local Spatio-temporal Learning and Frequency Attention
• Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations
• Exploiting Class Learnability in Noisy Data
• Seeking the Shape of Sound: An Adaptive Framework for Learning Voice-Face Association