Data Valuation using Reinforcement Learning

Abstract

Quantifying the value of data is a fundamental problem in machine learning. Data valuation has multiple important use cases: (1) building insights about the learning task, (2) domain adaptation, (3) corrupted sample discovery, and (4) robust learning. To adaptively learn data values jointly with the target task predictor model, we propose a meta learning framework which we name Data Valuation using Reinforcement Learning (DVRL). We employ a data value estimator (modeled by a deep neural network) to learn how likely each datum is used in training of the predictor model. We train the data value estimator using a reinforcement signal of the reward obtained on a small validation set that reflects performance on the target task. We demonstrate that DVRL yields superior data value estimates compared to alternative methods across different types of datasets and in a diverse set of application scenarios. The corrupted sample discovery performance of DVRL is close to optimal in many regimes (i.e. as if the noisy samples were known apriori), and for domain adaptation and robust learning DVRL significantly outperforms state-of-the-art by 14.6% and 10.8%, respectively.

17 pages, 12 figures, 6 tables

References in corpus (10)

• Categorical Reparameterization with Gumbel-Softmax
• Wide Residual Networks
• Understanding Black-box Predictions via Influence Functions
• Deep Learning Scaling is Predictable, Empirically
• Data Shapley: Equitable Valuation of Data for Machine Learning
• An Empirical Model of Large-Batch Training
• Learning with Bad Training Data via Iterative Trimmed Loss Minimization
• An Empirical Study of Example Forgetting during Deep Neural Network Learning
• Domain Adaptive Transfer Learning with Specialist Models
• Robust Learning at Noisy Labeled Medical Images: Applied to Skin Lesion Classification

Cited by in corpus (6)

• Improving Fairness for Data Valuation in Horizontal Federated Learning
• Image quality assessment for machine learning tasks using meta-reinforcement learning
• HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks
• Image quality assessment by overlapping task-specific and task-agnostic measures: application to prostate multiparametric MR images for cancer segmentation
• Efficient Client Contribution Evaluation for Horizontal Federated Learning
• Multitask Adaptation by Retrospective Exploration with Learned World Models