A Distributional Framework for Data Valuation

Abstract

Shapley value is a classic notion from game theory, historically used to quantify the contributions of individuals within groups, and more recently applied to assign values to data points when training machine learning models. Despite its foundational role, a key limitation of the data Shapley framework is that it only provides valuations for points within a fixed data set. It does not account for statistical aspects of the data and does not give a way to reason about points outside the data set. To address these limitations, we propose a novel framework -- distributional Shapley -- where the value of a point is defined in the context of an underlying data distribution. We prove that distributional Shapley has several desirable statistical properties; for example, the values are stable under perturbations to the data points themselves and to the underlying data distribution. We leverage these properties to develop a new algorithm for estimating values from data, which comes with formal guarantees and runs two orders of magnitude faster than state-of-the-art algorithms for computing the (non-distributional) data Shapley values. We apply distributional Shapley to diverse data sets and demonstrate its utility in a data market setting.

References in corpus (5)

• Towards Efficient Data Valuation Based on the Shapley Value
• Data Shapley: Equitable Valuation of Data for Machine Learning
• Interpretation of Neural Networks is Fragile
• L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data
• Who's responsible? Jointly quantifying the contribution of the learning algorithm and training data

Cited by in corpus (7)

• Improving Fairness for Data Valuation in Horizontal Federated Learning
• Better, Not Just More: Data-Centric Machine Learning for Earth Observation
• Efficient computation and analysis of distributional Shapley values
• Who's responsible? Jointly quantifying the contribution of the learning algorithm and training data
• Analyzing and Improving Neural Networks by Generating Semantic Counterexamples through Differentiable Rendering
• The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research
• DU-Shapley: A Shapley Value Proxy for Efficient Dataset Valuation