Analyzing Data Selection Techniques with Tools from the Theory of Information Losses

Abstract

In this paper, we present and illustrate some new tools for rigorously analyzing training data selection methods. These tools focus on the information theoretic losses that occur when sampling data. We use this framework to prove that two methods, Facility Location Selection and Transductive Experimental Design, reduce these losses. These are meant to act as generalizable theoretical examples of applying the field of Information Theoretic Deep Learning Theory to the fields of data selection and active learning. Both analyses yield insight into their respective methods and increase their interpretability. In the case of Transductive Experimental Design, the provided analysis greatly increases the method's scope as well.

arXiv commentThis paper has now been published as a conference proceeding in IEEE Big Data 2021

References in corpus (7)

• OpenML: networked science in machine learning
• Opening the Black Box of Deep Neural Networks via Information
• Active Learning for Convolutional Neural Networks: A Core-Set Approach
• Deep Bayesian Active Learning with Image Data
• Agnostic Active Learning Without Constraints
• UPAL: Unbiased Pool Based Active Learning
• Information Losses in Neural Classifiers from Sampling

Cited by in corpus (1)

• On the Maximum Mutual Information Capacity of Neural Architectures