Learning Dependency Structures for Weak Supervision Models

Abstract

Labeling training data is a key bottleneck in the modern machine learning pipeline. Recent weak supervision approaches combine labels from multiple noisy sources by estimating their accuracies without access to ground truth labels; however, estimating the dependencies among these sources is a critical challenge. We focus on a robust PCA-based algorithm for learning these dependency structures, establish improved theoretical recovery rates, and outperform existing methods on various real-world tasks. Under certain conditions, we show that the amount of unlabeled data needed can scale sublinearly or even logarithmically with the number of sources $m$, improving over previous efforts that ignore the sparsity pattern in the dependency structure and scale linearly in $m$. We provide an information-theoretic lower bound on the minimum sample complexity of the weak supervision setting. Our method outperforms weak supervision approaches that assume conditionally-independent sources by up to 4.64 F1 points and previous structure learning approaches by up to 4.41 F1 points on real-world relation extraction and image classification tasks.

References in corpus (4)

• Latent variable graphical model selection via convex optimization
• Inferring Generative Model Structure with Static Analysis
• Socratic Learning: Augmenting Generative Models to Incorporate Latent Subsets in Training Data
• On the Information Theoretic Limits of Learning Ising Models

Cited by in corpus (11)

• Ontology-driven weak supervision for clinical entity classification in electronic health records
• Trustable Co-label Learning from Multiple Noisy Annotators
• Learning with Weak Supervision for Email Intent Detection
• Overton: A Data System for Monitoring and Improving Machine-Learned Products
• Inspector Gadget: A Data Programming-based Labeling System for Industrial Images
• Knodle: Modular Weakly Supervised Learning with PyTorch
• Pairwise Feedback for Data Programming
• Comparing the Value of Labeled and Unlabeled Data in Method-of-Moments Latent Variable Estimation
• Learning from Multiple Noisy Partial Labelers
• Active WeaSuL: Improving Weak Supervision with Active Learning
• End-to-End Weak Supervision