Local Boosting for Weakly-Supervised Learning

Abstract

Boosting is a commonly used technique to enhance the performance of a set of base models by combining them into a strong ensemble model. Though widely adopted, boosting is typically used in supervised learning where the data is labeled accurately. However, in weakly supervised learning, where most of the data is labeled through weak and noisy sources, it remains nontrivial to design effective boosting approaches. In this work, we show that the standard implementation of the convex combination of base learners can hardly work due to the presence of noisy labels. Instead, we propose $\text{LocalBoost}$, a novel framework for weakly-supervised boosting. LocalBoost iteratively boosts the ensemble model from two dimensions, i.e., intra-source and inter-source. The intra-source boosting introduces locality to the base learners and enables each base learner to focus on a particular feature regime by training new base learners on granularity-varying error regions. For the inter-source boosting, we leverage a conditional function to indicate the weak source where the sample is more likely to appear. To account for the weak labels, we further design an estimate-then-modify approach to compute the model weights. Experiments on seven datasets show that our method significantly outperforms vanilla boosting methods and other weakly-supervised methods.

Accepted by KDD 2023 Research Track

References in corpus (12)

• SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals
• A Survey on Programmatic Weak Supervision
• WRENCH: A Comprehensive Benchmark for Weak Supervision
• Learning from Rules Generalizing Labeled Exemplars
• Language Models in the Loop: Incorporating Prompting into Weak Supervision
• Weakly-Supervised Scientific Document Classification via Retrieval-Augmented Multi-Stage Training
• COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learning
• Do Not Blindly Imitate the Teacher: Using Perturbed Loss for Knowledge Distillation
• Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Prediction
• Leveraging Instance Features for Label Aggregation in Programmatic Weak Supervision
• ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval
• SepLL: Separating Latent Class Labels from Weak Supervision Noise

Cited by in corpus (1)

• R-Mixup: Riemannian Mixup for Biological Networks