Neural-Hidden-CRF: A Robust Weakly-Supervised Sequence Labeler

Abstract

We propose a neuralized undirected graphical model called Neural-Hidden-CRF to solve the weakly-supervised sequence labeling problem. Under the umbrella of probabilistic undirected graph theory, the proposed Neural-Hidden-CRF embedded with a hidden CRF layer models the variables of word sequence, latent ground truth sequence, and weak label sequence with the global perspective that undirected graphical models particularly enjoy. In Neural-Hidden-CRF, we can capitalize on the powerful language model BERT or other deep models to provide rich contextual semantic knowledge to the latent ground truth sequence, and use the hidden CRF layer to capture the internal label dependencies. Neural-Hidden-CRF is conceptually simple and empirically powerful. It obtains new state-of-the-art results on one crowdsourcing benchmark and three weak-supervision benchmarks, including outperforming the recent advanced model CHMM by 2.80 F1 points and 2.23 F1 points in average generalization and inference performance, respectively.

arXiv comment13 pages, 4 figures, accepted by SIGKDD-2023

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition
• Snorkel: Rapid Training Data Creation with Weak Supervision
• Data Programming: Creating Large Training Sets, Quickly
• An Overview of Deep Semi-Supervised Learning
• Knowledge Learning with Crowdsourcing: A Brief Review and Systematic Perspective
• A Survey on Programmatic Weak Supervision
• WRENCH: A Comprehensive Benchmark for Weak Supervision
• Sparse Conditional Hidden Markov Model for Weakly Supervised Named Entity Recognition