A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning

Abstract

Much progress has been made in semi-supervised learning (SSL) by combining methods that exploit different aspects of the data distribution, e.g. consistency regularisation relies on properties of $p(x)$, whereas entropy minimisation pertains to the label distribution $p(y|x)$. Focusing on the latter, we present a probabilistic model for discriminative SSL, that mirrors its classical generative counterpart. Under the assumption $y|x$ is deterministic, the prior over latent variables becomes discrete. We show that several well-known SSL methods can be interpreted as approximating this prior, and can be improved upon. We extend the discriminative model to neuro-symbolic SSL, where label features satisfy logical rules, by showing such rules relate directly to the above prior, thus justifying a family of methods that link statistical learning and logical reasoning, and unifying them with regular SSL.

References in corpus (4)

• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
• Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge
• Semi-Supervised Learning using Differentiable Reasoning

Cited by in corpus (1)

• Learning the Prediction Distribution for Semi-Supervised Learning with Normalising Flows