On the Transferability of VAE Embeddings using Relational Knowledge with Semi-Supervision

Abstract

We propose a new model for relational VAE semi-supervision capable of balancing disentanglement and low complexity modelling of relations with different symbolic properties. We compare the relative benefits of relation-decoder complexity and latent space structure on both inductive and transductive transfer learning. Our results depict a complex picture where enforcing structure on semi-supervised representations can greatly improve zero-shot transductive transfer, but may be less favourable or even impact negatively the capacity for inductive transfer.

References in corpus (13)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Auto-Encoding Variational Bayes
• A Review of Relational Machine Learning for Knowledge Graphs
• Semi-Supervised Learning with Deep Generative Models
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• A simple neural network module for relational reasoning
• Understanding disentangling in $\beta$-VAE
• Measuring abstract reasoning in neural networks
• Weakly-Supervised Disentanglement Without Compromises
• Neural Arithmetic Logic Units
• From Knowledge Graph Embedding to Ontology Embedding? An Analysis of the Compatibility between Vector Space Representations and Rules
• Neural Arithmetic Units
• Robust Ordinal VAE: Employing Noisy Pairwise Comparisons for Disentanglement