Learning Universal Graph Neural Network Embeddings With Aid Of Transfer Learning

Abstract

Learning powerful data embeddings has become a center piece in machine learning, especially in natural language processing and computer vision domains. The crux of these embeddings is that they are pretrained on huge corpus of data in a unsupervised fashion, sometimes aided with transfer learning. However currently in the graph learning domain, embeddings learned through existing graph neural networks (GNNs) are task dependent and thus cannot be shared across different datasets. In this paper, we present a first powerful and theoretically guaranteed graph neural network that is designed to learn task-independent graph embeddings, thereafter referred to as deep universal graph embedding (DUGNN). Our DUGNN model incorporates a novel graph neural network (as a universal graph encoder) and leverages rich Graph Kernels (as a multi-task graph decoder) for both unsupervised learning and (task-specific) adaptive supervised learning. By learning task-independent graph embeddings across diverse datasets, DUGNN also reaps the benefits of transfer learning. Through extensive experiments and ablation studies, we show that the proposed DUGNN model consistently outperforms both the existing state-of-art GNN models and Graph Kernels by an increased accuracy of 3% - 8% on graph classification benchmark datasets.

arXiv commentPrevious Paper Title: Deep Universal Graph Embedding Neural Network

References in corpus (25)

• Distributed Representations of Words and Phrases and their Compositionality
• Semi-Supervised Classification with Graph Convolutional Networks
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Spectral Networks and Locally Connected Networks on Graphs
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Deep Convolutional Networks on Graph-Structured Data
• Biological network comparison using graphlet degree distribution
• Deep Fragment Embeddings for Bidirectional Image Sentence Mapping
• Discriminative Embeddings of Latent Variable Models for Structured Data
• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
• MolGAN: An implicit generative model for small molecular graphs
• How Powerful are Graph Neural Networks?
• The Natural Language Decathlon: Multitask Learning as Question Answering
• Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation
• Anonymous Walk Embeddings
• The Multiscale Laplacian Graph Kernel
• Adaptive Graph Convolutional Neural Networks
• Graph Capsule Convolutional Neural Networks
• Learning Graph Representations with Embedding Propagation
• Deriving Neural Architectures from Sequence and Graph Kernels
• Unifying local and non-local signal processing with graph CNNs
• Fast Eigenspace Approximation using Random Signals
• Stability and Generalization of Graph Convolutional Neural Networks

Cited by in corpus (2)

• Time Series Data Augmentation for Neural Networks by Time Warping with a Discriminative Teacher
• Boosting Multitask Learning on Graphs through Higher-Order Task Affinities