Graph Transformer Networks

Abstract

Graph neural networks (GNNs) have been widely used in representation learning on graphs and achieved state-of-the-art performance in tasks such as node classification and link prediction. However, most existing GNNs are designed to learn node representations on the fixed and homogeneous graphs. The limitations especially become problematic when learning representations on a misspecified graph or a heterogeneous graph that consists of various types of nodes and edges. In this paper, we propose Graph Transformer Networks (GTNs) that are capable of generating new graph structures, which involve identifying useful connections between unconnected nodes on the original graph, while learning effective node representation on the new graphs in an end-to-end fashion. Graph Transformer layer, a core layer of GTNs, learns a soft selection of edge types and composite relations for generating useful multi-hop connections so-called meta-paths. Our experiments show that GTNs learn new graph structures, based on data and tasks without domain knowledge, and yield powerful node representation via convolution on the new graphs. Without domain-specific graph preprocessing, GTNs achieved the best performance in all three benchmark node classification tasks against the state-of-the-art methods that require pre-defined meta-paths from domain knowledge.

Neural Information Processing Systems (NeurIPS), 2019

Cited by in corpus (96)

• Graph Neural Networks: A Review of Methods and Applications
• Molecular Contrastive Learning of Representations via Graph Neural Networks
• A Generalization of Transformer Networks to Graphs
• A Survey of Knowledge-Enhanced Text Generation
• Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings
• FlowTransformer: A Transformer Framework for Flow-based Network Intrusion Detection Systems
• Graph Neural Networks: Taxonomy, Advances and Trends
• Do Transformers Really Perform Bad for Graph Representation?
• Boosting the Speed of Entity Alignment 10*: Dual Attention Matching Network with Normalized Hard Sample Mining
• Current and future directions in network biology
• Comprehensive evaluation of deep and graph learning on drug-drug interactions prediction
• Consistent Multiple Graph Embedding for Multi-View Clustering
• Learning Feature Recovery Transformer for Occluded Person Re-identification
• Multiplex Heterogeneous Graph Convolutional Network
• Knowledge-Preserving Incremental Social Event Detection via Heterogeneous GNNs
• Global Self-Attention as a Replacement for Graph Convolution
• Multi-Behavior Graph Neural Networks for Recommender System
• Graph Neural Networks for Natural Language Processing: A Survey
• Anomaly Detection in Dynamic Graphs: A Comprehensive Survey
• DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks
• Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
• Generalization and Representational Limits of Graph Neural Networks
• Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
• An Interdisciplinary Survey on Origin-destination Flows Modeling: Theory and Techniques
• xFraud: Explainable Fraud Transaction Detection
• Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries
• GraphiT: Encoding Graph Structure in Transformers
• AEGCN: An Autoencoder-Constrained Graph Convolutional Network
• A Learned Performance Model for Tensor Processing Units
• Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs
• Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interaction
• Homophily-oriented Heterogeneous Graph Rewiring
• Learning Graph Neural Networks with Positive and Unlabeled Nodes
• Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network
• Heterogeneous Graph Transformer
• Graph-Guided Network for Irregularly Sampled Multivariate Time Series
• Entity Alignment with Reliable Path Reasoning and Relation-Aware Heterogeneous Graph Transformer
• GripNet: Graph Information Propagation on Supergraph for Heterogeneous Graphs
• Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads
• Inductive Graph Transformer for Delivery Time Estimation
• Predicting cognitive scores with graph neural networks through sample selection learning
• Higher-Order Attribute-Enhancing Heterogeneous Graph Neural Networks
• Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph
• Group Identification via Transitional Hypergraph Convolution with Cross-view Self-supervised Learning
• Zero-Shot Learning with Common Sense Knowledge Graphs
• Forecaster: A Graph Transformer for Forecasting Spatial and Time-Dependent Data
• Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing
• Deep Generation of Heterogeneous Networks
• GraphConfRec: A Graph Neural Network-Based Conference Recommender System
• Heterogeneous Information Network-based Interest Composition with Graph Neural Network for Recommendation
• You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks
• AWB-GCN: A Graph Convolutional Network Accelerator with Runtime Workload Rebalancing
• Time-varying Signals Recovery via Graph Neural Networks
• HAWK: Rapid Android Malware Detection through Heterogeneous Graph Attention Networks
• Leveraging Large Language Model-based Room-Object Relationships Knowledge for Enhancing Multimodal-Input Object Goal Navigation
• R-GSN: The Relation-based Graph Similar Network for Heterogeneous Graph
• Graph-Aware Transformer: Is Attention All Graphs Need?
• Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networks
• CogDL: A Comprehensive Library for Graph Deep Learning
• Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networks
• Learning Graph Structure With A Finite-State Automaton Layer
• Learning on heterogeneous graphs using high-order relations
• Learning by Sampling and Compressing: Efficient Graph Representation Learning with Extremely Limited Annotations
• Node Classification Meets Link Prediction on Knowledge Graphs
• Self-supervised Auxiliary Learning for Graph Neural Networks via Meta-Learning
• DyFormer: A Scalable Dynamic Graph Transformer with Provable Benefits on Generalization Ability
• Hop-Hop Relation-aware Graph Neural Networks
• TransCamP: Graph Transformer for 6-DoF Camera Pose Estimation
• Leveraging Meta-path Contexts for Classification in Heterogeneous Information Networks
• Semantic Graph Neural Network with Multi-measure Learning for Semi-supervised Classification
• Meta-path Free Semi-supervised Learning for Heterogeneous Networks
• The Infinity Mirror Test for Graph Models
• A Deep Latent Space Model for Graph Representation Learning
• Pairwise Learning for Name Disambiguation in Large-Scale Heterogeneous Academic Networks
• Uniting Heterogeneity, Inductiveness, and Efficiency for Graph Representation Learning
• Knowledge Embedding Based Graph Convolutional Network
• Temporal Graph Network Embedding with Causal Anonymous Walks Representations
• GIPA: A General Information Propagation Algorithm for Graph Learning
• GSA-Forecaster: Forecasting Graph-Based Time-Dependent Data with Graph Sequence Attention
• Livewired Neural Networks: Making Neurons That Fire Together Wire Together
• Graph Mining under Data scarcity
• Optimizing Graph Transformer Networks with Graph-based Techniques
• Meta-Path Learning for Multi-relational Graph Neural Networks
• Sentence Structure and Word Relationship Modeling for Emphasis Selection
• Font Completion and Manipulation by Cycling Between Multi-Modality Representations
• Neural PathSim for Inductive Similarity Search in Heterogeneous Information Networks
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization
• Heterogeneous Graph Neural Network with Multi-view Representation Learning
• Improving Expressivity of Graph Neural Networks
• Graph Relation Transformer: Incorporating pairwise object features into the Transformer architecture
• Generating the Graph Gestalt: Kernel-Regularized Graph Representation Learning
• On Inductive Biases for Machine Learning in Data Constrained Settings
• VSGM -- Enhance robot task understanding ability through visual semantic graph
• Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning
• Sparse Graph to Sequence Learning for Vision Conditioned Long Textual Sequence Generation
• Layer-stacked Attention for Heterogeneous Network Embedding