HDMI: High-order Deep Multiplex Infomax

Abstract

Networks have been widely used to represent the relations between objects such as academic networks and social networks, and learning embedding for networks has thus garnered plenty of research attention. Self-supervised network representation learning aims at extracting node embedding without external supervision. Recently, maximizing the mutual information between the local node embedding and the global summary (e.g. Deep Graph Infomax, or DGI for short) has shown promising results on many downstream tasks such as node classification. However, there are two major limitations of DGI. Firstly, DGI merely considers the extrinsic supervision signal (i.e., the mutual information between node embedding and global summary) while ignores the intrinsic signal (i.e., the mutual dependence between node embedding and node attributes). Secondly, nodes in a real-world network are usually connected by multiple edges with different relations, while DGI does not fully explore the various relations among nodes. To address the above-mentioned problems, we propose a novel framework, called High-order Deep Multiplex Infomax (HDMI), for learning node embedding on multiplex networks in a self-supervised way. To be more specific, we first design a joint supervision signal containing both extrinsic and intrinsic mutual information by high-order mutual information, and we propose a High-order Deep Infomax (HDI) to optimize the proposed supervision signal. Then we propose an attention based fusion module to combine node embedding from different layers of the multiplex network. Finally, we evaluate the proposed HDMI on various downstream tasks such as unsupervised clustering and supervised classification. The experimental results show that HDMI achieves state-of-the-art performance on these tasks.

arXiv commentAccepted by WWW'2021 Updated section 3.1.1

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Distributed Representations of Words and Phrases and their Compositionality
• DeepWalk: Online Learning of Social Representations
• Semi-Supervised Classification with Graph Convolutional Networks
• LINE: Large-scale Information Network Embedding
• Inductive Representation Learning on Large Graphs
• Spectral Networks and Locally Connected Networks on Graphs
• Learning deep representations by mutual information estimation and maximization
• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
• Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering
• Graph Neural Networks: A Review of Methods and Applications
• Representation Learning for Attributed Multiplex Heterogeneous Network
• MINE: Mutual Information Neural Estimation
• InfoVAE: Information Maximizing Variational Autoencoders
• mvn2vec: Preservation and Collaboration in Multi-View Network Embedding

Cited by in corpus (22)

• Graph Self-Supervised Learning: A Survey
• Multiplex Heterogeneous Graph Convolutional Network
• Graph Masked Autoencoder for Sequential Recommendation
• Graph Communal Contrastive Learning
• HMSG: Heterogeneous Graph Neural Network based on Metapath Subgraph Learning
• RawlsGCN: Towards Rawlsian Difference Principle on Graph Convolutional Network
• Networked Time Series Imputation via Position-aware Graph Enhanced Variational Autoencoders
• X-GOAL: Multiplex Heterogeneous Graph Prototypical Contrastive Learning
• Relational Self-Supervised Learning on Graphs
• Hierarchical Aggregations for High-Dimensional Multiplex Graph Embedding
• Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputation
• Balanced Multi-Relational Graph Clustering
• Unsupervised Multiplex Graph Learning with Complementary and Consistent Information
• A Versatile Framework for Attributed Network Clustering via K-Nearest Neighbor Augmentation
• Multi-facet Contextual Bandits: A Neural Network Perspective
• InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learning
• Automated Contrastive Learning Strategy Search for Time Series
• Deep Contrastive Multiview Network Embedding
• A Geometric Perspective for High-Dimensional Multiplex Graphs
• Data-Driven Self-Supervised Graph Representation Learning
• Multiplex Graph Neural Network for Extractive Text Summarization
• Efficient Integration of Multi-View Attributed Graphs for Clustering and Embedding