Multiple Kernel Representation Learning on Networks

Abstract

Learning representations of nodes in a low dimensional space is a crucial task with numerous interesting applications in network analysis, including link prediction, node classification, and visualization. Two popular approaches for this problem are matrix factorization and random walk-based models. In this paper, we aim to bring together the best of both worlds, towards learning node representations. In particular, we propose a weighted matrix factorization model that encodes random walk-based information about nodes of the network. The benefit of this novel formulation is that it enables us to utilize kernel functions without realizing the exact proximity matrix so that it enhances the expressiveness of existing matrix decomposition methods with kernels and alleviates their computational complexities. We extend the approach with a multiple kernel learning formulation that provides the flexibility of learning the kernel as the linear combination of a dictionary of kernels in data-driven fashion. We perform an empirical evaluation on real-world networks, showing that the proposed model outperforms baseline node embedding algorithms in downstream machine learning tasks.

arXiv commentThis manuscript is an extended version of the previous work entitled "Kernel Node Embeddings" (arXiv:1909.03416), and it has been accepted for publication in IEEE Transactions on Knowledge and Data Engineering

References in corpus (27)

• Scikit-learn: Machine Learning in Python
• Fast unfolding of communities in large networks
• Distributed Representations of Words and Phrases and their Compositionality
• DeepWalk: Online Learning of Social Representations
• LINE: Large-scale Information Network Embedding
• Kernel methods in machine learning
• Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec
• Clustering and Community Detection in Directed Networks: A Survey
• A Survey on Graph Kernels
• Connecting the Dots: Identifying Network Structure via Graph Signal Processing
• Graph Evolution: Densification and Shrinking Diameters
• VERSE: Versatile Graph Embeddings from Similarity Measures
• Nonnegative Matrix Factorization for Signal and Data Analytics: Identifiability, Algorithms, and Applications
• Kernel-based Reconstruction of Graph Signals
• Watch Your Step: Learning Node Embeddings via Graph Attention
• HARP: Hierarchical Representation Learning for Networks
• NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization
• Is a Single Embedding Enough? Learning Node Representations that Capture Multiple Social Contexts
• Kernel-Based Structural Equation Models for Topology Identification of Directed Networks
• GEMSEC: Graph Embedding with Self Clustering
• Don't Walk, Skip! Online Learning of Multi-scale Network Embeddings
• Adaptive Diffusions for Scalable Learning over Graphs
• Rethinking Kernel Methods for Node Representation Learning on Graphs
• GraphKKE: Graph Kernel Koopman Embedding for Human Microbiome Analysis
• Topic-aware latent models for representation learning on networks
• TNE: A Latent Model for Representation Learning on Networks
• InfiniteWalk: Deep Network Embeddings as Laplacian Embeddings with a Nonlinearity