DSSLP: A Distributed Framework for Semi-supervised Link Prediction

Abstract

Link prediction is widely used in a variety of industrial applications, such as merchant recommendation, fraudulent transaction detection, and so on. However, it's a great challenge to train and deploy a link prediction model on industrial-scale graphs with billions of nodes and edges. In this work, we present a scalable and distributed framework for semi-supervised link prediction problem (named DSSLP), which is able to handle industrial-scale graphs. Instead of training model on the whole graph, DSSLP is proposed to train on the \emph{$k$-hops neighborhood} of nodes in a mini-batch setting, which helps reduce the scale of the input graph and distribute the training procedure. In order to generate negative examples effectively, DSSLP contains a distributed batched runtime sampling module. It implements uniform and dynamic sampling approaches, and is able to adaptively construct positive and negative examples to guide the training process. Moreover, DSSLP proposes a model-split strategy to accelerate the speed of inference process of the link prediction task. Experimental results demonstrate that the effectiveness and efficiency of DSSLP in serval public datasets as well as real-world datasets of industrial-scale graphs.

References in corpus (11)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Semi-Supervised Classification with Graph Convolutional Networks
• Finding community structure in networks using the eigenvectors of matrices
• Hierarchical structure and the prediction of missing links in networks
• Clustering and preferential attachment in growing networks
• Graph Embedding Techniques, Applications, and Performance: A Survey
• Predicting Missing Links via Local Information
• Vertex similarity in networks
• Variational Graph Auto-Encoders
• Link Prediction Based on Local Random Walk
• Information filtering based on transferring similarity