Accelerating Dynamic Network Embedding with Billions of Parameter Updates to Milliseconds

Abstract

Network embedding, a graph representation learning method illustrating network topology by mapping nodes into lower-dimension vectors, is challenging to accommodate the ever-changing dynamic graphs in practice. Existing research is mainly based on node-by-node embedding modifications, which falls into the dilemma of efficient calculation and accuracy. Observing that the embedding dimensions are usually much smaller than the number of nodes, we break this dilemma with a novel dynamic network embedding paradigm that rotates and scales the axes of embedding space instead of a node-by-node update. Specifically, we propose the Dynamic Adjacency Matrix Factorization (DAMF) algorithm, which achieves an efficient and accurate dynamic network embedding by rotating and scaling the coordinate system where the network embedding resides with no more than the number of edge modifications changes of node embeddings. Moreover, a dynamic Personalized PageRank is applied to the obtained network embeddings to enhance node embeddings and capture higher-order neighbor information dynamically. Experiments of node classification, link prediction, and graph reconstruction on different-sized dynamic graphs suggest that DAMF advances dynamic network embedding. Further, we unprecedentedly expand dynamic network embedding experiments to billion-edge graphs, where DAMF updates billion-level parameters in less than 10ms.

References in corpus (13)

• DeepWalk: Online Learning of Social Representations
• LINE: Large-scale Information Network Embedding
• Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec
• Predict then Propagate: Graph Neural Networks meet Personalized PageRank
• Attributed Network Embedding for Learning in a Dynamic Environment
• Scaling Graph Neural Networks with Approximate PageRank
• DynGEM: Deep Embedding Method for Dynamic Graphs
• VERSE: Versatile Graph Embeddings from Similarity Measures
• Node Embedding over Temporal Graphs
• NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization
• GloDyNE: Global Topology Preserving Dynamic Network Embedding
• Approximate Graph Propagation
• Network Embedding: An Overview