Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation

Abstract

Graph autoencoders (GAE) and variational graph autoencoders (VGAE) emerged as two powerful groups of unsupervised node embedding methods, with various applications to graph-based machine learning problems such as link prediction and community detection. Nonetheless, at the beginning of this Ph.D. project, GAE and VGAE models were also suffering from key limitations, preventing them from being adopted in the industry. In this thesis, we present several contributions to improve these models, with the general aim of facilitating their use to address industrial-level problems involving graph representations. Firstly, we propose two strategies to overcome the scalability issues of previous GAE and VGAE models, permitting to effectively train these models on large graphs with millions of nodes and edges. These strategies leverage graph degeneracy and stochastic subgraph decoding techniques, respectively. Besides, we introduce Gravity-Inspired GAE and VGAE, providing the first extensions of these models for directed graphs, that are ubiquitous in industrial applications. We also consider extensions of GAE and VGAE models for dynamic graphs. Furthermore, we argue that GAE and VGAE models are often unnecessarily complex, and we propose to simplify them by leveraging linear encoders. Lastly, we introduce Modularity-Aware GAE and VGAE to improve community detection on graphs, while jointly preserving good performances on link prediction. In the last part of this thesis, we evaluate our methods on several graphs extracted from the music streaming service Deezer. We put the emphasis on graph-based music recommendation problems. In particular, we show that our methods can improve the detection of communities of similar musical items to recommend to users, that they can effectively rank similar artists in a cold start setting, and that they permit modeling the music genre perception across cultures.

Ph.D. thesis defended at École Polytechnique (IPP) in March 2022. As mentioned in this thesis, several chapters present results also published in scientific articles written with co-authors

References in corpus (27)

• Fast unfolding of communities in large networks
• Modularity and community structure in networks
• LINE: Large-scale Information Network Embedding
• Cross-lingual Language Model Pretraining
• Deep Convolutional Networks on Graph-Structured Data
• Simplifying Graph Convolutional Networks
• Graph Convolutional Matrix Completion
• graph2vec: Learning Distributed Representations of Graphs
• A Generalization of Transformer Networks to Graphs
• Evaluating Link Prediction Methods
• OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
• Graph Neural Networks in Recommender Systems: A Survey
• Cascading Bandits: Learning to Rank in the Cascade Model
• Variational Graph Normalized Auto-Encoders
• Neural Expectation Maximization
• Graph-MLP: Node Classification without Message Passing in Graph
• Making Neural Networks Interpretable with Attribution: Application to Implicit Signals Prediction
• Cold Start Similar Artists Ranking with Gravity-Inspired Graph Autoencoders
• A Methodology for the Offline Evaluation of Recommender Systems in a User Interface with Multiple Carousels
• Predicting potential drug targets and repurposable drugs for COVID-19 via a deep generative model for graphs
• Dirichlet Graph Variational Autoencoder
• Mask-GVAE: Blind Denoising Graphs via Partition
• Unsupervised Community Detection with Modularity-Based Attention Model
• Deepened Graph Auto-Encoders Help Stabilize and Enhance Link Prediction
• Graph Autoencoders with Deconvolutional Networks
• New bounds for $k$-means and information $k$-means
• Modularity-Aware Graph Autoencoders for Joint Community Detection and Link Prediction