Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation

Abstract

Graph autoencoders (GAE) and variational graph autoencoders (VGAE) emerged as two powerful groups of unsupervised node embedding methods, with various applications to graph-based machine learning problems such as link prediction and community detection. Nonetheless, at the beginning of this Ph.D. project, GAE and VGAE models were also suffering from key limitations, preventing them from being adopted in the industry. In this thesis, we present several contributions to improve these models, with the general aim of facilitating their use to address industrial-level problems involving graph representations. Firstly, we propose two strategies to overcome the scalability issues of previous GAE and VGAE models, permitting to effectively train these models on large graphs with millions of nodes and edges. These strategies leverage graph degeneracy and stochastic subgraph decoding techniques, respectively. Besides, we introduce Gravity-Inspired GAE and VGAE, providing the first extensions of these models for directed graphs, that are ubiquitous in industrial applications. We also consider extensions of GAE and VGAE models for dynamic graphs. Furthermore, we argue that GAE and VGAE models are often unnecessarily complex, and we propose to simplify them by leveraging linear encoders. Lastly, we introduce Modularity-Aware GAE and VGAE to improve community detection on graphs, while jointly preserving good performances on link prediction. In the last part of this thesis, we evaluate our methods on several graphs extracted from the music streaming service Deezer. We put the emphasis on graph-based music recommendation problems. In particular, we show that our methods can improve the detection of communities of similar musical items to recommend to users, that they can effectively rank similar artists in a cold start setting, and that they permit modeling the music genre perception across cultures.

arXiv commentPh.D. thesis defended at École Polytechnique (IPP) in March 2022. As mentioned in this thesis, several chapters present results also published in scientific articles written with co-authors

References in corpus (68)

• Scikit-learn: Machine Learning in Python
• Fast unfolding of communities in large networks
• Auto-Encoding Variational Bayes
• Modularity and community structure in networks
• Community detection in graphs
• TensorFlow: A system for large-scale machine learning
• LINE: Large-scale Information Network Embedding
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• Neural Message Passing for Quantum Chemistry
• Link Prediction in Complex Networks: A Survey
• Spectral Networks and Locally Connected Networks on Graphs
• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Cross-lingual Language Model Pretraining
• Exploiting Similarities among Languages for Machine Translation
• Tutorial on Variational Autoencoders
• Deep Convolutional Networks on Graph-Structured Data
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• Simplifying Graph Convolutional Networks
• Graph Convolutional Matrix Completion
• Representation Learning on Graphs with Jumping Knowledge Networks
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Pitfalls of Graph Neural Network Evaluation
• graph2vec: Learning Distributed Representations of Graphs
• Discriminative Embeddings of Latent Variable Models for Structured Data
• The Measure and Mismeasure of Fairness
• Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms
• Learning Structural Node Embeddings Via Diffusion Wavelets
• The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond
• Constrained Graph Variational Autoencoders for Molecule Design
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• A Generalization of Transformer Networks to Graphs
• Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
• Evaluating Link Prediction Methods
• Adaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization
• Stochastic Training of Graph Convolutional Networks with Variance Reduction
• Diffusion Improves Graph Learning
• Efficient Graph Generation with Graph Recurrent Attention Networks
• Online Learning: A Comprehensive Survey
• Universal Invariant and Equivariant Graph Neural Networks
• OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
• Graph Neural Networks in Recommender Systems: A Survey
• Cascading Bandits: Learning to Rank in the Cascade Model
• Solving Cold Start Problem in Recommendation with Attribute Graph Neural Networks
• Variational Graph Normalized Auto-Encoders
• Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-armed Bandit Problem with Multiple Plays
• Semi-Implicit Graph Variational Auto-Encoders
• DCM Bandits: Learning to Rank with Multiple Clicks
• Neural Expectation Maximization
• Graph-MLP: Node Classification without Message Passing in Graph
• An Ensemble Method to Produce High-Quality Word Embeddings (2016)
• Weakly chained matrices, policy iteration, and impulse control
• Making Neural Networks Interpretable with Attribution: Application to Implicit Signals Prediction
• Latent Contextual Bandits and their Application to Personalized Recommendations for New Users
• RWR-GAE: Random Walk Regularization for Graph Auto Encoders
• Multiple-Play Bandits in the Position-Based Model
• Multi-Task Graph Autoencoders
• Cold Start Similar Artists Ranking with Gravity-Inspired Graph Autoencoders
• A Methodology for the Offline Evaluation of Recommender Systems in a User Interface with Multiple Carousels
• Predicting potential drug targets and repurposable drugs for COVID-19 via a deep generative model for graphs
• Dirichlet Graph Variational Autoencoder
• Mask-GVAE: Blind Denoising Graphs via Partition
• Deepened Graph Auto-Encoders Help Stabilize and Enhance Link Prediction
• Unsupervised Community Detection with Modularity-Based Attention Model
• Graph Autoencoders with Deconvolutional Networks
• Extending the Davis-Kahan theorem for comparing eigenvectors of two symmetric matrices I: Theory
• New bounds for $k$-means and information $k$-means
• Modularity-Aware Graph Autoencoders for Joint Community Detection and Link Prediction