Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back

Abstract

A graph is a powerful concept for representation of relations between pairs of entities. Data with underlying graph structure can be found across many disciplines and there is a natural desire for understanding such data better. Deep learning (DL) has achieved significant breakthroughs in a variety of machine learning tasks in recent years, especially where data is structured on a grid, such as in text, speech, or image understanding. However, surprisingly little has been done to explore the applicability of DL on arbitrary graph-structured data directly. The goal of this thesis is to investigate architectures for DL on graphs and study how to transfer, adapt or generalize concepts that work well on sequential and image data to this domain. We concentrate on two important primitives: embedding graphs or their nodes into a continuous vector space representation (encoding) and, conversely, generating graphs from such vectors back (decoding). To that end, we make the following contributions. First, we introduce Edge-Conditioned Convolutions (ECC), a convolution-like operation on graphs performed in the spatial domain where filters are dynamically generated based on edge attributes. The method is used to encode graphs with arbitrary and varying structure. Second, we propose SuperPoint Graph, an intermediate point cloud representation with rich edge attributes encoding the contextual relationship between object parts. Based on this representation, ECC is employed to segment large-scale point clouds without major sacrifice in fine details. Third, we present GraphVAE, a graph generator allowing us to decode graphs with variable but upper-bounded number of nodes making use of approximate graph matching for aligning the predictions of an autoencoder with its inputs. The method is applied to the task of molecule generation.

PhD Thesis

References in corpus (35)

• Emergence of scaling in random networks
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• Conditional Random Fields as Recurrent Neural Networks
• Generative Adversarial Text to Image Synthesis
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Pixel Recurrent Neural Networks
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Learning Combinatorial Optimization Algorithms over Graphs
• Variational Graph Auto-Encoders
• Interaction Networks for Learning about Objects, Relations and Physics
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Discriminative Embeddings of Latent Variable Models for Structured Data
• MolGAN: An implicit generative model for small molecular graphs
• Learning Deep Generative Models of Graphs
• Designing Neural Network Architectures using Reinforcement Learning
• Generative Moment Matching Networks
• Convolutional Neural Networks over Tree Structures for Programming Language Processing
• GSPBOX: A toolbox for signal processing on graphs
• Fully Connected Deep Structured Networks
• Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• NetGAN: Generating Graphs via Random Walks
• A Spectral Graph Uncertainty Principle
• Improving Variational Inference with Inverse Autoregressive Flow
• Chebyshev Polynomial Approximation for Distributed Signal Processing
• subgraph2vec: Learning Distributed Representations of Rooted Sub-graphs from Large Graphs
• On the Graph Fourier Transform for Directed Graphs
• N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials
• Graph Based Convolutional Neural Network