Discriminative Embeddings of Latent Variable Models for Structured Data

Abstract

Kernel classifiers and regressors designed for structured data, such as sequences, trees and graphs, have significantly advanced a number of interdisciplinary areas such as computational biology and drug design. Typically, kernels are designed beforehand for a data type which either exploit statistics of the structures or make use of probabilistic generative models, and then a discriminative classifier is learned based on the kernels via convex optimization. However, such an elegant two-stage approach also limited kernel methods from scaling up to millions of data points, and exploiting discriminative information to learn feature representations. We propose, structure2vec, an effective and scalable approach for structured data representation based on the idea of embedding latent variable models into feature spaces, and learning such feature spaces using discriminative information. Interestingly, structure2vec extracts features by performing a sequence of function mappings in a way similar to graphical model inference procedures, such as mean field and belief propagation. In applications involving millions of data points, we showed that structure2vec runs 2 times faster, produces models which are $10,000$ times smaller, while at the same time achieving the state-of-the-art predictive performance.

ICML 2016

References in corpus (5)

• Loopy Belief Propagation for Approximate Inference: An Empirical Study
• Deep Convolutional Networks on Graph-Structured Data
• Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
• Learning Deep Structured Models
• Kernel Belief Propagation

Cited by in corpus (152)

• Inductive Representation Learning on Large Graphs
• Graph Neural Networks: A Review of Methods and Applications
• Learning Combinatorial Optimization Algorithms over Graphs
• Graph Contrastive Learning with Augmentations
• Hierarchical Graph Representation Learning with Differentiable Pooling
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• GNNExplainer: Generating Explanations for Graph Neural Networks
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Learning Structural Node Embeddings Via Diffusion Wavelets
• Self-Attention Graph Pooling
• A Comprehensive Survey of Scene Graphs: Generation and Application
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• Graph-based Deep Learning for Communication Networks: A Survey
• Link Prediction Based on Graph Neural Networks
• Graph networks as learnable physics engines for inference and control
• Attention-based Graph Neural Network for Semi-supervised Learning
• Neural Relational Inference for Interacting Systems
• Syntax-Directed Variational Autoencoder for Structured Data
• Topology Adaptive Graph Convolutional Networks
• An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem
• Deep Reinforcement Learning for Electric Vehicle Routing Problem with Time Windows
• Towards Sparse Hierarchical Graph Classifiers
• LanczosNet: Multi-Scale Deep Graph Convolutional Networks
• Reinforcement Learning for Solving the Vehicle Routing Problem
• Learning Combinatorial Optimization on Graphs: A Survey with Applications to Networking
• Deep Learning on Graphs: A Survey
• Adversarial Attack on Graph Structured Data
• Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
• A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Easing Embedding Learning by Comprehensive Transcription of Heterogeneous Information Networks
• POMO: Policy Optimization with Multiple Optima for Reinforcement Learning
• Hierarchical Generation of Molecular Graphs using Structural Motifs
• Revisiting Binary Code Similarity Analysis using Interpretable Feature Engineering and Lessons Learned
• Advances of Machine Learning in Materials Science: Ideas and Techniques
• Quantum chemistry-augmented neural networks for reactivity prediction: Performance, generalizability and interpretability
• A Survey on The Expressive Power of Graph Neural Networks
• Representation Learning for Natural Language Processing
• Accurate Learning of Graph Representations with Graph Multiset Pooling
• Language-Agnostic Representation Learning of Source Code from Structure and Context
• Graph Kernels: State-of-the-Art and Future Challenges
• Learning the Multiple Traveling Salesmen Problem with Permutation Invariant Pooling Networks
• Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning
• mvn2vec: Preservation and Collaboration in Multi-View Network Embedding
• Implicit Graph Neural Networks
• Generalization and Representational Limits of Graph Neural Networks
• Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification
• Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs
• Representation Learning over Dynamic Graphs
• LibDB: An Effective and Efficient Framework for Detecting Third-Party Libraries in Binaries
• Coloring Big Graphs with AlphaGoZero
• Learning Permutations with Sinkhorn Policy Gradient
• Efficient Probabilistic Logic Reasoning with Graph Neural Networks
• V-Fuzz: Vulnerability-Oriented Evolutionary Fuzzing
• Graphite: Iterative Generative Modeling of Graphs
• Nested Graph Neural Networks
• M-Evolve: Structural-Mapping-Based Data Augmentation for Graph Classification
• Approximating Network Centrality Measures Using Node Embedding and Machine Learning
• Inductive Matrix Completion Based on Graph Neural Networks
• Node Injection Attacks on Graphs via Reinforcement Learning
• Hierarchical Graph-to-Graph Translation for Molecules
• Network representation learning systematic review: ancestors and current development state
• Semantic Learning and Emulation Based Cross-platform Binary Vulnerability Seeker
• Spectral Multigraph Networks for Discovering and Fusing Relationships in Molecules
• DISCO: Influence Maximization Meets Network Embedding and Deep Learning
• Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning
• KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings
• Graph-based Solutions with Residuals for Intrusion Detection: the Modified E-GraphSAGE and E-ResGAT Algorithms
• Optimal Transport Graph Neural Networks
• Deriving Neural Architectures from Sequence and Graph Kernels
• Solving NP-Hard Problems on Graphs with Extended AlphaGo Zero
• TitAnt: Online Real-time Transaction Fraud Detection in Ant Financial
• Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach
• A Bi-Level Framework for Learning to Solve Combinatorial Optimization on Graphs
• Spatial Dual-Modality Graph Reasoning for Key Information Extraction
• SimGNN: A Neural Network Approach to Fast Graph Similarity Computation
• Goal-directed graph construction using reinforcement learning
• Interpreting the Mechanism of Synergism for Drug Combinations Using Attention-Based Hierarchical Graph Pooling
• A State Aggregation Approach for Solving Knapsack Problem with Deep Reinforcement Learning
• Deep Representation Learning for Social Network Analysis
• A Deep Reinforcement Learning Algorithm Using Dynamic Attention Model for Vehicle Routing Problems
• Multi-scale Dynamic Graph Convolutional Network for Hyperspectral Image Classification
• Stability and Generalization of Graph Convolutional Neural Networks
• Deep Graph Similarity Learning: A Survey
• Solve Traveling Salesman Problem by Monte Carlo Tree Search and Deep Neural Network
• Memory-Based Graph Networks
• How to Evaluate Machine Learning Approaches for Combinatorial Optimization: Application to the Travelling Salesman Problem
• Can Graph Neural Networks Help Logic Reasoning?
• A Graph Neural Network Approach for Scalable Wireless Power Control
• Learning Universal Graph Neural Network Embeddings With Aid Of Transfer Learning
• Unsupervised Features Extraction for Binary Similarity Using Graph Embedding Neural Networks
• Zoo Guide to Network Embedding
• Graph Convolutional Networks with EigenPooling
• Learning Graph-Level Representations with Recurrent Neural Networks
• Enforcing Predictive Invariance across Structured Biomedical Domains
• Deep Learning in Information Security
• RetCL: A Selection-based Approach for Retrosynthesis via Contrastive Learning
• Adversarial Attacks on Graph Classification via Bayesian Optimisation
• Multi-Channel Graph Convolutional Networks
• Feature Propagation on Graph: A New Perspective to Graph Representation Learning
• Topology Aware Deep Learning for Wireless Network Optimization
• Amortized Bethe Free Energy Minimization for Learning MRFs
• Heterogeneous Graph Neural Networks for Malicious Account Detection
• InfDetect: a Large Scale Graph-based Fraud Detection System for E-Commerce Insurance
• DeepGG: a Deep Graph Generator
• GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategy
• Black-box Attacks Against Neural Binary Function Detection
• MxPool: Multiplex Pooling for Hierarchical Graph Representation Learning
• Deep Hypergraph U-Net for Brain Graph Embedding and Classification
• Neural Message Passing for Multi-Label Classification
• DPPIN: A Biological Repository of Dynamic Protein-Protein Interaction Network Data
• CommPOOL: An Interpretable Graph Pooling Framework for Hierarchical Graph Representation Learning
• Graph Cross Networks with Vertex Infomax Pooling
• Graph Polish: A Novel Graph Generation Paradigm for Molecular Optimization
• Graph Neural Networks for Scalable Radio Resource Management: Architecture Design and Theoretical Analysis
• Train on Small, Play the Large: Scaling Up Board Games with AlphaZero and GNN
• CORE: Automatic Molecule Optimization Using Copy & Refine Strategy
• Co-training for Policy Learning
• A Biased Graph Neural Network Sampler with Near-Optimal Regret
• An In-depth Summary of Recent Artificial Intelligence Applications in Drug Design
• Controlled Molecule Generator for Optimizing Multiple Chemical Properties
• Process-Level Representation of Scientific Protocols with Interactive Annotation
• Histopathology WSI Encoding based on GCNs for Scalable and Efficient Retrieval of Diagnostically Relevant Regions
• Active Screening for Recurrent Diseases: A Reinforcement Learning Approach
• Online Multi-Agent Forecasting with Interpretable Collaborative Graph Neural Network
• Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing
• Revocable Deep Reinforcement Learning with Affinity Regularization for Outlier-Robust Graph Matching
• Sparse hierarchical representation learning on molecular graphs
• Unsupervised Hierarchical Graph Representation Learning by Mutual Information Maximization
• Inductive Subgraph Embedding for Link Prediction
• Learning Features of Network Structures Using Graphlets
• GIPA: A General Information Propagation Algorithm for Graph Learning
• Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back
• Semi-Supervised Graph Classification: A Hierarchical Graph Perspective
• Ego-CNN: Distributed, Egocentric Representations of Graphs for Detecting Critical Structures
• Solving Graph-based Public Good Games with Tree Search and Imitation Learning
• OpenGraphGym-MG: Using Reinforcement Learning to Solve Large Graph Optimization Problems on MultiGPU Systems
• MSHCNet: Multi-Stream Hybridized Convolutional Networks with Mixed Statistics in Euclidean/Non-Euclidean Spaces and Its Application to Hyperspectral Image Classification
• SeaPearl: A Constraint Programming Solver guided by Reinforcement Learning
• Zero Training Overhead Portfolios for Learning to Solve Combinatorial Problems
• Beyond Observed Connections : Link Injection
• A General Framework for Charger Scheduling Optimization Problems
• Customized Graph Embedding: Tailoring Embedding Vectors to different Applications
• cube2net: Efficient Query-Specific Network Construction with Data Cube Organization
• Graph Coarsening with Preserved Spectral Properties
• Graph Convolutional Policy for Solving Tree Decomposition via Reinforcement Learning Heuristics
• MGA: Momentum Gradient Attack on Network
• Accelerating Metropolis-Hastings with Lightweight Inference Compilation
• New Insights into Graph Convolutional Networks using Neural Tangent Kernels
• Vulcan: Solving the Steiner Tree Problem with Graph Neural Networks and Deep Reinforcement Learning
• Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation
• SAFE: Self-Attentive Function Embeddings for Binary Similarity