Discriminative Embeddings of Latent Variable Models for Structured Data

Abstract

Kernel classifiers and regressors designed for structured data, such as sequences, trees and graphs, have significantly advanced a number of interdisciplinary areas such as computational biology and drug design. Typically, kernels are designed beforehand for a data type which either exploit statistics of the structures or make use of probabilistic generative models, and then a discriminative classifier is learned based on the kernels via convex optimization. However, such an elegant two-stage approach also limited kernel methods from scaling up to millions of data points, and exploiting discriminative information to learn feature representations. We propose, structure2vec, an effective and scalable approach for structured data representation based on the idea of embedding latent variable models into feature spaces, and learning such feature spaces using discriminative information. Interestingly, structure2vec extracts features by performing a sequence of function mappings in a way similar to graphical model inference procedures, such as mean field and belief propagation. In applications involving millions of data points, we showed that structure2vec runs 2 times faster, produces models which are $10,000$ times smaller, while at the same time achieving the state-of-the-art predictive performance.

ICML 2016

References in corpus (8)

• Conditional Random Fields as Recurrent Neural Networks
• Loopy Belief Propagation for Approximate Inference: An Empirical Study
• Deep Convolutional Networks on Graph-Structured Data
• Convolutional Neural Networks over Tree Structures for Programming Language Processing
• Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
• Learning Deep Structured Models
• Kernel Belief Propagation
• Deeply Learning the Messages in Message Passing Inference

Cited by in corpus (153)

• Inductive Representation Learning on Large Graphs
• Graph Neural Networks: A Review of Methods and Applications
• Learning Combinatorial Optimization Algorithms over Graphs
• Graph Contrastive Learning with Augmentations
• Hierarchical Graph Representation Learning with Differentiable Pooling
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• GNNExplainer: Generating Explanations for Graph Neural Networks
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Learning Structural Node Embeddings Via Diffusion Wavelets
• Self-Attention Graph Pooling
• A Comprehensive Survey of Scene Graphs: Generation and Application
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• Graph-based Deep Learning for Communication Networks: A Survey
• Link Prediction Based on Graph Neural Networks
• Graph networks as learnable physics engines for inference and control
• Attention-based Graph Neural Network for Semi-supervised Learning
• Neural Relational Inference for Interacting Systems
• Syntax-Directed Variational Autoencoder for Structured Data
• Topology Adaptive Graph Convolutional Networks
• An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem
• Deep Reinforcement Learning for Electric Vehicle Routing Problem with Time Windows
• Towards Sparse Hierarchical Graph Classifiers
• LanczosNet: Multi-Scale Deep Graph Convolutional Networks
• Reinforcement Learning for Solving the Vehicle Routing Problem
• Learning Combinatorial Optimization on Graphs: A Survey with Applications to Networking
• Deep Learning on Graphs: A Survey
• Adversarial Attack on Graph Structured Data
• Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
• A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Easing Embedding Learning by Comprehensive Transcription of Heterogeneous Information Networks
• POMO: Policy Optimization with Multiple Optima for Reinforcement Learning
• The Multiscale Laplacian Graph Kernel
• Hierarchical Generation of Molecular Graphs using Structural Motifs
• Revisiting Binary Code Similarity Analysis using Interpretable Feature Engineering and Lessons Learned
• Advances of Machine Learning in Materials Science: Ideas and Techniques
• Quantum chemistry-augmented neural networks for reactivity prediction: Performance, generalizability and interpretability
• A Survey on The Expressive Power of Graph Neural Networks
• Representation Learning for Natural Language Processing
• Accurate Learning of Graph Representations with Graph Multiset Pooling
• Language-Agnostic Representation Learning of Source Code from Structure and Context
• Graph Kernels: State-of-the-Art and Future Challenges
• Learning the Multiple Traveling Salesmen Problem with Permutation Invariant Pooling Networks
• Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning
• mvn2vec: Preservation and Collaboration in Multi-View Network Embedding
• Implicit Graph Neural Networks
• Generalization and Representational Limits of Graph Neural Networks
• Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification
• Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs
• Representation Learning over Dynamic Graphs
• LibDB: An Effective and Efficient Framework for Detecting Third-Party Libraries in Binaries
• Coloring Big Graphs with AlphaGoZero
• Learning Permutations with Sinkhorn Policy Gradient
• Efficient Probabilistic Logic Reasoning with Graph Neural Networks
• Graphite: Iterative Generative Modeling of Graphs
• V-Fuzz: Vulnerability-Oriented Evolutionary Fuzzing
• Nested Graph Neural Networks
• Approximating Network Centrality Measures Using Node Embedding and Machine Learning
• M-Evolve: Structural-Mapping-Based Data Augmentation for Graph Classification
• Node Injection Attacks on Graphs via Reinforcement Learning
• Inductive Matrix Completion Based on Graph Neural Networks
• Hierarchical Graph-to-Graph Translation for Molecules
• Network representation learning systematic review: ancestors and current development state
• Semantic Learning and Emulation Based Cross-platform Binary Vulnerability Seeker
• Spectral Multigraph Networks for Discovering and Fusing Relationships in Molecules
• KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings
• Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning
• DISCO: Influence Maximization Meets Network Embedding and Deep Learning
• Deriving Neural Architectures from Sequence and Graph Kernels
• Graph-based Solutions with Residuals for Intrusion Detection: the Modified E-GraphSAGE and E-ResGAT Algorithms
• Optimal Transport Graph Neural Networks
• Solving NP-Hard Problems on Graphs with Extended AlphaGo Zero
• TitAnt: Online Real-time Transaction Fraud Detection in Ant Financial
• Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach
• A Bi-Level Framework for Learning to Solve Combinatorial Optimization on Graphs
• SimGNN: A Neural Network Approach to Fast Graph Similarity Computation
• Spatial Dual-Modality Graph Reasoning for Key Information Extraction
• Goal-directed graph construction using reinforcement learning
• A State Aggregation Approach for Solving Knapsack Problem with Deep Reinforcement Learning
• Interpreting the Mechanism of Synergism for Drug Combinations Using Attention-Based Hierarchical Graph Pooling
• Deep Representation Learning for Social Network Analysis
• Solve Traveling Salesman Problem by Monte Carlo Tree Search and Deep Neural Network
• Multi-scale Dynamic Graph Convolutional Network for Hyperspectral Image Classification
• Deep Graph Similarity Learning: A Survey
• A Deep Reinforcement Learning Algorithm Using Dynamic Attention Model for Vehicle Routing Problems
• Stability and Generalization of Graph Convolutional Neural Networks
• Memory-Based Graph Networks
• How to Evaluate Machine Learning Approaches for Combinatorial Optimization: Application to the Travelling Salesman Problem
• Learning Universal Graph Neural Network Embeddings With Aid Of Transfer Learning
• Can Graph Neural Networks Help Logic Reasoning?
• A Graph Neural Network Approach for Scalable Wireless Power Control
• Unsupervised Features Extraction for Binary Similarity Using Graph Embedding Neural Networks
• Graph Convolutional Networks with EigenPooling
• Zoo Guide to Network Embedding
• Learning Graph-Level Representations with Recurrent Neural Networks
• Enforcing Predictive Invariance across Structured Biomedical Domains
• Adversarial Attacks on Graph Classification via Bayesian Optimisation
• Deep Learning in Information Security
• Multi-Channel Graph Convolutional Networks
• RetCL: A Selection-based Approach for Retrosynthesis via Contrastive Learning
• Feature Propagation on Graph: A New Perspective to Graph Representation Learning
• Topology Aware Deep Learning for Wireless Network Optimization
• Amortized Bethe Free Energy Minimization for Learning MRFs
• Heterogeneous Graph Neural Networks for Malicious Account Detection
• Neural Message Passing for Multi-Label Classification
• GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategy
• InfDetect: a Large Scale Graph-based Fraud Detection System for E-Commerce Insurance
• MxPool: Multiplex Pooling for Hierarchical Graph Representation Learning
• Deep Hypergraph U-Net for Brain Graph Embedding and Classification
• DeepGG: a Deep Graph Generator
• Black-box Attacks Against Neural Binary Function Detection
• Graph Neural Networks for Scalable Radio Resource Management: Architecture Design and Theoretical Analysis
• Train on Small, Play the Large: Scaling Up Board Games with AlphaZero and GNN
• CommPOOL: An Interpretable Graph Pooling Framework for Hierarchical Graph Representation Learning
• Graph Polish: A Novel Graph Generation Paradigm for Molecular Optimization
• DPPIN: A Biological Repository of Dynamic Protein-Protein Interaction Network Data
• Graph Cross Networks with Vertex Infomax Pooling
• Co-training for Policy Learning
• Controlled Molecule Generator for Optimizing Multiple Chemical Properties
• CORE: Automatic Molecule Optimization Using Copy & Refine Strategy
• An In-depth Summary of Recent Artificial Intelligence Applications in Drug Design
• A Biased Graph Neural Network Sampler with Near-Optimal Regret
• Process-Level Representation of Scientific Protocols with Interactive Annotation
• Histopathology WSI Encoding based on GCNs for Scalable and Efficient Retrieval of Diagnostically Relevant Regions
• Active Screening for Recurrent Diseases: A Reinforcement Learning Approach
• Online Multi-Agent Forecasting with Interpretable Collaborative Graph Neural Network
• Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing
• Revocable Deep Reinforcement Learning with Affinity Regularization for Outlier-Robust Graph Matching
• Ego-CNN: Distributed, Egocentric Representations of Graphs for Detecting Critical Structures
• Learning Features of Network Structures Using Graphlets
• Sparse hierarchical representation learning on molecular graphs
• Unsupervised Hierarchical Graph Representation Learning by Mutual Information Maximization
• Inductive Subgraph Embedding for Link Prediction
• GIPA: A General Information Propagation Algorithm for Graph Learning
• Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back
• Semi-Supervised Graph Classification: A Hierarchical Graph Perspective
• Vulcan: Solving the Steiner Tree Problem with Graph Neural Networks and Deep Reinforcement Learning
• cube2net: Efficient Query-Specific Network Construction with Data Cube Organization
• Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation
• Graph Coarsening with Preserved Spectral Properties
• SAFE: Self-Attentive Function Embeddings for Binary Similarity
• Solving Graph-based Public Good Games with Tree Search and Imitation Learning
• OpenGraphGym-MG: Using Reinforcement Learning to Solve Large Graph Optimization Problems on MultiGPU Systems
• MSHCNet: Multi-Stream Hybridized Convolutional Networks with Mixed Statistics in Euclidean/Non-Euclidean Spaces and Its Application to Hyperspectral Image Classification
• SeaPearl: A Constraint Programming Solver guided by Reinforcement Learning
• Zero Training Overhead Portfolios for Learning to Solve Combinatorial Problems
• Beyond Observed Connections : Link Injection
• A General Framework for Charger Scheduling Optimization Problems
• Graph Convolutional Policy for Solving Tree Decomposition via Reinforcement Learning Heuristics
• MGA: Momentum Gradient Attack on Network
• Customized Graph Embedding: Tailoring Embedding Vectors to different Applications
• Accelerating Metropolis-Hastings with Lightweight Inference Compilation
• New Insights into Graph Convolutional Networks using Neural Tangent Kernels