Relate and Predict: Structure-Aware Prediction with Jointly Optimized Neural DAG

Abstract

Understanding relationships between feature variables is one important way humans use to make decisions. However, state-of-the-art deep learning studies either focus on task-agnostic statistical dependency learning or do not model explicit feature dependencies during prediction. We propose a deep neural network framework, dGAP, to learn neural dependency Graph and optimize structure-Aware target Prediction simultaneously. dGAP trains towards a structure self-supervision loss and a target prediction loss jointly. Our method leads to an interpretable model that can disentangle sparse feature relationships, informing the user how relevant dependencies impact the target task. We empirically evaluate dGAP on multiple simulated and real datasets. dGAP is not only more accurate, but can also recover correct dependency structure.

arXiv comment8 pages, 6 figures, version appeared in ICML Workshop 2020 Graph Representation Learning and Beyond (GRL+)

References in corpus (11)

• Graph Neural Networks: A Review of Methods and Applications
• Consistent Individualized Feature Attribution for Tree Ensembles
• Neural Relational Inference for Interacting Systems
• DAGs with NO TEARS: Continuous Optimization for Structure Learning
• Learning Discrete Structures for Graph Neural Networks
• Learning Neural Causal Models from Unknown Interventions
• Detecting Statistical Interactions from Neural Network Weights
• Learning Sparse Nonparametric DAGs
• Learning Global Pairwise Interactions with Bayesian Neural Networks
• Feature Interaction Interpretability: A Case for Explaining Ad-Recommendation Systems via Neural Interaction Detection
• Gradient-Based Neural DAG Learning