Visual Causal Feature Learning

Abstract

We provide a rigorous definition of the visual cause of a behavior that is broadly applicable to the visually driven behavior in humans, animals, neurons, robots and other perceiving systems. Our framework generalizes standard accounts of causal learning to settings in which the causal variables need to be constructed from micro-variables. We prove the Causal Coarsening Theorem, which allows us to gain causal knowledge from observational data with minimal experimental effort. The theorem provides a connection to standard inference techniques in machine learning that identify features of an image that correlate with, but may not cause, the target behavior. Finally, we propose an active learning scheme to learn a manipulator function that performs optimal manipulations on the image to automatically identify the visual cause of a target behavior. We illustrate our inference and learning algorithms in experiments based on both synthetic and real data.

Accepted at UAI 2015

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Improving neural networks by preventing co-adaptation of feature detectors
• Theano: new features and speed improvements
• Computational Mechanics: Pattern and Prediction, Structure and Simplicity
• Strong Completeness and Faithfulness in Bayesian Networks
• Pylearn2: a machine learning research library
• Deep Visual-Semantic Alignments for Generating Image Descriptions

Cited by in corpus (13)

• Causality in cognitive neuroscience: concepts, challenges, and distributional robustness
• Causal Induction from Visual Observations for Goal Directed Tasks
• Representation Learning via Invariant Causal Mechanisms
• iPerceive: Applying Common-Sense Reasoning to Multi-Modal Dense Video Captioning and Video Question Answering
• Distilling Causal Effect of Data in Class-Incremental Learning
• Treatment Learning Causal Transformer for Noisy Image Classification
• Human Trajectory Prediction via Counterfactual Analysis
• Resolving Spurious Correlations in Causal Models of Environments via Interventions
• MERLiN: Mixture Effect Recovery in Linear Networks
• iReason: Multimodal Commonsense Reasoning using Videos and Natural Language with Interpretability
• Interventional Video Grounding with Dual Contrastive Learning
• Doubly Robust Off-Policy Learning on Low-Dimensional Manifolds by Deep Neural Networks
• The Blessings of Unlabeled Background in Untrimmed Videos