Are Disentangled Representations Helpful for Abstract Visual Reasoning?

Abstract

A disentangled representation encodes information about the salient factors of variation in the data independently. Although it is often argued that this representational format is useful in learning to solve many real-world down-stream tasks, there is little empirical evidence that supports this claim. In this paper, we conduct a large-scale study that investigates whether disentangled representations are more suitable for abstract reasoning tasks. Using two new tasks similar to Raven's Progressive Matrices, we evaluate the usefulness of the representations learned by 360 state-of-the-art unsupervised disentanglement models. Based on these representations, we train 3600 abstract reasoning models and observe that disentangled representations do in fact lead to better down-stream performance. In particular, they enable quicker learning using fewer samples.

arXiv commentAccepted to NeurIPS 2019

References in corpus (34)

• Deep Learning in Neural Networks: An Overview
• Auto-Encoding Variational Bayes
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Deep Convolutional Inverse Graphics Network
• Disentangling by Factorising
• Recent Advances in Autoencoder-Based Representation Learning
• Towards a Definition of Disentangled Representations
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Understanding disentangling in $\beta$-VAE
• Disentangling factors of variation in deep representations using adversarial training
• Multi-Object Representation Learning with Iterative Variational Inference
• Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
• Measuring abstract reasoning in neural networks
• Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
• Discovering Hidden Factors of Variation in Deep Networks
• Learning Deep Disentangled Embeddings with the F-Statistic Loss
• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• Structured Disentangled Representations
• Information Constraints on Auto-Encoding Variational Bayes
• Disentangled Sequential Autoencoder
• SCAN: Learning Hierarchical Compositional Visual Concepts
• Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
• SOM-VAE: Interpretable Discrete Representation Learning on Time Series
• Discovering objects and their relations from entangled scene representations
• Learning Latent Subspaces in Variational Autoencoders
• Bayesian representation learning with oracle constraints
• Learning the Irreducible Representations of Commutative Lie Groups
• Disentangling Factors of Variation Using Few Labels
• Learning to Make Analogies by Contrasting Abstract Relational Structure
• Improving Generalization for Abstract Reasoning Tasks Using Disentangled Feature Representations
• Understanding Visual Concepts with Continuation Learning
• Learning to Linearize Under Uncertainty
• Learning Disentangled Representations with Reference-Based Variational Autoencoders
• Competitive Training of Mixtures of Independent Deep Generative Models

Cited by in corpus (28)

• A Review on Deep Learning Techniques for Video Prediction
• GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
• Weakly-Supervised Disentanglement Without Compromises
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Disentangling Factors of Variation Using Few Labels
• On Disentangled Representations Learned From Correlated Data
• Unsupervised Model Selection for Variational Disentangled Representation Learning
• A multimodal dynamical variational autoencoder for audiovisual speech representation learning
• q-VAE for Disentangled Representation Learning and Latent Dynamical Systems
• Symmetry-Based Disentangled Representation Learning requires Interaction with Environments
• Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity
• Learning Group Structure and Disentangled Representations of Dynamical Environments
• On the Transfer of Disentangled Representations in Realistic Settings
• On the Fairness of Disentangled Representations
• Generalization and Robustness Implications in Object-Centric Learning
• Adversarial Graph Disentanglement
• Data-efficient visuomotor policy training using reinforcement learning and generative models
• Weakly-Supervised Reinforcement Learning for Controllable Behavior
• Better Set Representations For Relational Reasoning
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling
• Random Polytope Descriptors
• Modeling Gestalt Visual Reasoning on the Raven's Progressive Matrices Intelligence Test Using Generative Image Inpainting Techniques
• DAReN: A Collaborative Approach Towards Reasoning And Disentangling
• Be More Active! Understanding the Differences between Mean and Sampled Representations of Variational Autoencoders
• Boxhead: A Dataset for Learning Hierarchical Representations
• A Feature-based Generalizable Prediction Model for Both Perceptual and Abstract Reasoning
• Constellation: Learning relational abstractions over objects for compositional imagination
• Adversarial Canonical Correlation Analysis