Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)

Abstract

I review unsupervised or self-supervised neural networks playing minimax games in game-theoretic settings: (i) Artificial Curiosity (AC, 1990) is based on two such networks. One network learns to generate a probability distribution over outputs, the other learns to predict effects of the outputs. Each network minimizes the objective function maximized by the other. (ii) Generative Adversarial Networks (GANs, 2010-2014) are an application of AC where the effect of an output is 1 if the output is in a given set, and 0 otherwise. (iii) Predictability Minimization (PM, 1990s) models data distributions through a neural encoder that maximizes the objective function minimized by a neural predictor of the code components. I correct a previously published claim that PM is not based on a minimax game.

15 pages, 1 figure, 104 references

References in corpus (11)

• Deep Learning in Neural Networks: An Overview
• Conditional Generative Adversarial Nets
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Domain Separation Networks
• A Neural Representation of Sketch Drawings
• Synthesizing Programs for Images using Reinforced Adversarial Learning
• Connecting Generative Adversarial Networks and Actor-Critic Methods
• On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games
• Learning to Paint With Model-based Deep Reinforcement Learning
• Neural Painters: A learned differentiable constraint for generating brushstroke paintings
• One Big Net For Everything

Cited by in corpus (3)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Investigating Object Compositionality in Generative Adversarial Networks
• MIME: Mutual Information Minimisation Exploration