A spin-glass model for the loss surfaces of generative adversarial networks

Abstract

We present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model's critical points using techniques from Random Matrix Theory. The result is insights into the loss surfaces of large GANs that build upon prior insights for simpler networks, but also reveal new structure unique to this setting.

26 pages, 9 figures

References in corpus (3)

• Conditional Generative Adversarial Nets
• The Loss Surfaces of Multilayer Networks
• Unveiling the structure of wide flat minima in neural networks

Cited by in corpus (5)

• Exponential growth of random determinants beyond invariance
• Universal characteristics of deep neural network loss surfaces from random matrix theory
• Local convexity of the TAP free energy and AMP convergence for Z2-synchronization
• Appearance of Random Matrix Theory in Deep Learning
• Landscape Complexity for the Empirical Risk of Generalized Linear Models: Discrimination between Structured Data