Synthetic-Neuroscore: Using A Neuro-AI Interface for Evaluating Generative Adversarial Networks

Abstract

Generative adversarial networks (GANs) are increasingly attracting attention in the computer vision, natural language processing, speech synthesis and similar domains. Arguably the most striking results have been in the area of image synthesis. However, evaluating the performance of GANs is still an open and challenging problem. Existing evaluation metrics primarily measure the dissimilarity between real and generated images using automated statistical methods. They often require large sample sizes for evaluation and do not directly reflect human perception of image quality. In this work, we describe an evaluation metric we call Neuroscore, for evaluating the performance of GANs, that more directly reflects psychoperceptual image quality through the utilization of brain signals. Our results show that Neuroscore has superior performance to the current evaluation metrics in that: (1) It is more consistent with human judgment; (2) The evaluation process needs much smaller numbers of samples; and (3) It is able to rank the quality of images on a per GAN basis. A convolutional neural network (CNN) based neuro-AI interface is proposed to predict Neuroscore from GAN-generated images directly without the need for neural responses. Importantly, we show that including neural responses during the training phase of the network can significantly improve the prediction capability of the proposed model. Materials related to this work are provided at https://github.com/villawang/Neuro-AI-Interface.

References in corpus (9)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Understanding Black-box Predictions via Influence Functions
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Generative Moment Matching Networks
• An empirical study on evaluation metrics of generative adversarial networks
• A Note on the Inception Score
• Quick and Easy Time Series Generation with Established Image-based GANs
• Spatial Filtering Pipeline Evaluation of Cortically Coupled Computer Vision System for Rapid Serial Visual Presentation

Cited by in corpus (5)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning
• IROS 2019 Lifelong Robotic Vision Challenge -- Lifelong Object Recognition Report
• CatNet: Class Incremental 3D ConvNets for Lifelong Egocentric Gesture Recognition
• A Neuro-AI Interface for Evaluating Generative Adversarial Networks