Improving the Evaluation of Generative Models with Fuzzy Logic
arXiv:2002.03772
Abstract
Objective and interpretable metrics to evaluate current artificial intelligent systems are of great importance, not only to analyze the current state of such systems but also to objectively measure progress in the future. In this work, we focus on the evaluation of image generation tasks. We propose a novel approach, called Fuzzy Topology Impact (FTI), that determines both the quality and diversity of an image set using topology representations combined with fuzzy logic. When compared to current evaluation methods, FTI shows better and more stable performance on multiple experiments evaluating the sensitivity to noise, mode dropping and mode inventing.
arXiv commentAAAI 2020 Meta-Eval
References in corpus (7)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- A note on the evaluation of generative models
- Assessing Generative Models via Precision and Recall
- Geometry Score: A Method For Comparing Generative Adversarial Networks