An Unsupervised Information-Theoretic Perceptual Quality Metric

Abstract

Tractable models of human perception have proved to be challenging to build. Hand-designed models such as MS-SSIM remain popular predictors of human image quality judgements due to their simplicity and speed. Recent modern deep learning approaches can perform better, but they rely on supervised data which can be costly to gather: large sets of class labels such as ImageNet, image quality ratings, or both. We combine recent advances in information-theoretic objective functions with a computational architecture informed by the physiology of the human visual system and unsupervised training on pairs of video frames, yielding our Perceptual Information Metric (PIM). We show that PIM is competitive with supervised metrics on the recent and challenging BAPPS image quality assessment dataset and outperforms them in predicting the ranking of image compression methods in CLIC 2020. We also perform qualitative experiments using the ImageNet-C dataset, and establish that PIM is robust with respect to architectural details.

arXiv comment19 pages, 10 figures. Presented at NeurIPS 2020. Code available at https://github.com/google-research/perceptual-quality

References in corpus (6)

• Representation Learning with Contrastive Predictive Coding
• Image Quality Assessment: Unifying Structure and Texture Similarity
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• The Conditional Entropy Bottleneck
• Eigen-Distortions of Hierarchical Representations
• Towards a Semantic Perceptual Image Metric

Cited by in corpus (2)

• DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation
• Human Imperceptible Attacks and Applications to Improve Fairness