KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment

Abstract

Deep learning methods for image quality assessment (IQA) are limited due to the small size of existing datasets. Extensive datasets require substantial resources both for generating publishable content and annotating it accurately. We present a systematic and scalable approach to creating KonIQ-10k, the largest IQA dataset to date, consisting of 10,073 quality scored images. It is the first in-the-wild database aiming for ecological validity, concerning the authenticity of distortions, the diversity of content, and quality-related indicators. Through the use of crowdsourcing, we obtained 1.2 million reliable quality ratings from 1,459 crowd workers, paving the way for more general IQA models. We propose a novel, deep learning model (KonCept512), to show an excellent generalization beyond the test set (0.921 SROCC), to the current state-of-the-art database LIVE-in-the-Wild (0.825 SROCC). The model derives its core performance from the InceptionResNet architecture, being trained at a higher resolution than previous models (512x384). Correlation analysis shows that KonCept512 performs similar to having 9 subjective scores for each test image.

arXiv commentPublished

References in corpus (5)

• YFCC100M: The New Data in Multimedia Research
• NIMA: Neural Image Assessment
• Massive Online Crowdsourced Study of Subjective and Objective Picture Quality
• On the Use of Deep Learning for Blind Image Quality Assessment
• MS-UNIQUE: Multi-model and Sharpness-weighted Unsupervised Image Quality Estimation

Cited by in corpus (19)

• Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild
• UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content
• Image Quality Assessment using Contrastive Learning
• A Deep Learning based No-reference Quality Assessment Model for UGC Videos
• RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content
• Blind Quality Assessment for in-the-Wild Images via Hierarchical Feature Fusion and Iterative Mixed Database Training
• Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment
• Confusing Image Quality Assessment: Towards Better Augmented Reality Experience
• Large Multi-modality Model Assisted AI-Generated Image Quality Assessment
• SUR-FeatNet: Predicting the Satisfied User Ratio Curvefor Image Compression with Deep Feature Learning
• Gap-closing Matters: Perceptual Quality Evaluation and Optimization of Low-Light Image Enhancement
• Consolidated Dataset and Metrics for High-Dynamic-Range Image Quality
• Unveiling The Factors of Aesthetic Preferences with Explainable AI
• MS-SCANet: A Multiscale Transformer-Based Architecture with Dual Attention for No-Reference Image Quality Assessment
• Generalised Score Distribution: A Two-Parameter Discrete Distribution Accurately Describing Responses from Quality of Experience Subjective Experiments
• Fairness in generative modeling
• Building Reasonable Inference for Vision-Language Models in Blind Image Quality Assessment
• MMP-2K: A Benchmark Multi-Labeled Macro Photography Image Quality Assessment Database
• Fill2SR: Repurposing Inpainting Diffusion Transformers for Real-World Super-Resolution