Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training

Abstract

Video quality assessment (VQA) is an important problem in computer vision. The videos in computer vision applications are usually captured in the wild. We focus on automatically assessing the quality of in-the-wild videos, which is a challenging problem due to the absence of reference videos, the complexity of distortions, and the diversity of video contents. Moreover, the video contents and distortions among existing datasets are quite different, which leads to poor performance of data-driven methods in the cross-dataset evaluation setting. To improve the performance of quality assessment models, we borrow intuitions from human perception, specifically, content dependency and temporal-memory effects of human visual system. To face the cross-dataset evaluation challenge, we explore a mixed datasets training strategy for training a single VQA model with multiple datasets. The proposed unified framework explicitly includes three stages: relative quality assessor, nonlinear mapping, and dataset-specific perceptual scale alignment, to jointly predict relative quality, perceptual quality, and subjective quality. Experiments are conducted on four publicly available datasets for VQA in the wild, i.e., LIVE-VQC, LIVE-Qualcomm, KoNViD-1k, and CVD2014. The experimental results verify the effectiveness of the mixed datasets training strategy and prove the superior performance of the unified model in comparison with the state-of-the-art models. For reproducible research, we make the PyTorch implementation of our method available at https://github.com/lidq92/MDTVSFA.

20 pages, 12 figures, 7 tables, accepted by IJCV. This is the version provided to IJCV office

References in corpus (3)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment

Cited by in corpus (11)

• A Deep Learning based No-reference Quality Assessment Model for UGC Videos
• Blind Quality Assessment for in-the-Wild Images via Hierarchical Feature Fusion and Iterative Mixed Database Training
• Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach
• ChipQA: No-Reference Video Quality Prediction via Space-Time Chips
• No-Reference Image Quality Assessment by Hallucinating Pristine Features
• RankDVQA: Deep VQA based on Ranking-inspired Hybrid Training
• A Generalised Deep Meta-Learning Model for Automated Quality Control of Cardiovascular Magnetic Resonance Images
• StarVQA: Space-Time Attention for Video Quality Assessment
• Efficient Bitrate Ladder Construction using Transfer Learning and Spatio-Temporal Features
• Highly Efficient No-reference 4K Video Quality Assessment with Full-Pixel Covering Sampling and Training Strategy
• Subjective and Objective Quality Assessment of Banding Artifacts on Compressed Videos