Learning Generalized Spatial-Temporal Deep Feature Representation for No-Reference Video Quality Assessment

Abstract

In this work, we propose a no-reference video quality assessment method, aiming to achieve high-generalization capability in cross-content, -resolution and -frame rate quality prediction. In particular, we evaluate the quality of a video by learning effective feature representations in spatial-temporal domain. In the spatial domain, to tackle the resolution and content variations, we impose the Gaussian distribution constraints on the quality features. The unified distribution can significantly reduce the domain gap between different video samples, resulting in a more generalized quality feature representation. Along the temporal dimension, inspired by the mechanism of visual perception, we propose a pyramid temporal aggregation module by involving the short-term and long-term memory to aggregate the frame-level quality. Experiments show that our method outperforms the state-of-the-art methods on cross-dataset settings, and achieves comparable performance on intra-dataset configurations, demonstrating the high-generalization capability of the proposed method.

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• YFCC100M: The New Data in Multimedia Research
• Image Quality Assessment: Unifying Structure and Texture Similarity
• Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network
• Domain Generalization via Invariant Feature Representation
• Adversarially Robust Generalization Requires More Data
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Large-Scale Study of Perceptual Video Quality
• Heterogeneous Domain Generalization via Domain Mixup
• RAN4IQA: Restorative Adversarial Nets for No-Reference Image Quality Assessment