Efficient Bitrate Ladder Construction using Transfer Learning and Spatio-Temporal Features

Abstract

Providing high-quality video with efficient bitrate is a main challenge in video industry. The traditional one-size-fits-all scheme for bitrate ladders is inefficient and reaching the best content-aware decision computationally impractical due to extensive encodings required. To mitigate this, we propose a bitrate and complexity efficient bitrate ladder prediction method using transfer learning and spatio-temporal features. We propose: (1) using feature maps from well-known pre-trained DNNs to predict rate-quality behavior with limited training data; and (2) improving highest quality rung efficiency by predicting minimum bitrate for top quality and using it for the top rung. The method tested on 102 video scenes demonstrates 94.1% reduction in complexity versus brute-force at 1.71% BD-Rate expense. Additionally, transfer learning was thoroughly studied through four networks and ablation studies.

arXiv comment7 pages, 9 figures, 7 tables, Copyright 2024 IEEE - Presented in IEEE MVIP 2024

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• The Kinetics Human Action Video Dataset
• Quality Assessment of In-the-Wild Videos
• A Deep Learning based No-reference Quality Assessment Model for UGC Videos
• Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training
• Complexity Analysis Of Next-Generation VVC Encoding and Decoding
• Convex Hull Prediction for Adaptive Video Streaming by Recurrent Learning