Extract-Transform-Load for Video Streams

Abstract

Social media, self-driving cars, and traffic cameras produce video streams at large scales and cheap cost. However, storing and querying video at such scales is prohibitively expensive. We propose to treat large-scale video analytics as a data warehousing problem: Video is a format that is easy to produce but needs to be transformed into an application-specific format that is easy to query. Analogously, we define the problem of Video Extract-Transform-Load (V-ETL). V-ETL systems need to reduce the cost of running a user-defined V-ETL job while also giving throughput guarantees to keep up with the rate at which data is produced. We find that no current system sufficiently fulfills both needs and therefore propose Skyscraper, a system tailored to V-ETL. Skyscraper can execute arbitrary video ingestion pipelines and adaptively tunes them to reduce cost at minimal or no quality degradation, e.g., by adjusting sampling rates and resolutions to the ingested content. Skyscraper can hereby be provisioned with cheap on-premises compute and uses a combination of buffering and cloud bursting to deal with peaks in workload caused by expensive processing configurations. In our experiments, we find that Skyscraper significantly reduces the cost of V-ETL ingestion compared to adaptions of current SOTA systems, while at the same time giving robustness guarantees that these systems are lacking.

arXiv comment26 pages, 23 figures

References in corpus (19)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Distilling the Knowledge in a Neural Network
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• High-Speed Tracking with Kernelized Correlation Filters
• MOT20: A benchmark for multi object tracking in crowded scenes
• Device Placement Optimization with Reinforcement Learning
• Focus: Querying Large Video Datasets with Low Latency and Low Cost
• Evolving Boxes for Fast Vehicle Detection
• VStore: A Data Store for Analytics on Large Videos
• NoScope: Optimizing Neural Network Queries over Video at Scale
• Physical Representation-based Predicate Optimization for a Visual Analytics Database
• Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning
• BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics
• Accelerating Approximate Aggregation Queries with Expensive Predicates
• Zeus: Efficiently Localizing Actions in Videos using Reinforcement Learning
• THIA: Accelerating Video Analytics using Early Inference and Fine-Grained Query Planning
• TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking
• EKO: Adaptive Sampling of Compressed Video Data

Cited by in corpus (3)

• iRAG: Advancing RAG for Videos with an Incremental Approach
• Blueprinting the Cloud: Unifying and Automatically Optimizing Cloud Data Infrastructures with BRAD -- Extended Version
• TrafficLens: Multi-Camera Traffic Video Analysis Using LLMs