BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics

Abstract

Recent advances in neural networks (NNs) have enabled automatic querying of large volumes of video data with high accuracy. While these deep NNs can produce accurate annotations of an object's position and type in video, they are computationally expensive and require complex, imperative deployment code to answer queries. Prior work uses approximate filtering to reduce the cost of video analytics, but does not handle two important classes of queries, aggregation and limit queries; moreover, these approaches still require complex code to deploy. To address the computational and usability challenges of querying video at scale, we introduce BlazeIt, a system that optimizes queries of spatiotemporal information of objects in video. BlazeIt accepts queries via FrameQL, a declarative extension of SQL for video analytics that enables video-specific query optimization. We introduce two new query optimization techniques in BlazeIt that are not supported by prior work. First, we develop methods of using NNs as control variates to quickly answer approximate aggregation queries with error bounds. Second, we present a novel search algorithm for cardinality-limited video queries. Through these these optimizations, BlazeIt can deliver up to 83x speedups over the recent literature on video processing.

References in corpus (4)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Distilling the Knowledge in a Neural Network
• Compressing Neural Networks with the Hashing Trick
• Scanner: Efficient Video Analysis at Scale

Cited by in corpus (13)

• Video Big Data Analytics in the Cloud: A Reference Architecture, Survey, Opportunities, and Open Research Issues
• VidCEP: Complex Event Processing Framework to Detect Spatiotemporal Patterns in Video Streams
• VID-WIN: Fast Video Event Matching with Query-Aware Windowing at the Edge for the Internet of Multimedia Things
• Rekall: Specifying Video Events using Compositions of Spatiotemporal Labels
• Extract-Transform-Load for Video Streams
• DeepLens: Towards a Visual Data Management System
• Zeus: Efficiently Localizing Actions in Videos using Reinforcement Learning
• OneAdapt: Fast Configuration Adaptation for Video Analytics Applications via Backpropagation
• SurveilEdge: Real-time Video Query based on Collaborative Cloud-Edge Deep Learning
• Robust, Extensible, and Fast: Teamed Classifiers for Vehicle Tracking and Vehicle Re-ID in Multi-Camera Networks
• EKO: Adaptive Sampling of Compressed Video Data
• Pool of Experts: Realtime Querying Specialized Knowledge in Massive Neural Networks
• A Survey of Performance Optimization in Neural Network-Based Video Analytics Systems