Learning Temporal Regularity in Video Sequences

Abstract

Perceiving meaningful activities in a long video sequence is a challenging problem due to ambiguous definition of 'meaningfulness' as well as clutters in the scene. We approach this problem by learning a generative model for regular motion patterns, termed as regularity, using multiple sources with very limited supervision. Specifically, we propose two methods that are built upon the autoencoders for their ability to work with little to no supervision. We first leverage the conventional handcrafted spatio-temporal local features and learn a fully connected autoencoder on them. Second, we build a fully convolutional feed-forward autoencoder to learn both the local features and the classifiers as an end-to-end learning framework. Our model can capture the regularities from multiple datasets. We evaluate our methods in both qualitative and quantitative ways - showing the learned regularity of videos in various aspects and demonstrating competitive performance on anomaly detection datasets as an application.

CVPR 2016

Cited by in corpus (11)

• Generative Probabilistic Novelty Detection with Adversarial Autoencoders
• Future Frame Prediction for Anomaly Detection -- A New Baseline
• Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language
• Image/Video Deep Anomaly Detection: A Survey
• Training Adversarial Discriminators for Cross-channel Abnormal Event Detection in Crowds
• Energy-based Models for Video Anomaly Detection
• Temporal Human Action Segmentation via Dynamic Clustering
• Video Anomaly Detection By The Duality Of Normality-Granted Optical Flow
• Detecting abnormal events in video using Narrowed Normality Clusters
• 3D-MOV: Audio-Visual LSTM Autoencoder for 3D Reconstruction of Multiple Objects from Video
• Discovery and recognition of motion primitives in human activities