Deep Sequence Learning for Video Anticipation: From Discrete and Deterministic to Continuous and Stochastic

Abstract

Video anticipation is the task of predicting one/multiple future representation(s) given limited, partial observation. This is a challenging task due to the fact that given limited observation, the future representation can be highly ambiguous. Based on the nature of the task, video anticipation can be considered from two viewpoints: the level of details and the level of determinism in the predicted future. In this research, we start from anticipating a coarse representation of a deterministic future and then move towards predicting continuous and fine-grained future representations of a stochastic process. The example of the former is video action anticipation in which we are interested in predicting one action label given a partially observed video and the example of the latter is forecasting multiple diverse continuations of human motion given partially observed one. In particular, in this thesis, we make several contributions to the literature of video anticipation...

arXiv commentThe draft of my PhD thesis

References in corpus (16)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• On the difficulty of training Recurrent Neural Networks
• Neural Discrete Representation Learning
• Unsupervised Learning of Video Representations using LSTMs
• InfoVAE: Information Maximizing Variational Autoencoders
• Deep Image: Scaling up Image Recognition
• QuaterNet: A Quaternion-based Recurrent Model for Human Motion
• Diversity-Sensitive Conditional Generative Adversarial Networks
• Diverse Trajectory Forecasting with Determinantal Point Processes
• Driver Action Prediction Using Deep (Bidirectional) Recurrent Neural Network
• Mixture Models for Diverse Machine Translation: Tricks of the Trade
• Human Motion Modeling using DVGANs
• Incorporating Network Built-in Priors in Weakly-supervised Semantic Segmentation
• A Survey on Document-level Neural Machine Translation: Methods and Evaluation
• ArTIST: Autoregressive Trajectory Inpainting and Scoring for Tracking