Differentiable Grammars for Videos
arXiv:1902.00505
Abstract
This paper proposes a novel algorithm which learns a formal regular grammar from real-world continuous data, such as videos. Learning latent terminals, non-terminals, and production rules directly from continuous data allows the construction of a generative model capturing sequential structures with multiple possibilities. Our model is fully differentiable, and provides easily interpretable results which are important in order to understand the learned structures. It outperforms the state-of-the-art on several challenging datasets and is more accurate for forecasting future activities in videos. We plan to open-source the code. https://sites.google.com/view/differentiable-grammars
References in corpus (7)
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Categorical Reparameterization with Gumbel-Softmax
- Towards A Rigorous Science of Interpretable Machine Learning
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- Predictive-Corrective Networks for Action Detection
- Asynchronous Temporal Fields for Action Recognition