Generating Long-term Trajectories Using Deep Hierarchical Networks

Abstract

We study the problem of modeling spatiotemporal trajectories over long time horizons using expert demonstrations. For instance, in sports, agents often choose action sequences with long-term goals in mind, such as achieving a certain strategic position. Conventional policy learning approaches, such as those based on Markov decision processes, generally fail at learning cohesive long-term behavior in such high-dimensional state spaces, and are only effective when myopic modeling lead to the desired behavior. The key difficulty is that conventional approaches are "shallow" models that only learn a single state-action policy. We instead propose a hierarchical policy class that automatically reasons about both long-term and short-term goals, which we instantiate as a hierarchical neural network. We showcase our approach in a case study on learning to imitate demonstrated basketball trajectories, and show that it generates significantly more realistic trajectories compared to non-hierarchical baselines as judged by professional sports analysts.

Published in NIPS 2016

Cited by in corpus (18)

• TNT: Target-driveN Trajectory Prediction
• A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions
• Data-driven Analysis for Understanding Team Sports Behaviors
• Deep Learning for Spatio-Temporal Data Mining: A Survey
• What-If Motion Prediction for Autonomous Driving
• Multi-agent Trajectory Prediction with Fuzzy Query Attention
• Fine-Grained Retrieval of Sports Plays using Tree-Based Alignment of Trajectories
• Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling
• Multiresolution Tensor Learning for Efficient and Interpretable Spatial Analysis
• baller2vec: A Multi-Entity Transformer For Multi-Agent Spatiotemporal Modeling
• Empowering A* Search Algorithms with Neural Networks for Personalized Route Recommendation
• A Graph Attention Based Approach for Trajectory Prediction in Multi-agent Sports Games
• About latent roles in forecasting players in team sports
• baller2vec++: A Look-Ahead Multi-Entity Transformer For Modeling Coordinated Agents
• Learning interaction rules from multi-animal trajectories via augmented behavioral models
• Improved Structural Discovery and Representation Learning of Multi-Agent Data
• Associative Embedding for Game-Agnostic Team Discrimination
• Learning Calibratable Policies using Programmatic Style-Consistency