Generating Long-term Trajectories Using Deep Hierarchical Networks

Abstract

We study the problem of modeling spatiotemporal trajectories over long time horizons using expert demonstrations. For instance, in sports, agents often choose action sequences with long-term goals in mind, such as achieving a certain strategic position. Conventional policy learning approaches, such as those based on Markov decision processes, generally fail at learning cohesive long-term behavior in such high-dimensional state spaces, and are only effective when myopic modeling lead to the desired behavior. The key difficulty is that conventional approaches are "shallow" models that only learn a single state-action policy. We instead propose a hierarchical policy class that automatically reasons about both long-term and short-term goals, which we instantiate as a hierarchical neural network. We showcase our approach in a case study on learning to imitate demonstrated basketball trajectories, and show that it generates significantly more realistic trajectories compared to non-hierarchical baselines as judged by professional sports analysts.

Published in NIPS 2016

Cited by in corpus (11)

• TNT: Target-driveN Trajectory Prediction
• A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions
• Deep Learning for Spatio-Temporal Data Mining: A Survey
• What-If Motion Prediction for Autonomous Driving
• Multi-agent Trajectory Prediction with Fuzzy Query Attention
• Fine-Grained Retrieval of Sports Plays using Tree-Based Alignment of Trajectories
• Empowering A* Search Algorithms with Neural Networks for Personalized Route Recommendation
• A Graph Attention Based Approach for Trajectory Prediction in Multi-agent Sports Games
• Improved Structural Discovery and Representation Learning of Multi-Agent Data
• Learning interaction rules from multi-animal trajectories via augmented behavioral models
• Associative Embedding for Game-Agnostic Team Discrimination