Scene-LSTM: A Model for Human Trajectory Prediction

Abstract

We develop a human movement trajectory prediction system that incorporates the scene information (Scene-LSTM) as well as human movement trajectories (Pedestrian movement LSTM) in the prediction process within static crowded scenes. We superimpose a two-level grid structure (scene is divided into grid cells each modeled by a scene-LSTM, which are further divided into smaller sub-grids for finer spatial granularity) and explore common human trajectories occurring in the grid cell (e.g., making a right or left turn onto sidewalks coming out of an alley; or standing still at bus/train stops). Two coupled LSTM networks, Pedestrian movement LSTMs (one per target) and the corresponding Scene-LSTMs (one per grid-cell) are trained simultaneously to predict the next movements. We show that such common path information greatly influences prediction of future movement. We further design a scene data filter that holds important non-linear movement information. The scene data filter allows us to select the relevant parts of the information from the grid cell's memory relative to a target's state. We evaluate and compare two versions of our method with the Linear and several existing LSTM-based methods on five crowded video sequences from the UCY [1] and ETH [2] datasets. The results show that our method reduces the location displacement errors compared to related methods and specifically about 80% reduction compared to social interaction methods.

arXiv comment9 pages, 5 figures

References in corpus (3)

• Sequence to Sequence Learning with Neural Networks
• DRAW: A Recurrent Neural Network For Image Generation
• Human Trajectory Prediction using Spatially aware Deep Attention Models

Cited by in corpus (24)

• Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction
• Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation
• VRUNet: Multi-Task Learning Model for Intent Prediction of Vulnerable Road Users
• Learning the Pedestrian-Vehicle Interaction for Pedestrian Trajectory Prediction
• The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
• TrajPRed: Trajectory Prediction with Region-based Relation Learning
• Scene Gated Social Graph: Pedestrian Trajectory Prediction Based on Dynamic Social Graphs and Scene Constraints
• Pedestrian Intention Prediction: A Multi-task Perspective
• exploRNN: Understanding Recurrent Neural Networks through Visual Exploration
• Reciprocal Learning Networks for Human Trajectory Prediction
• Lane Attention: Predicting Vehicles' Moving Trajectories by Learning Their Attention over Lanes
• View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrums
• Predicting Vehicles Trajectories in Urban Scenarios with Transformer Networks and Augmented Information
• CANet: A Context-Aware Network for Shadow Removal
• MSN: Multi-Style Network for Trajectory Prediction
• Object Tracking Using Spatio-Temporal Future Prediction
• BGM: Building a Dynamic Guidance Map without Visual Images for Trajectory Prediction
• Dynamic and Static Context-aware LSTM for Multi-agent Motion Prediction
• Visual-information-driven model for crowd simulation using temporal convolutional network
• From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video
• Res-GCNN: A Lightweight Residual Graph Convolutional Neural Networks for Human Trajectory Forecasting
• A Spatial-Temporal Attentive Network with Spatial Continuity for Trajectory Prediction
• MCENET: Multi-Context Encoder Network for Homogeneous Agent Trajectory Prediction in Mixed Traffic