SimAug: Learning Robust Representations from Simulation for Trajectory Prediction

Abstract

This paper studies the problem of predicting future trajectories of people in unseen cameras of novel scenarios and views. We approach this problem through the real-data-free setting in which the model is trained only on 3D simulation data and applied out-of-the-box to a wide variety of real cameras. We propose a novel approach to learn robust representation through augmenting the simulation training data such that the representation can better generalize to unseen real-world test data. The key idea is to mix the feature of the hardest camera view with the adversarial feature of the original view. We refer to our method as SimAug. We show that SimAug achieves promising results on three real-world benchmarks using zero real training data, and state-of-the-art performance in the Stanford Drone and the VIRAT/ActEV dataset when using in-domain training data.

arXiv commentAccepted by ECCV 2020. Project website: https://next.cs.cmu.edu/simaug

References in corpus (11)

• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• ADADELTA: An Adaptive Learning Rate Method
• mixup: Beyond Empirical Risk Minimization
• Ensemble Adversarial Training: Attacks and Defenses
• Learning to Reweight Examples for Robust Deep Learning
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction
• Argoverse: 3D Tracking and Forecasting with Rich Maps
• Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels
• Focal Visual-Text Attention for Visual Question Answering
• RSA: Randomized Simulation as Augmentation for Robust Human Action Recognition

Cited by in corpus (7)

• Stepwise Goal-Driven Networks for Trajectory Prediction
• Regularizing Generative Adversarial Networks under Limited Data
• Faster Meta Update Strategy for Noise-Robust Deep Learning
• Human Trajectory Prediction via Counterfactual Analysis
• View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrums
• Spectral Temporal Graph Neural Network for Trajectory Prediction
• MSN: Multi-Style Network for Trajectory Prediction