Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective
arXiv:2111.14820
Abstract
Learning behavioral patterns from observational data has been a de-facto approach to motion forecasting. Yet, the current paradigm suffers from two shortcomings: brittle under distribution shifts and inefficient for knowledge transfer. In this work, we propose to address these challenges from a causal representation perspective. We first introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables, namely invariant variables, style confounders, and spurious features. We then introduce a learning framework that treats each group separately: (i) unlike the common practice mixing datasets collected from different locations, we exploit their subtle distinctions by means of an invariance loss encouraging the model to suppress spurious correlations; (ii) we devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a sparse causal graph; (iii) we introduce a style contrastive loss that not only enforces the structure of style representations but also serves as a self-supervisory signal for test-time refinement on the fly. Experiments on synthetic and real datasets show that our proposed method improves the robustness and reusability of learned motion representations, significantly outperforming prior state-of-the-art motion forecasting models for out-of-distribution generalization and low-shot transfer.
arXiv commentCVPR 2022. Code is available at https://github.com/vita-epfl/causalmotion. v4: fixed typo
References in corpus (12)
- Social Force Model for Pedestrian Dynamics
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Invariant Risk Minimization
- Towards a Definition of Disentangled Representations
- Learning Independent Causal Mechanisms
- An Investigation of Why Overparameterization Exacerbates Spurious Correlations
- Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond
- Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changes
- Human Trajectory Prediction via Counterfactual Analysis
- CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
- You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction