Schrödinger's Cat: Probabilistic Representation and Prediction of Potential Scene Kinematics
arXiv:2607.25984
The paper introduces GARFIELD, a probabilistic model that learns a structured spatio‑temporal latent representation of possible future scene motions from a single image and optional constraints, enabling fast joint sampling of trajectories and direct density estimation for uncertainty‑aware planning.
Abstract
Predicting how a scene may evolve from partial observations requires reasoning about multiple possible futures rather than committing to a single trajectory. Existing approaches either generate appearance-dominated video predictions or sample a small number of trajectories without explicitly modeling the distribution of possible motion. We introduce Goal-Aware Representations of Future kInEmatic Latent Distributions (GARFIELD), a probabilistic model of scene kinematics that learns a structured spatio-temporal latent representation of the distribution over possible futures given an image and optional spatio-temporally sparse constraints. The same latent representation enables both joint sampling of all trajectories and direct access to the underlying motion distribution through an efficient deterministic density decoder. As a result, uncertainty about future motion can be localized to specific scene elements and timesteps and progressively refined through additional constraints. Experiments demonstrate strong motion planning performance competitive with large video generation models while sampling trajectories faster. Our method further estimates motion densities two orders of magnitude faster than Monte-Carlo sampling from motion generation models, enabling interactive exploration and uncertainty-aware planning.
Accepted at ECCV 2026. Project page: https://compvis.github.io/schroedingers_cat