From Scene-Centric to Observer-Centric: Modeling Observer-Aware Relations for 3D Scene Graph Generation

Abstract

3D Scene Graph Generation (3DSGG) represents 3D scenes as structured object--relation--object graphs for spatial understanding. In observer-centric spatial perception, the same scene may be expressed under different local observer frames while its structure remains unchanged. However, existing models typically assume a fixed scene-aligned reference frame and may produce semantically inconsistent predictions when the scene is re-expressed in another observer frame. We attribute this failure to the heterogeneous frame dependency of relational predicates. Directional predicates such as $\text{left}$, $\text{front}$, $\text{right}$, and $\text{behind}$ are $\text{Observer-Dependent Relations}$, whereas most contact, support, and semantic predicates, such as $\text{standing on}$ and $\text{attached to}$, are approximately $\text{Observer-Independent Relations}$. Conventional models do not distinguish these frame responses, leading to degraded relation prediction under observer-frame reorientation. We introduce $\text{Observer-Aware Relations (OAR)}$, which combines observer-aware geometric encoding and relation specialization, supported by frame-stable object encoding, for unified multi-label predicate prediction. Experiments on 3DSSG show that OAR consistently outperforms baselines across controlled observer-frame reorientations without training-time frame-reorientation augmentation, while remaining competitive on the standard benchmark. The project page is available at https://oar-predicate.github.io/.