ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Models

Abstract

Reference-image-conditioned video generators are increasingly used as neural simulators for embodied planning and policy learning, yet their ability to anticipate physical hazards is rarely evaluated. We introduce ICAT, which grounds test construction in incident reports and safety guidance through structured risk memories, constrained case generation, and human verification. The resulting benchmark contains 909 evaluation units with explicit initial conditions, triggering actions, and documented consequences. Across six generation systems, severe outcomes are frequently missing or attenuated even when the initial state is preserved and the risky action is executed: the strongest system achieves 23.5% conditional strict success under best-of-eight selection. Expert auditing, image-generator sensitivity analysis, matched safe-risky pairs, and alternative-outcome scoring provide complementary checks on this system-level finding. ICAT exposes a gap between reproducing risky actions and generating their documented consequences.

arXiv commentAccepted to NeurIPS 2026 (Poster)