Can VLMs Predict Future States? Bootstrapping World Models from Inverse Dynamics

Yifu Qiu, Yftah Ziser, Anna Korhonen +2

Can unified vision-language models (VLMs) perform forward dynamics prediction (FDP), i.e., predicting the future state (in image form) given the previous observation and an action…