Video as the New Language for Real-World Decision Making

Sherry Yang, Jacob Walker, Jack Parker-Holder +5

Both text and video data are abundant on the internet and support large-scale self-supervised learning through next token or frame prediction. However, they have not been equally l…