Unfettered Forceful Skill Acquisition with Physical Reasoning and Coordinate Frame Labeling

William Xie, Max Conway, Yutong Zhang +1

Vision language models (VLMs) exhibit vast knowledge of the physical world, including intuition of physical and spatial properties, affordances, and motion. With fine-tuning, VLMs…