LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning

Dantong Niu, Yuvan Sharma, Giscard Biamby +5

In recent years, instruction-tuned Large Multimodal Models (LMMs) have been successful at several tasks, including image captioning and visual question answering; yet leveraging th…