InsTALL: Context-aware Instructional Task Assistance with Multi-modal Large Language Models

Pha Nguyen, Sailik Sengupta, Girik Malik +2

The improved competence of generative models can help building multi-modal virtual assistants that leverage modalities beyond language. By observing humans performing multi-step ta…