1 citations · 1 across the 4 of their papers we have counts for
5 papers
Anticipatory Planning for Multimodal AI Agents
Yongyuan Liang, Shijie Zhou, Yu Gu +6
Recent advances in multimodal agents have improved computer-use interaction and tool-usage, yet most existing systems remain reactive, optimizing actions in isolation without reaso…
SlowBA: An efficiency backdoor attack towards VLM-based GUI agents
Junxian Li, Tu Lan, Haozhen Tan +2
Modern vision-language-model (VLM) based graphical user interface (GUI) agents are expected not only to execute actions accurately but also to respond to user instructions with low…
MS4UI: A Dataset for Multi-modal Summarization of User Interface Instructional Videos
Yuan Zang, Hao Tan, Seunghyun Yoon +5
We study multi-modal summarization for instructional videos, whose goal is to provide users an efficient way to learn skills in the form of text instructions and key video frames.…
Identifying Speakers in Dialogue Transcripts: A Text-based Approach Using Pretrained Language Models
Minh Nguyen, Franck Dernoncourt, Seunghyun Yoon +6
We introduce an approach to identifying speaker names in dialogue transcripts, a crucial task for enhancing content accessibility and searchability in digital media archives. Despi…
Pre-trained Vision-Language Models Learn Discoverable Visual Concepts
Yuan Zang, Tian Yun, Hao Tan +2
Do vision-language models (VLMs) pre-trained to caption an image of a "durian" learn visual concepts such as "brown" (color) and "spiky" (texture) at the same time? We aim to answe…