1 citations · 1 across the 9 of their papers we have counts for
9 papers
Demo2Tutorial: From Human Experience to Multimodal Software Tutorials
Zechen Bai, Zhiheng Chen, Yiqi Lin +5
Human experience in digital environments offers a vast, underexplored resource of authentic, untrimmed interactions that contain rich procedural knowledge. We introduce Demo2Tutori…
CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing
Haobo Hu, Xiangwu Guo, Zhiheng Chen +4
While GUI agents have made significant progress in web navigation and basic operating system tasks, their capabilities in professional creative workflows remain largely underexplor…
ShowUI-Aloha: Human-Taught GUI Agent
Yichun Zhang, Xiangwu Guo, Yauhong Goh +5
Graphical User Interfaces (GUIs) are central to human-computer interaction, yet automating complex GUI tasks remains a major challenge for autonomous agents, largely due to a lack…
Factorized Learning for Temporally Grounded Video-Language Models
Wenzheng Zeng, Difei Gao, Mike Zheng Shou +1
Recent video-language models have shown great potential for video understanding, but still struggle with accurate temporal grounding for event-level perception. We observe that two…
AUTO-Explorer: Automated Data Collection for GUI Agent
Xiangwu Guo, Difei Gao, Mike Zheng Shou
Recent advancements in GUI agents have significantly expanded their ability to interpret natural language commands to manage software interfaces. However, acquiring GUI data remain…
EmoAgent: A Multi-Agent Framework for Diverse Affective Image Manipulation
Qi Mao, Haobo Hu, Yujie He +3
Affective Image Manipulation (AIM) aims to alter visual elements within an image to evoke specific emotional responses from viewers. However, existing AIM approaches rely on rigid…
WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point
Henry Hengyuan Zhao, Kaiming Yang, Wendi Yu +2
Recent progress in GUI agents has substantially improved visual grounding, yet robust planning remains challenging, particularly when the environment deviates from a canonical init…
The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Use
Siyuan Hu, Mingyu Ouyang, Difei Gao +1
The recently released model, Claude 3.5 Computer Use, stands out as the first frontier AI model to offer computer use in public beta as a graphical user interface (GUI) agent. As a…
Learning to Learn: How to Continuously Teach Humans and Machines
Parantak Singh, You Li, Ankur Sikarwar +7
Curriculum design is a fundamental component of education. For example, when we learn mathematics at school, we build upon our knowledge of addition to learn multiplication. These…