3 papers
cs.AI2024
On the Effects of Data Scale on UI Control Agents
Wei Li, William Bishop, Alice Li +4
Autonomous agents that control computer interfaces to accomplish human tasks are emerging. Leveraging LLMs to power such agents has been of special interest, but unless fine-tuned…
cs.AI2024
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
Christopher Rawles, Sarah Clinckemaillie, Yifan Chang +12
Autonomous agents that execute human tasks by controlling computers can enhance human productivity and application accessibility. However, progress in this field will be driven by…
cs.HC2023
UINav: A Practical Approach to Train On-Device Automation Agents
Wei Li, Fu-Lin Hsu, Will Bishop +3
Automation systems that can autonomously drive application user interfaces to complete user tasks are of great benefit, especially when users are situationally or permanently impai…