4 papers
CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception
Zejun Xu, Taiyi Chen, Jin Li +13
Large language models are increasingly deployed as autonomous agents that interact with the web through browsers. While recent progress has been driven by benchmarks that evaluate…
Software as Content: Dynamic Applications as the Human-Agent Interaction Layer
Mulong Xie, Yang Xie
Chat-based natural language interfaces have emerged as the dominant paradigm for human-agent interaction, yet they fundamentally constrain engagement with structured information an…
WebNavigator: Global Web Navigation via Interaction Graph Retrieval
Xuanwang Zhang, Yuteng Han, Jinnan Qi +3
Despite significant advances in autonomous web navigation, current methods remain far from human-level performance in complex web environments. We argue that this limitation stems…
Towards Human-AI Synergy in UI Design: Supporting Iterative Generation with LLMs
Mingyue Yuan, Jieshan Chen, Yongquan Hu +5
In automated UI design generation, a key challenge is the lack of support for iterative processes, as most systems focus solely on end-to-end output. This stems from limited capabi…