143 citations · 180 across the 10 of their papers we have counts for
10 papers
CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception
Zejun Xu, Taiyi Chen, Jin Li +13
Large language models are increasingly deployed as autonomous agents that interact with the web through browsers. While recent progress has been driven by benchmarks that evaluate…
Software as Content: Dynamic Applications as the Human-Agent Interaction Layer
Mulong Xie, Yang Xie
Chat-based natural language interfaces have emerged as the dominant paradigm for human-agent interaction, yet they fundamentally constrain engagement with structured information an…
WebNavigator: Global Web Navigation via Interaction Graph Retrieval
Xuanwang Zhang, Yuteng Han, Jinnan Qi +5
Despite significant advances in autonomous web navigation, current methods remain far from human-level performance in complex web environments. We argue that this limitation stems…
Towards Human-AI Synergy in UI Design: Supporting Iterative Generation with LLMs
Mingyue Yuan, Jieshan Chen, Yongquan Hu +5
In automated UI design generation, a key challenge is the lack of support for iterative processes, as most systems focus solely on end-to-end output. This stems from limited capabi…
NiCro: Purely Vision-based, Non-intrusive Cross-Device and Cross-Platform GUI Testing
Mulong Xie, Jiaming Ye, Zhenchang Xing +1
To ensure app compatibility and smoothness of user experience across diverse devices and platforms, developers have to perform cross-device, cross-platform testing of their apps, w…
Read It, Don't Watch It: Captioning Bug Recordings Automatically
Sidong Feng, Mulong Xie, Yinxing Xue +1
Screen recordings of mobile applications are easy to capture and include a wealth of information, making them a popular mechanism for users to inform developers of the problems enc…
Towards Benchmarking GUI Compatibility Testing on Mobile Applications
Jiaming Ye, Mulong Xie, Siyuan Chen +4
GUI is a bridge connecting user and application. Existing GUI testing tasks can be categorized into two groups: functionality testing and compatibility testing. While the functiona…
Efficiency Matters: Speeding Up Automated Testing with GUI Rendering Inference
Sidong Feng, Mulong Xie, Chunyang Chen
Due to the importance of Android app quality assurance, many automated GUI testing tools have been developed. Although the test algorithms have been improved, the impact of GUI ren…
Psychologically-Inspired, Unsupervised Inference of Perceptual Groups of GUI Widgets from GUI Images
Mulong Xie, Zhenchang Xing, Sidong Feng +3
Graphical User Interface (GUI) is not merely a collection of individual and unrelated widgets, but rather partitions discrete widgets into groups by various visual cues, thus formi…
Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?
Jieshan Chen, Mulong Xie, Zhenchang Xing +4
Detecting Graphical User Interface (GUI) elements in GUI images is a domain-specific object detection task. It supports many software engineering tasks, such as GUI animation and t…