39 citations · 178 across the 15 of their papers we have counts for
4 papers · 1 filter
Spotlight: Mobile UI Understanding using Vision-Language Models with a Focus
Gang Li, Yang Li
Mobile UI understanding is important for enabling various interaction tasks such as UI automation and accessibility. Previous mobile UI modeling often depends on the view hierarchy…
Enabling Conversational Interaction with Mobile UI using Large Language Models
Bryan Wang, Gang Li, Yang Li
Conversational agents show the promise to allow users to interact with mobile devices using language. However, to perform diverse UI tasks with natural language, developers typical…
Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis
Eldon Schoop, Xin Zhou, Gang Li +3
We use a deep learning based approach to predict whether a selected element in a mobile UI screenshot will be perceived by users as tappable, based on pixels only instead of view h…
Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale
Gang Li, Gilles Baechler, Manuel Tragut +1
The layout of a mobile screen is a critical data source for UI design research and semantic understanding of the screen. However, UI layouts in existing datasets are often noisy, h…