61 citations · 131 across the 8 of their papers we have counts for
8 papers
Never-ending Learning of User Interfaces
Jason Wu, Rebecca Krosnick, Eldon Schoop +3
Machine learning models have been trained to predict semantic information about user interfaces (UIs) to make apps more accessible, easier to test, and to automate. Currently, most…
WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics
Jason Wu, Siyan Wang, Siman Shen +3
Modeling user interfaces (UIs) from visual information allows systems to make inferences about the functionality and semantics needed to support use cases in accessibility, app aut…
Screen Correspondence: Mapping Interchangeable Elements between UIs
Jason Wu, Amanda Swearngin, Xiaoyi Zhang +2
Understanding user interface (UI) functionality is a useful yet challenging task for both machines and people. In this paper, we investigate a machine learning approach for screen…
Extracting Replayable Interactions from Videos of Mobile App Usage
Jieshan Chen, Amanda Swearngin, Jason Wu +3
Screen recordings of mobile apps are a popular and readily available way for users to share how they interact with apps, such as in online tutorial videos, user reviews, or as atta…
Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots
Jason Wu, Xiaoyi Zhang, Jeff Nichols +1
Automated understanding of user interfaces (UIs) from their pixels can improve accessibility, enable task automation, and facilitate interface design without relying on developers…
When Can Accessibility Help?: An Exploration of Accessibility Feature Recommendation on Mobile Devices
Jason Wu, Gabriel Reyes, Sam C. White +2
Numerous accessibility features have been developed and included in consumer operating systems to provide people with a variety of disabilities additional ways to access computing…