61 citations · 183 across the 12 of their papers we have counts for
5 papers · 1 filter
Towards Automated Accessibility Report Generation for Mobile Apps
Amanda Swearngin, Jason Wu, Xiaoyi Zhang +8
Many apps have basic accessibility issues, like missing labels or low contrast. Automated tools can help app developers catch basic issues, but can be laborious or require writing…
ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations
Yue Jiang, Eldon Schoop, Amanda Swearngin +1
Multimodal Vision-Language Models (VLMs) enable powerful applications from their fused understanding of images and language, but many perform poorly on UI tasks due to the lack of…
AXNav: Replaying Accessibility Tests from Natural Language
Maryam Taeb, Amanda Swearngin, Eldon Schoop +3
Developers and quality assurance testers often rely on manual testing to test accessibility features throughout the product lifecycle. Unfortunately, manual testing can be tedious,…
Never-ending Learning of User Interfaces
Jason Wu, Rebecca Krosnick, Eldon Schoop +3
Machine learning models have been trained to predict semantic information about user interfaces (UIs) to make apps more accessible, easier to test, and to automate. Currently, most…
WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics
Jason Wu, Siyan Wang, Siman Shen +3
Modeling user interfaces (UIs) from visual information allows systems to make inferences about the functionality and semantics needed to support use cases in accessibility, app aut…