50 citations · 124 across the 13 of their papers we have counts for
3 papers · 1 filter
ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations
Yue Jiang, Eldon Schoop, Amanda Swearngin +1
Multimodal Vision-Language Models (VLMs) enable powerful applications from their fused understanding of images and language, but many perform poorly on UI tasks due to the lack of…
AXNav: Replaying Accessibility Tests from Natural Language
Maryam Taeb, Amanda Swearngin, Eldon Schoop +3
Developers and quality assurance testers often rely on manual testing to test accessibility features throughout the product lifecycle. Unfortunately, manual testing can be tedious,…
Never-ending Learning of User Interfaces
Jason Wu, Rebecca Krosnick, Eldon Schoop +3
Machine learning models have been trained to predict semantic information about user interfaces (UIs) to make apps more accessible, easier to test, and to automate. Currently, most…