5 papers
Training Computer Use Agents to Assess the Usability of Graphical User Interfaces
Alice Gao, Weixi Tong, Rishab Vempati +4
Usability testing with experts and potential users can assess the effectiveness, efficiency, and user satisfaction of graphical user interfaces (GUIs) but doing so remains a costly…
Efficient Personalization of Generative User Interfaces
Yi-Hao Peng, Samarth Das, Jeffrey P. Bigham +1
Generative user interfaces (UIs) create new opportunities to adapt interfaces to individual users on demand, but personalization remains difficult because desirable UI properties a…
DesignPref: Capturing Personal Preferences in Visual Design Generation
Yi-Hao Peng, Jeffrey P. Bigham, Jason Wu
Generative models, such as large language models and text-to-image diffusion models, are increasingly used to create visual designs like user interfaces (UIs) and presentation slid…
CodeA11y: Making AI Coding Assistants Useful for Accessible Web Development
Peya Mowar, Yi-Hao Peng, Jason Wu +2
A persistent challenge in accessible computing is ensuring developers produce web UI code that supports assistive technologies. Despite numerous specialized accessibility tools, no…
DreamStruct: Understanding Slides and User Interfaces via Synthetic Data Generation
Yi-Hao Peng, Faria Huq, Yue Jiang +4
Enabling machines to understand structured visuals like slides and user interfaces is essential for making them accessible to people with disabilities. However, achieving such unde…