6 papers
GUIDE: Designer-in-the-loop Authoring of Conformant Generative User Interfaces
Hyewon Lee, Ziying Wang, Aiden Moy +2
Generative User Interfaces (GenUIs) enable applications to generate interfaces on demand from user needs and context. Like conventional UIs, they must still reflect designers' inte…
Training Computer Use Agents to Assess the Usability of Graphical User Interfaces
Alice Gao, Weixi Tong, Rishab Vempati +4
Usability testing with experts and potential users can assess the effectiveness, efficiency, and user satisfaction of graphical user interfaces (GUIs) but doing so remains a costly…
Efficient Personalization of Generative User Interfaces
Yi-Hao Peng, Samarth Das, Jeffrey P. Bigham +1
Generative user interfaces (GenUIs) create new opportunities to adapt interfaces to individual users on demand. Yet personalization is difficult because it is not possible to provi…
DesignPref: Capturing Personal Preferences in Visual Design Generation
Yi-Hao Peng, Jeffrey P. Bigham, Jason Wu
Generative models, such as large language models and text-to-image diffusion models, are increasingly used to create visual designs like user interfaces (UIs) and presentation slid…
CodeA11y: Making AI Coding Assistants Useful for Accessible Web Development
Peya Mowar, Yi-Hao Peng, Jason Wu +2
A persistent challenge in accessible computing is ensuring developers produce web UI code that supports assistive technologies. Despite numerous specialized accessibility tools, no…
DreamStruct: Understanding Slides and User Interfaces via Synthetic Data Generation
Yi-Hao Peng, Faria Huq, Yue Jiang +4
Enabling machines to understand structured visuals like slides and user interfaces is essential for making them accessible to people with disabilities. However, achieving such unde…