361 citations
- Carnegie Mellon UniversityUS12 papers
- Stanford UniversityUS6 papers
- Google (United States)US5 papers
- University of WashingtonUS5 papers
- Amazon (United States)US4 papers
- Apple (Germany)DE4 papers
- Apple (Israel)IL4 papers
- Arizona State UniversityUS4 papers
- Centre National de la Recherche ScientifiqueFR4 papers
- Jet Propulsion LaboratoryUS4 papers
- Duke UniversityUS3 papers
- École Polytechnique Fédérale de LausanneCH3 papers
27 papers · 1 filter
Mapping the Design Space of User Experience for Computer Use Agents
Ruijia Cheng, Jenny T. Liang, Eldon Schoop +1
Large language model (LLM)-based computer use agents execute user commands by interacting with available UI elements, but little is known about how users want to interact with thes…
Improving User Interface Generation Models from Designer Feedback
Jason Wu, Amanda Swearngin, Arun Krishna Vajjala +3
Despite being trained on vast amounts of data, most LLMs are unable to reliably generate well-designed UIs. Designer feedback is essential to improving performance on UI generation…
VibE: A Visual Analytics Workflow for Semantic Error Analysis of CVML Models at Subgroup Level
Jun Yuan, Kevin Miao, Heyin Oh +4
Effective error analysis is critical for the successful development and deployment of CVML models. One approach to understanding model errors is to summarize the common characteris…
eaSEL: Promoting Social-Emotional Learning and Parent-Child Interaction through AI-Mediated Content Consumption
Jocelyn Shen, Jennifer King Chen, Leah Findlater +1
As children increasingly consume media on devices, parents look for ways this usage can support learning and growth, especially in domains like social-emotional learning. We introd…
From Interaction to Impact: Towards Safer AI Agents Through Understanding and Evaluating Mobile UI Operation Impacts
Zhuohao Jerry Zhang, Eldon Schoop, Jeffrey Nichols +2
With advances in generative AI, there is increasing work towards creating autonomous agents that can manage daily tasks by operating user interfaces (UIs). While prior research has…
Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors
Michelle S. Lam, Fred Hohman, Dominik Moritz +3
AI policy sets boundaries on acceptable behavior for AI models, but this is challenging in the context of large language models (LLMs): how do you ensure coverage over a vast behav…