497 citations · 1.9k across the 38 of their papers we have counts for
12 papers · 1 filter
Rethinking Model Evaluation as Narrowing the Socio-Technical Gap
Q. Vera Liao, Ziang Xiao
The recent development of generative large language models (LLMs) poses new challenges for model evaluation that the research community and industry have been grappling with. While…
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap
Q. Vera Liao, Jennifer Wortman Vaughan
The rise of powerful large language models (LLMs) brings about tremendous opportunities for innovation but also looming risks for individuals and society at large. We have reached…
Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theory
Ziang Xiao, Susu Zhang, Vivian Lai +1
We address a fundamental challenge in Natural Language Generation (NLG) model evaluation -- the design and evaluation of evaluation metrics. Recognizing the limitations of existing…
Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding
Ziang Xiao, Xingdi Yuan, Q. Vera Liao +2
Qualitative analysis of textual contents unpacks rich and valuable information by assigning labels to the data. However, this process is often labor-intensive, particularly when wo…
Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling
Haotian Li, Yun Wang, Q. Vera Liao +1
Data storytelling plays an important role in data workers' daily jobs since it boosts team collaboration and public communication. However, to make an appealing data story, data wo…
Sensing Wellbeing in the Workplace, Why and For Whom? Envisioning Impacts with Organizational Stakeholders
Anna Kawakami, Shreya Chowdhary, Shamsi T. Iqbal +4
With the heightened digitization of the workplace, alongside the rise of remote and hybrid work prompted by the pandemic, there is growing corporate interest in using passive sensi…
fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks
Steven Moore, Q. Vera Liao, Hariharan Subramonyam
To design with AI models, user experience (UX) designers must assess the fit between the model and user needs. Based on user research, they need to contextualize the model's behavi…
Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience
Q. Vera Liao, Hariharan Subramonyam, Jennifer Wang +1
Despite the widespread use of artificial intelligence (AI), designing user experiences (UX) for AI-powered systems remains challenging. UX designers face hurdles understanding AI t…
Human-Centered Responsible Artificial Intelligence: Current & Future Trends
Mohammad Tahaei, Marios Constantinides, Daniele Quercia +13
In recent years, the CHI community has seen significant growth in research on Human-Centered Responsible Artificial Intelligence. While different research communities may use diffe…
Powering an AI Chatbot with Expert Sourcing to Support Credible Health Information Access
Ziang Xiao, Q. Vera Liao, Michelle X. Zhou +2
During a public health crisis like the COVID-19 pandemic, a credible and easy-to-access information portal is highly desirable. It helps with disease prevention, public health plan…
Selective Explanations: Leveraging Human Input to Align Explainable AI
Vivian Lai, Yiming Zhang, Chacha Chen +2
While a vast collection of explainable AI (XAI) algorithms have been developed in recent years, they are often criticized for significant gaps with how humans produce and consume e…
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Valerie Chen, Q. Vera Liao, Jennifer Wortman Vaughan +1
AI explanations are often mentioned as a way to improve human-AI decision-making, but empirical studies have not found consistent evidence of explanations' effectiveness and, on th…