705 citations · 1.7k across the 20 of their papers we have counts for
22 papers
Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI
Q. Vera Liao, Yunfeng Zhang, Ronny Luss +2
Recent years have seen a surge of interest in the field of explainable AI (XAI), with a plethora of algorithms proposed in the literature. However, a lack of consensus on how to ev…
Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Vivian Lai, Samuel Carton, Rajat Bhatnagar +3
Despite impressive performance in many benchmark datasets, AI models can still make mistakes, especially among out-of-distribution examples. It remains an open question how such im…
AI Explainability 360: Impact and Design
Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen +17
As artificial intelligence and machine learning algorithms become increasingly prevalent in society, multiple stakeholders are calling for these algorithms to provide explanations.…
Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AI
Soumya Ghosh, Q. Vera Liao, Karthikeyan Natesan Ramamurthy +4
In this paper, we describe an open source Python toolkit named Uncertainty Quantification 360 (UQ360) for the uncertainty quantification of AI models. The goal of this toolkit is t…
D2S: Document-to-Slide Generation Via Query-Based Text Summarization
Edward Sun, Yufang Hou, Dakuo Wang +2
Presentations are critical for communication in all areas of our lives, yet the creation of slide decks is often tedious and time-consuming. There has been limited research aiming…
Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML
Shweta Narkar, Yunfeng Zhang, Q. Vera Liao +2
Automated Machine Learning (AutoML) is a rapidly growing set of technologies that automate the model development pipeline by searching model space and generating candidate models.…
How Much Automation Does a Data Scientist Want?
Dakuo Wang, Q. Vera Liao, Yunfeng Zhang +5
Data science and machine learning (DS/ML) are at the heart of the recent advancements of many Artificial Intelligence (AI) applications. There is an active research thread in AI, \…
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Umang Bhatt, Javier Antorán, Yunfeng Zhang +12
Algorithmic transparency entails exposing system properties to various stakeholders for purposes that include understanding, improving, and contesting predictions. Until now, most…
Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
Charvi Rastogi, Yunfeng Zhang, Dennis Wei +3
Several strands of research have aimed to bridge the gap between artificial intelligence (AI) and human decision-makers in AI-assisted decision-making, where humans are the consume…
Active Learning++: Incorporating Annotator's Rationale using Local Model Explanation
Bhavya Ghai, Q. Vera Liao, Yunfeng Zhang +1
We propose a new active learning (AL) framework, Active Learning++, which can utilize an annotator's labels as well as its rationale. Annotators can provide their rationale for cho…
Measuring Social Biases of Crowd Workers using Counterfactual Queries
Bhavya Ghai, Q. Vera Liao, Yunfeng Zhang +1
Social biases based on gender, race, etc. have been shown to pollute machine learning (ML) pipeline predominantly via biased training datasets. Crowdsourcing, a popular cost-effect…
Model Agnostic Multilevel Explanations
Karthikeyan Natesan Ramamurthy, Bhanukiran Vinzamuri, Yunfeng Zhang +1
In recent years, post-hoc local instance-level and global dataset-level explainability of black-box models has received a lot of attention. Much less attention has been given to ob…
Business (mis)Use Cases of Generative AI
Stephanie Houde, Vera Liao, Jacquelyn Martino +5
Generative AI is a class of machine learning technology that learns to generate new data from training data. While deep fakes and media-and art-related generative AI breakthroughs…
Joint Optimization of AI Fairness and Utility: A Human-Centered Approach
Yunfeng Zhang, Rachel K. E. Bellamy, Kush R. Varshney
Today, AI is increasingly being used in many high-stakes decision-making applications in which fairness is an important concern. Already, there are many examples of AI being biased…
Consumer-Driven Explanations for Machine Learning Decisions: An Empirical Study of Robustness
Michael Hind, Dennis Wei, Yunfeng Zhang
Many proposed methods for explaining machine learning predictions are in fact challenging to understand for nontechnical consumers. This paper builds upon an alternative consumer-d…
Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
Yunfeng Zhang, Q. Vera Liao, Rachel K. E. Bellamy
Today, AI is being increasingly used to help human experts make decisions in high-stakes scenarios. In these scenarios, full automation is often undesirable, not only due to the si…
Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience
Bhavya Ghai, Q. Vera Liao, Yunfeng Zhang +2
The wide adoption of Machine Learning technologies has created a rapidly growing demand for people who can train ML models. Some advocated the term "machine teacher" to refer to th…
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen +17
As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their…
Leveraging Latent Features for Local Explanations
Ronny Luss, Pin-Yu Chen, Amit Dhurandhar +4
As the application of deep neural networks proliferates in numerous areas such as medical imaging, video surveillance, and self driving cars, the need for explaining the decisions…
Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment
Jonathan Dodge, Q. Vera Liao, Yunfeng Zhang +2
Ensuring fairness of machine learning systems is a human-in-the-loop process. It relies on developers, users, and the general public to identify fairness problems and make improvem…
Bootstrapping Conversational Agents With Weak Supervision
Neil Mallinar, Abhishek Shah, Rajendra Ugrani +9
Many conversational agents in the market today follow a standard bot development framework which requires training intent classifiers to recognize user input. The need to create a…
AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
Rachel K. E. Bellamy, Kuntal Dey, Michael Hind +15
Fairness is an increasingly important concern as machine learning models are used to support decision making in high-stakes applications such as mortgage lending, hiring, and priso…