Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process

Abstract

AI is anticipated to enhance human decision-making in high-stakes domains like aviation, but adoption is often hindered by challenges such as inappropriate reliance and poor alignment with users' decision-making. Recent research suggests that a core underlying issue is the recommendation-centric design of many AI systems, i.e., they give end-to-end recommendations and ignore the rest of the decision-making process. Alternative support paradigms are rare, and it remains unclear how the few that do exist compare to recommendation-centric support. In this work, we aimed to empirically compare recommendation-centric support to an alternative paradigm, continuous support, in the context of diversions in aviation. We conducted a mixed-methods study with 32 professional pilots in a realistic setting. To ensure the quality of our study scenarios, we conducted a focus group with four additional pilots prior to the study. We found that continuous support can support pilots' decision-making in a forward direction, allowing them to think more beyond the limits of the system and make faster decisions when combined with recommendations, though the forward support can be disrupted. Participants' statements further suggest a shift in design goal away from providing recommendations, to supporting quick information gathering. Our results show ways to design more helpful and effective AI decision support that goes beyond end-to-end recommendations.

Accepted to CSCW 2024, to be published in PACM HCI Vol. 8, No. CSCW2

References in corpus (14)

• To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• Expanding Explainability: Towards Social Transparency in AI systems
• On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
• Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes
• Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
• Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens
• Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning
• Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making
• Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction
• Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis
• "If I Had All the Time in the World": Ophthalmologists' Perceptions of Anchoring Bias Mitigation in Clinical AI Support
• Augmenting Pathologists with NaviPath: Design and Evaluation of a Human-AI Collaborative Navigation System
• Diagrammatization and Abduction to Improve AI Interpretability With Domain-Aligned Explanations for Medical Diagnosis

Cited by in corpus (4)

• AIdeation: Designing a Human-AI Collaborative Ideation System for Concept Designers
• AI, Help Me Think$\unicode{x2014}$but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Support
• Design Considerations for Human Oversight of AI: Insights from Co-Design Workshops and Work Design Theory
• Give Me a Choice: The Consequences of Restricting Choices Through AI-Support for Perceived Autonomy, Motivational Variables, and Decision Performance