42 citations · 142 across the 52 of their papers we have counts for
56 papers
From Task Success to Productive Success: Evaluating Human-AI Collaboration by Quality and Cost
Saki Imai, Mert İnan, Malihe Alikhani
AI productivity is often measured by task completion time, economic value, or improvements in outcome quality. However, these measures usually treat collaboration as a black box wh…
MixDPO: Modeling Preference Strength for Pluralistic Alignment
Saki Imai, Pedram Heydari, Anthony Sicilia +3
Preference based alignment objectives implicitly assume that all human preferences are expressed with equal strength. In practice, however, preference strength varies across indivi…
"Nothing about us without us": Perspectives of Global Deaf and Hard-of-hearing Community Members on Sign Language Technologies
Katherine Atwell, Saki Imai, Danielle Bragg +1
There is accelerating interest in sign language technologies (SLTs), with increasing attention from both industry and academia. However, the perspectives of Deaf and Hard-of-hearin…
Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generation
Mert İnan, Anthony Sicilia, Alex Xie +3
Establishing shared goals is a fundamental step in human-AI communication. However, ambiguities can lead to outputs that seem correct but fail to reflect the speaker's intent. In t…
How Pragmatics Shape Articulation: A Computational Case Study in STEM ASL Discourse
Saki Imai, Lee Kezar, Laurel Aichler +5
Most state-of-the-art sign language models are trained on interpreter or isolated vocabulary data, which overlooks the variability that characterizes natural dialogue. However, hum…
Measuring How (Not Just Whether) VLMs Build Common Ground
Saki Imai, Mert İnan, Anthony Sicilia +1
Large vision language models (VLMs) increasingly claim reasoning skills, yet current benchmarks evaluate them in single-turn or question answering settings. However, grounding is a…
SiLVERScore: Semantically-Aware Embeddings for Sign Language Generation Evaluation
Saki Imai, Mert İnan, Anthony Sicilia +1
Evaluating sign language generation is often done through back-translation, where generated signs are first recognized back to text and then compared to a reference using text-base…
BASIL: Bayesian Assessment of Sycophancy in LLMs
Katherine Atwell, Pedram Heydari, Anthony Sicilia +1
Sycophancy (overly agreeable or flattering behavior) poses a fundamental challenge for human-AI collaboration, particularly in high-stakes decision-making domains such as health, l…
OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation
Ziyi Wang, Yuxuan Lu, Wenbo Li +13
Can large language models (LLMs) accurately simulate the next web action of a specific user? While LLMs have shown promising capabilities in generating ``believable'' human behavio…
Human-centered explanation does not fit all: The interplay of sociotechnical, cognitive, and individual factors in the effect AI explanations in algorithmic decision-making
Yongsu Ahn, Yu-Ru Lin, Malihe Alikhani +1
Recent XAI studies have investigated what constitutes a \textit{good} explanation in AI-assisted decision-making. Despite the widely accepted human-friendly properties of explanati…
Better Slow than Sorry: Introducing Positive Friction for Reliable Dialogue Systems
Mert İnan, Anthony Sicilia, Suvodip Dey +6
While theories of discourse and cognitive science have long recognized the value of unhurried pacing, recent dialogue research tends to minimize friction in conversational systems.…
Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AI
Yuya Asano, Sabit Hassan, Paras Sharma +4
General-purpose automatic speech recognition (ASR) systems do not always perform well in goal-oriented dialogue. Existing ASR correction methods rely on prior user data or named en…
Fairness at Every Intersection: Uncovering and Mitigating Intersectional Biases in Multimodal Clinical Predictions
Resmi Ramachandranpillai, Kishore Sampath, Ayaazuddin Mohammad +1
Biases in automated clinical decision-making using Electronic Healthcare Records (EHR) impose significant disparities in patient care and treatment outcomes. Conventional approache…
Accounting for Sycophancy in Language Model Uncertainty Estimation
Anthony Sicilia, Mert Inan, Malihe Alikhani
Effective human-machine collaboration requires machine learning models to externalize uncertainty, so users can reflect and intervene when necessary. For language models, these rep…
Eliciting Uncertainty in Chain-of-Thought to Mitigate Bias against Forecasting Harmful User Behaviors
Anthony Sicilia, Malihe Alikhani
Conversation forecasting tasks a model with predicting the outcome of an unfolding conversation. For instance, it can be applied in social media moderation to predict harmful user…
Learning Multimodal Cues of Children's Uncertainty
Qi Cheng, Mert İnan, Rahma Mbarki +6
Understanding uncertainty plays a critical role in achieving common ground (Clark et al.,1983). This is especially important for multimodal AI systems that collaborate with users t…
Generating Signed Language Instructions in Large-Scale Dialogue Systems
Mert İnan, Katherine Atwell, Anthony Sicilia +2
We introduce a goal-oriented conversational AI system enhanced with American Sign Language (ASL) instructions, presenting the first implementation of such a system on a worldwide m…
Active Learning for Robust and Representative LLM Generation in Safety-Critical Scenarios
Sabit Hassan, Anthony Sicilia, Malihe Alikhani
Ensuring robust safety measures across a wide range of scenarios is crucial for user-facing systems. While Large Language Models (LLMs) can generate valuable data for safety measur…
Studying and Mitigating Biases in Sign Language Understanding Models
Katherine Atwell, Danielle Bragg, Malihe Alikhani
Ensuring that the benefits of sign language technologies are distributed equitably among all community members is crucial. Thus, it is important to address potential biases and ine…
Coherence-Driven Multimodal Safety Dialogue with Active Learning for Embodied Agents
Sabit Hassan, Hye-Young Chung, Xiang Zhi Tan +1
When assisting people in daily tasks, robots need to accurately interpret visual cues and respond effectively in diverse safety-critical situations, such as sharp objects on the fl…
An Active Learning Framework for Inclusive Generation by Large Language Models
Sabit Hassan, Anthony Sicilia, Malihe Alikhani
Ensuring that Large Language Models (LLMs) generate text representative of diverse sub-populations is essential, particularly when key concepts related to under-represented groups…
Evaluating Theory of (an uncertain) Mind: Predicting the Uncertain Beliefs of Others in Conversation Forecasting
Anthony Sicilia, Malihe Alikhani
Typically, when evaluating Theory of Mind, we consider the beliefs of others to be binary: held or not held. But what if someone is unsure about their own beliefs? How can we quant…
Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community
Casey Kennington, Malihe Alikhani, Heather Pon-Barry +20
The ability to interact with machines using natural human language is becoming not just commonplace, but expected. The next step is not just text interfaces, but speech interfaces…
Learning to Generate Context-Sensitive Backchannel Smiles for Embodied AI Agents with Applications in Mental Health Dialogues
Maneesh Bilalpur, Mert Inan, Dorsa Zeinali +2
Addressing the critical shortage of mental health resources for effective screening, diagnosis, and treatment remains a significant challenge. This scarcity underscores the need fo…