4 citations · 21 across the 53 of their papers we have counts for
34 papers · 1 filter
Emergent Unfaithfulness: How Alignment Training Causes Language Models to Silently Override Task Faithfulness
Pardis Sadat Zahraei, Janvijay Singh, Gokhan Tur +1
Large language models are characterized by three key properties: capability, alignment, and faithfulness. Prior work studies the tradeoffs between capability and alignment, and bet…
Hear2Act: Benchmarking When Prosody Should Change What an Assistant Does
Xinyi Liu, Hooshang Nayyeri, Dilek Hakkani-Tur +5
Prosodic cues can carry task-relevant information that words omit, but existing benchmarks rarely test whether it changes downstream decisions. We introduce Hear2Act, a 480-scenari…
ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Traces
Jinu Lee, Shivam Agarwal, Amruta Parulekar +3
Large reasoning models (LRMs) produce reasoning traces with non-linear structures, such as backtracking and self-correction, that complicate the evaluation and monitoring of the re…
MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models
Hyeonjeong Ha, Jeonghwan Kim, Cheng Qian +7
Memory-augmented large language models extend reasoning beyond a fixed context window by maintaining long-term memory across interactions. However, existing memory systems often co…
Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systems
Vira Kasprova, Amruta Parulekar, Abdulrahman AlRabah +5
Large language models (LLMs) often exhibit sycophancy: agreement with user stance even when it conflicts with the model's opinion. While prior work has mostly studied this in singl…
Sparking Scientific Creativity via LLM-Driven Interdisciplinary Inspiration
Priyanka Kargupta, Shuhaib Mehri, Dilek Hakkani-Tur +1
Despite interdisciplinary research leading to larger and longer-term impact, most work remains confined to single-domain academic silos. Recent AI-based approaches to scientific di…
ReIn: Conversational Error Recovery with Reasoning Inception
Takyoung Kim, Jinseok Nam, Chandrayee Basu +5
Conversational agents powered by large language models (LLMs) with tool integration achieve strong performance on fixed task-oriented dialogue datasets but remain vulnerable to una…
ATOD: An Evaluation Framework and Benchmark for Agentic Task-Oriented Dialogue Systems
Yifei Zhang, Hooshang Nayyeri, Rinat Khaziev +5
Agentic task-oriented dialogue (TOD) requires systems to track concurrent goals, dependencies, and long-horizon state. We examine goal-lifecycle recovery from fixed dialogue trajec…
DialDefer: A Framework for Detecting and Mitigating LLM Dialogic Deference
Parisa Rabbani, Priyam Sahoo, Ruben Mathew +4
LLMs are increasingly used as third-party judges, yet their reliability when evaluating speakers in dialogue remains poorly understood. We show that LLMs judge identical claims dif…
Drift No More? Context Equilibria in Multi-Turn LLM Interactions
Vardhan Dongre, Ryan A. Rossi, Viet Dac Lai +3
Large Language Models (LLMs) excel at single-turn tasks such as instruction following and summarization, yet real-world deployments require sustained multi-turn interactions where…
From Fact to Judgment: Investigating the Impact of Task Framing on LLM Conviction in Dialogue Systems
Parisa Rabbani, Nimet Beyza Bozdag, Dilek Hakkani-Tür
LLMs are increasingly employed as judges across a variety of tasks, including those involving everyday social interactions. Yet, it remains unclear whether such LLM-judges can reli…
The Alignment Veto: How Safety Training Suppresses Cultural Knowledge in LLMs
Pardis Sadat Zahraei, Gokhan Tur, Dilek Hakkani-Tür +1
What happens inside a language model when alignment training conflicts with a cultural value it encodes? Across 16 MENA countries, 26 models, and 1.53M human survey responses, we s…
On the Shelf Life of Fine-Tuned LLM-Judges: Future-Proofing, Backward-Compatibility, and Question Generalization
Janvijay Singh, Austin Xu, Yilun Zhou +3
The LLM-as-a-judge paradigm is widely used in both evaluating free-text model responses and reward modeling for model alignment and fine-tuning. Recently, fine-tuning judges with j…
EiCAP: Beyond Fluency, Probing and Improving Emotional Intelligence in LLMs via Psychologically Grounded Multi-Turn Dialogue
Nizi Nazar, Pardis Sadat Zahraei, Dilek Hakkani-Tür +2
Large Language Models increasingly serve in emotionally sensitive roles, including mental health support, education, and crisis response, yet they lack a principled framework for a…
Question Generation for Assessing Early Literacy Reading Comprehension
Xiaocheng Yang, Sumuk Shashidhar, Dilek Hakkani-Tur
Assessment of reading comprehension through content-based interactions plays an important role in the reading acquisition process. In this paper, we propose a novel approach for ge…
Goal Alignment in LLM-Based User Simulators for Conversational AI
Shuhaib Mehri, Xiaocheng Yang, Takyoung Kim +3
User simulators are essential to conversational AI, enabling scalable agent development and evaluation through simulated interactions. While current Large Language Models (LLMs) ha…
ReasoningFlow: Semantic Structure of Complex Reasoning Traces
Jinu Lee, Sagnik Mukherjee, Dilek Hakkani-Tur +1
Large reasoning models (LRMs) generate complex reasoning traces with planning, reflection, verification, and backtracking. In this work, we introduce ReasoningFlow, a unified schem…
Enabling Chatbots with Eyes and Ears: An Immersive Multimodal Conversation System for Dynamic Interactions
Jihyoung Jang, Minwook Bae, Minji Kim +2
As chatbots continue to evolve toward human-like, real-world, interactions, multimodality remains an active area of research and exploration. So far, efforts to integrate multimoda…
Language Specific Knowledge: Do Models Know Better in X than in English?
Ishika Agarwal, Nimet Beyza Bozdag, Nisval Patel +1
Often, multilingual language models are trained with the objective to map semantically similar content (in different languages) in the same latent space. In this paper, we show a n…
Must Read: A Comprehensive Survey of Computational Persuasion
Nimet Beyza Bozdag, Shuhaib Mehri, Xiaocheng Yang +7
Persuasion is a fundamental aspect of communication, influencing decision-making across diverse contexts, from everyday conversations to high-stakes scenarios such as politics, mar…
AURA: A Diagnostic Framework for Tracking User Satisfaction of Interactive Planning Agents
Takyoung Kim, Janvijay Singh, Shuhaib Mehri +6
The growing capabilities of large language models (LLMs) in instruction-following and context-understanding lead to the era of agents with numerous applications. Among these, task…
YourBench: Easy Custom Evaluation Sets for Everyone
Sumuk Shashidhar, Clémentine Fourrier, Alina Lozovskia +3
Evaluating large language models (LLMs) effectively remains a critical bottleneck, as traditional static benchmarks suffer from saturation and contamination, while human evaluation…
TD-EVAL: Revisiting Task-Oriented Dialogue Evaluation by Combining Turn-Level Precision with Dialogue-Level Comparisons
Emre Can Acikgoz, Carl Guo, Suvodip Dey +4
Task-oriented dialogue (TOD) systems are experiencing a revolution driven by Large Language Models (LLMs), yet the evaluation methodologies for these systems remain insufficient fo…
Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models
Nimet Beyza Bozdag, Shuhaib Mehri, Gokhan Tur +1
Large Language Models (LLMs) demonstrate persuasive capabilities that rival human-level persuasion. While these capabilities can be used for social good, they also present risks of…