133 citations · 316 across the 39 of their papers we have counts for
26 papers · 1 filter
Emergent Unfaithfulness: How Alignment Training Causes Language Models to Silently Override Task Faithfulness
Pardis Sadat Zahraei, Janvijay Singh, Gokhan Tur +1
Large language models are characterized by three key properties: capability, alignment, and faithfulness. Prior work studies the tradeoffs between capability and alignment, and bet…
ReIn: Conversational Error Recovery with Reasoning Inception
Takyoung Kim, Jinseok Nam, Chandrayee Basu +5
Conversational agents powered by large language models (LLMs) with tool integration achieve strong performance on fixed task-oriented dialogue datasets but remain vulnerable to una…
ATOD: An Evaluation Framework and Benchmark for Agentic Task-Oriented Dialogue Systems
Yifei Zhang, Hooshang Nayyeri, Rinat Khaziev +5
Agentic task-oriented dialogue (TOD) requires systems to track concurrent goals, dependencies, and long-horizon state. We examine goal-lifecycle recovery from fixed dialogue trajec…
The Alignment Veto: How Safety Training Suppresses Cultural Knowledge in LLMs
Pardis Sadat Zahraei, Gokhan Tur, Dilek Hakkani-Tür +1
What happens inside a language model when alignment training conflicts with a cultural value it encodes? Across 16 MENA countries, 26 models, and 1.53M human survey responses, we s…
Goal Alignment in LLM-Based User Simulators for Conversational AI
Shuhaib Mehri, Xiaocheng Yang, Takyoung Kim +3
User simulators are essential to conversational AI, enabling scalable agent development and evaluation through simulated interactions. While current Large Language Models (LLMs) ha…
Must Read: A Comprehensive Survey of Computational Persuasion
Nimet Beyza Bozdag, Shuhaib Mehri, Xiaocheng Yang +7
Persuasion is a fundamental aspect of communication, influencing decision-making across diverse contexts, from everyday conversations to high-stakes scenarios such as politics, mar…
AURA: A Diagnostic Framework for Tracking User Satisfaction of Interactive Planning Agents
Takyoung Kim, Janvijay Singh, Shuhaib Mehri +6
The growing capabilities of large language models (LLMs) in instruction-following and context-understanding lead to the era of agents with numerous applications. Among these, task…
YourBench: Easy Custom Evaluation Sets for Everyone
Sumuk Shashidhar, Clémentine Fourrier, Alina Lozovskia +3
Evaluating large language models (LLMs) effectively remains a critical bottleneck, as traditional static benchmarks suffer from saturation and contamination, while human evaluation…
TD-EVAL: Revisiting Task-Oriented Dialogue Evaluation by Combining Turn-Level Precision with Dialogue-Level Comparisons
Emre Can Acikgoz, Carl Guo, Suvodip Dey +4
Task-oriented dialogue (TOD) systems are experiencing a revolution driven by Large Language Models (LLMs), yet the evaluation methodologies for these systems remain insufficient fo…
Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models
Nimet Beyza Bozdag, Shuhaib Mehri, Gokhan Tur +1
Large Language Models (LLMs) demonstrate persuasive capabilities that rival human-level persuasion. While these capabilities can be used for social good, they also present risks of…
Better Slow than Sorry: Introducing Positive Friction for Reliable Dialogue Systems
Mert İnan, Anthony Sicilia, Suvodip Dey +6
While theories of discourse and cognitive science have long recognized the value of unhurried pacing, recent dialogue research tends to minimize friction in conversational systems.…
Know Your Mistakes: Towards Preventing Overreliance on Task-Oriented Conversational AI Through Accountability Modeling
Suvodip Dey, Yi-Jyun Sun, Gokhan Tur +1
Recent LLMs have enabled significant advancements for conversational agents. However, they are also well known to hallucinate, producing responses that seem plausible but are factu…
Large Language Models as User-Agents for Evaluating Task-Oriented-Dialogue Systems
Taaha Kazi, Ruiliang Lyu, Sizhe Zhou +2
Traditionally, offline datasets have been used to evaluate task-oriented dialogue (TOD) models. These datasets lack context awareness, making them suboptimal benchmarks for convers…
ReSpAct: Harmonizing Reasoning, Speaking, and Acting Towards Building Large Language Model-Based Conversational AI Agents
Vardhan Dongre, Xiaocheng Yang, Emre Can Acikgoz +3
Large language model (LLM)-based agents are increasingly employed to interact with external environments (e.g., games, APIs, world models) to solve user-provided tasks. However, cu…
Simulating User Agents for Embodied Conversational-AI
Daniel Philipov, Vardhan Dongre, Gokhan Tur +1
Embodied agents designed to assist users with tasks must engage in natural language interactions, interpret instructions, execute actions, and communicate effectively to resolve is…
Confidence Estimation for LLM-Based Dialogue State Tracking
Yi-Jyun Sun, Suvodip Dey, Dilek Hakkani-Tur +1
Estimation of a model's confidence on its outputs is critical for Conversational AI systems based on large language models (LLMs), especially for reducing hallucination and prevent…
Dialog Flow Induction for Constrainable LLM-Based Chatbots
Stuti Agrawal, Nishi Uppuluri, Pranav Pillai +5
LLM-driven dialog systems are used in a diverse set of applications, ranging from healthcare to customer service. However, given their generalization capability, it is difficult to…
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
Jack FitzGerald, Shankar Ananthakrishnan, Konstantine Arkoudas +38
We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B, their subsequent distillation into smaller m…
MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
Jack FitzGerald, Christopher Hench, Charith Peris +13
We present the MASSIVE dataset--Multilingual Amazon Slu resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant Evaluation. MASSIVE contains 1M real…
Generative Conversational Networks
Alexandros Papangelis, Karthik Gopalakrishnan, Aishwarya Padmakumar +3
Inspired by recent work in meta-learning and generative teaching networks, we propose a framework called Generative Conversational Networks, in which conversational agents learn to…
Interactive Teaching for Conversational AI
Qing Ping, Feiyang Niu, Govind Thattai +7
Current conversational AI systems aim to understand a set of pre-designed requests and execute related actions, which limits them to evolve naturally and adapt based on human inter…
Controllable Text Generation with Focused Variation
Lei Shu, Alexandros Papangelis, Yi-Chia Wang +5
This work introduces Focused-Variation Network (FVN), a novel model to control language generation. The main problems in previous controlled language generation models range from t…
Joint Contextual Modeling for ASR Correction and Language Understanding
Yue Weng, Sai Sumanth Miryala, Chandra Khatri +8
The quality of automatic speech recognition (ASR) is critical to Dialogue Systems as ASR errors propagate to and directly impact downstream tasks such as language understanding (LU…
Exploration Based Language Learning for Text-Based Games
Andrea Madotto, Mahdi Namazifar, Joost Huizinga +7
This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describ…