5 citations · 17 across the 10 of their papers we have counts for
13 papers
Probabilities of Chat LLMs Are Miscalibrated but Still Predict Correctness on Multiple-Choice Q&A
Benjamin Plaut, Nguyen X. Khanh, Tu Trinh
We study 15 large language models (LLMs) fine-tuned for chat and find that their maximum softmax probabilities (MSPs) are consistently miscalibrated on multiple-choice Q&A. However…
Language-Guided World Models: A Model-Based Approach to AI Control
Alex Zhang, Khanh Nguyen, Jens Tuyls +2
This paper introduces the concept of Language-Guided World Models (LWMs) -- probabilistic models that can simulate environments by reading texts. Agents equipped with these models…
Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Models
Lingjun Zhao, Khanh Nguyen, Hal Daumé
Recent work studies the cognitive capabilities of language models through psychological tests designed for humans. While these studies are helpful for understanding the general cap…
A Framework for Learning to Request Rich and Contextually Useful Information from Humans
Khanh Nguyen, Yonatan Bisk, Hal Daumé
When deployed, AI agents will encounter problems that are beyond their autonomous problem-solving capabilities. Leveraging human assistance can help agents overcome their inherent…
Interactive Learning from Activity Description
Khanh Nguyen, Dipendra Misra, Robert Schapire +2
We present a novel interactive learning protocol that enables training request-fulfilling agents by verbally describing their activities. Unlike imitation learning (IL), our protoc…
Active Imitation Learning from Multiple Non-Deterministic Teachers: Formulation, Challenges, and Algorithms
Khanh Nguyen, Hal Daumé
We formulate the problem of learning to imitate multiple, non-deterministic teachers with minimal interaction cost. Rather than learning a specific policy as in standard imitation…
Global Voices: Crossing Borders in Automatic News Summarization
Khanh Nguyen, Hal Daumé
We construct Global Voices, a multilingual dataset for evaluating cross-lingual summarization methods. We extract social-network descriptions of Global Voices news articles to chea…
Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning
Khanh Nguyen, Hal Daumé
Mobile agents that can leverage help from humans can potentially accomplish more complex tasks than they could entirely on their own. We develop "Help, Anna!" (HANNA), an interacti…
Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention
Khanh Nguyen, Debadeepta Dey, Chris Brockett +1
We present Vision-based Navigation with Language-based Assistance (VNLA), a grounded vision-language task where an agent with visual perception is guided via language to find objec…
The UMD Neural Machine Translation Systems at WMT17 Bandit Learning Task
Amr Sharaf, Shi Feng, Khanh Nguyen +2
We describe the University of Maryland machine translation systems submitted to the WMT17 German-English Bandit Learning Task. The task is to adapt a translation system to a new do…
Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback
Khanh Nguyen, Hal Daumé, Jordan Boyd-Graber
Machine translation is a natural candidate problem for reinforcement learning from human feedback: users provide quick, dirty ratings on candidate translations to guide a system to…
Imitation Learning with Recurrent Neural Networks
Khanh Nguyen
We present a novel view that unifies two frameworks that aim to solve sequential prediction problems: learning to search (L2S) and recurrent neural networks (RNN). We point out equ…
Posterior calibration and exploratory analysis for natural language processing models
Khanh Nguyen, Brendan O'Connor
Many models in natural language processing define probabilistic distributions over linguistic structures. We argue that (1) the quality of a model' s posterior distribution can and…