4 citations · 7 across the 11 of their papers we have counts for
16 papers
Calibrating Verbalized Confidence with Self-Generated Distractors
Victor Wang, Elias Stengel-Eskin
Calibrated confidence estimates are necessary for large language model (LLM) outputs to be trusted by human users. While LLMs can express their confidence in human-interpretable wa…
Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models
Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal
An increasing number of vision-language tasks can be handled with little to no training, i.e., in a zero and few-shot manner, by marrying large language models (LLMs) to vision enc…
Zero and Few-shot Semantic Parsing with Ambiguous Inputs
Elias Stengel-Eskin, Kyle Rawlins, Benjamin Van Durme
Despite the frequent challenges posed by ambiguity when representing meaning via natural language, it is often ignored or deliberately removed in tasks mapping language to formally…
Did You Mean...? Confidence-based Trade-offs in Semantic Parsing
Elias Stengel-Eskin, Benjamin Van Durme
We illustrate how a calibrated model can help balance common trade-offs in task-oriented parsing. In a simulated annotator-in-the-loop experiment, we show that well-calibrated conf…
Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning
Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin +4
Visual Question Answering (VQA) models often perform poorly on out-of-distribution data and struggle on domain generalization. Due to the multi-modal nature of this task, multiple…
Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA
Elias Stengel-Eskin, Jimena Guallar-Blasco, Yi Zhou +1
Natural language is ambiguous. Resolving ambiguous questions is key to successfully answering them. Focusing on questions about images, we create a dataset of ambiguous examples. W…
Calibrated Interpretation: Confidence Estimation in Semantic Parsing
Elias Stengel-Eskin, Benjamin Van Durme
Sequence generation models are increasingly being used to translate natural language into programs, i.e. to perform executable semantic parsing. The fact that semantic parsing aims…
Visual Commonsense in Pretrained Unimodal and Multimodal Models
Chenyu Zhang, Benjamin Van Durme, Zhuowan Li +1
Our commonsense knowledge about objects includes their typical visual attributes; we know that bananas are typically yellow or green, and not purple. Text and image corpora, being…
When More Data Hurts: A Troubling Quirk in Developing Broad-Coverage Natural Language Understanding Systems
Elias Stengel-Eskin, Emmanouil Antonios Platanios, Adam Pauls +5
In natural language understanding (NLU) production systems, users' evolving needs necessitate the addition of new features over time, indexed by new symbols added to the meaning re…
The Curious Case of Control
Elias Stengel-Eskin, Benjamin Van Durme
Children acquiring English make systematic errors on subject control sentences even after they have reached near-adult competence (C. Chomsky, 1969), possibly due to heuristics bas…
Calibrating Concepts and Operations: Towards Symbolic Reasoning on Real Images
Zhuowan Li, Elias Stengel-Eskin, Yixiao Zhang +4
While neural symbolic methods demonstrate impressive performance in visual question answering on synthetic images, their performance suffers on real images. We identify that the lo…
Joint Universal Syntactic and Semantic Parsing
Elias Stengel-Eskin, Kenton Murray, Sheng Zhang +2
While numerous attempts have been made to jointly parse syntax and semantics, high performance in one domain typically comes at the price of performance in the other. This trade-of…
Iterative Paraphrastic Augmentation with Discriminative Span Alignment
Ryan Culkin, J. Edward Hu, Elias Stengel-Eskin +2
We introduce a novel paraphrastic augmentation strategy based on sentence-level lexically constrained paraphrasing and discriminative span alignment. Our approach allows for the la…
Universal Decompositional Semantic Parsing
Elias Stengel-Eskin, Aaron Steven White, Sheng Zhang +1
We introduce a transductive model for parsing into Universal Decompositional Semantics (UDS) representations, which jointly learns to map natural language utterances into UDS graph…
The Universal Decompositional Semantics Dataset and Decomp Toolkit
Aaron Steven White, Elias Stengel-Eskin, Siddharth Vashishtha +9
We present the Universal Decompositional Semantics (UDS) dataset (v1.0), which is bundled with the Decomp toolkit (v0.1). UDS1.0 unifies five high-quality, decompositional semantic…
A Discriminative Neural Model for Cross-Lingual Word Alignment
Elias Stengel-Eskin, Tzu-Ray Su, Matt Post +1
We introduce a novel discriminative word alignment model, which we integrate into a Transformer-based machine translation model. In experiments based on a small number of labeled e…