90 citations · 289 across the 20 of their papers we have counts for
19 papers · 1 filter
CT: Counterfactual Causal Reasoning for Sentiment Shifts in Social-Media Conversation Trees
S M Rafiuddin, Atriya Sen
Sentiment in social-media threads does not only vary across posts; it shifts as users react to claims, corrections, evidence, and hostility within a branching reply tree. We study…
Leveraging In-Context Learning for Language Model Agents
Shivanshu Gupta, Sameer Singh, Ashish Sabharwal +2
In-context learning (ICL) with dynamically selected demonstrations combines the flexibility of prompting large language models (LLMs) with the ability to leverage training data to…
AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
Ori Yoran, Samuel Joseph Amouyal, Chaitanya Malaviya +3
Language agents, built on top of language models (LMs), are systems that can interact with complex environments, such as the open web. In this work, we examine whether such agents…
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Luca Soldaini, Rodney Kinney, Akshita Bhagia +33
Information about pretraining corpora used to train the current best-performing language models is seldom discussed: commercial models rarely detail their data, and even open model…
Leveraging Code to Improve In-context Learning for Semantic Parsing
Ben Bogin, Shivanshu Gupta, Peter Clark +1
In-context learning (ICL) is an appealing approach for semantic parsing due to its few-shot nature and improved generalization. However, learning to parse to rare domain-specific l…
Answering Questions by Meta-Reasoning over Multiple Chains of Thought
Ori Yoran, Tomer Wolfson, Ben Bogin +3
Modern systems for multi-hop question answering (QA) typically break questions into a sequence of reasoning steps, termed chain-of-thought (CoT), before arriving at a final answer.…
Diverse Demonstrations Improve In-context Compositional Generalization
Itay Levy, Ben Bogin, Jonathan Berant
In-context learning has shown great success in i.i.d semantic parsing splits, where the training and test sets are drawn from the same distribution. In this setup, models are typic…
Training Vision-Language Models with Less Bimodal Supervision
Elad Segal, Ben Bogin, Jonathan Berant
Standard practice in pretraining multimodal models, such as vision-language models, is to rely on pairs of aligned inputs from both modalities, for example, aligned image-text pair…
Unobserved Local Structures Make Compositional Generalization Hard
Ben Bogin, Shivanshu Gupta, Jonathan Berant
While recent work has convincingly showed that sequence-to-sequence models struggle to generalize to new compositions (termed compositional generalization), little is known on what…
COVR: A test-bed for Visually Grounded Compositional Generalization with real images
Ben Bogin, Shivanshu Gupta, Matt Gardner +1
While interest in models that generalize at test time to new compositions has risen in recent years, benchmarks in the visually-grounded domain have thus far been restricted to syn…
Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data
Moshe Hazoom, Vibhor Malik, Ben Bogin
Most available semantic parsing datasets, comprising of pairs of natural utterances and logical forms, were collected solely for the purpose of training and evaluation of natural l…
Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering
Ben Bogin, Sanjay Subramanian, Matt Gardner +1
Answering questions that involve multi-step reasoning requires decomposing them and using the answers of intermediate steps to reach the final answer. However, state-of-the-art mod…
Obtaining Faithful Interpretations from Compositional Neural Networks
Sanjay Subramanian, Ben Bogin, Nitish Gupta +4
Neural module networks (NMNs) are a popular approach for modeling compositionality: they achieve high accuracy when applied to problems in language and vision, while reflecting the…
Evaluating Models' Local Decision Boundaries via Contrast Sets
Matt Gardner, Yoav Artzi, Victoria Basmova +23
Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluation…
Global Reasoning over Database Structures for Text-to-SQL Parsing
Ben Bogin, Matt Gardner, Jonathan Berant
State-of-the-art semantic parsers rely on auto-regressive decoding, emitting one symbol at a time. When tested against complex databases that are unobserved at training time (zero-…
Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing
Ben Bogin, Matt Gardner, Jonathan Berant
Research on parsing language to SQL has largely ignored the structure of the database (DB) schema, either because the DB was very simple, or because it was observed at both trainin…
Grammar-based Neural Text-to-SQL Generation
Kevin Lin, Ben Bogin, Mark Neumann +2
The sequence-to-sequence paradigm employed by neural text-to-SQL models typically performs token-level decoding and does not consider generating SQL hierarchically from a grammar.…
Emergence of Communication in an Interactive World with Consistent Speakers
Ben Bogin, Mor Geva, Jonathan Berant
Training agents to communicate with one another given task-based supervision only has attracted considerable attention recently, due to the growing interest in developing models fo…
Language Generation with Recurrent Generative Adversarial Networks without Pre-training
Ofir Press, Amir Bar, Ben Bogin +2
Generative Adversarial Networks (GANs) have shown great promise recently in image generation. Training GANs for language generation has proven to be more difficult, because of the…