117 citations · 269 across the 39 of their papers we have counts for
24 papers · 1 filter
DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck
Shubham Agarwal, Akshat Gupta, Mike Heddes +21
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fun…
RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards
Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang +9
Deep research agents plan, search, evaluate evidence, and synthesize long-form reports. Training these agents with reinforcement learning (RL) is challenging because they operate o…
Latent Factor Models Meets Instructions: Goal-conditioned Latent Factor Discovery without Task Supervision
Zhouhang Xie, Tushar Khot, Bhavana Dalvi Mishra +4
Instruction-following LLMs have recently allowed systems to discover hidden concepts from a collection of unstructured documents based on a natural language description of the purp…
From Models to Microtheories: Distilling a Model's Topical Knowledge for Grounded Question Answering
Nathaniel Weir, Bhavana Dalvi Mishra, Orion Weller +6
Recent reasoning methods (e.g., chain-of-thought, entailment reasoning) help users understand how language models (LMs) answer a single question, but they do little to reveal the L…
DiscoveryBench: Towards Data-Driven Discovery with Large Language Models
Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal +7
Can the rapid advances in code generation, function calling, and data analysis using large language models (LLMs) help automate the search and verification of hypotheses purely fro…
Naive Bayes-based Context Extension for Large Language Models
Jianlin Su, Murtadha Ahmed, Wenbo +3
Large Language Models (LLMs) have shown promising in-context learning abilities. However, conventional In-Context Learning (ICL) approaches are often impeded by length limitations…
Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic
Nathaniel Weir, Kate Sanders, Orion Weller +8
Recent language models enable new opportunities for structured reasoning with text, such as the construction of intuitive, proof-like textual entailment trees without relying on br…
BaRDa: A Belief and Reasoning Dataset that Separates Factual Accuracy and Reasoning Ability
Peter Clark, Bhavana Dalvi Mishra, Oyvind Tafjord
While there are numerous benchmarks comparing the performance of modern language models (LMs), end-task evaluations often conflate notions of *factual accuracy* ("truth") and *reas…
CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization
Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen +5
Language agents have shown some ability to interact with an external environment, e.g., a virtual world such as ScienceWorld, to perform complex tasks, e.g., growing a plant, witho…
Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTE
Yuling Gu, Yao Fu, Valentina Pyatkin +3
Figurative language (e.g., "he flew like the wind") is challenging to understand, as it is hard to tell what implicit information is being conveyed from the surface form alone. We…
Explaining Answers with Entailment Trees
Bhavana Dalvi, Peter Jansen, Oyvind Tafjord +4
Our goal, in the context of open-domain textual question-answering (QA), is to explain answers by showing the line of reasoning from what is known to the answer, rather than simply…
Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challenge
Sumithra Bhakthavatsalam, Daniel Khashabi, Tushar Khot +6
We present the ARC-DA dataset, a direct-answer ("open response", "freeform") version of the ARC (AI2 Reasoning Challenge) multiple-choice dataset. While ARC has been influential in…
ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language
Oyvind Tafjord, Bhavana Dalvi Mishra, Peter Clark
Transformers have been shown to emulate logical deduction over natural language theories (logical rules expressed in natural language), reliably assigning true/false labels to cand…
A Dataset for Tracking Entities in Open Domain Procedural Text
Niket Tandon, Keisuke Sakaguchi, Bhavana Dalvi Mishra +5
We present the first dataset for tracking state changes in procedural text from arbitrary domains by using an unrestricted (open) vocabulary. For example, in a text describing fog…
What-if I ask you to explain: Explaining the effects of perturbations in procedural text
Dheeraj Rajagopal, Niket Tandon, Bhavana Dalvi +2
We address the task of explaining the effects of perturbations in procedural text, an important test of process comprehension. Consider a passage describing a rabbit's life-cycle:…
Procedural Reading Comprehension with Attribute-Aware Context Flow
Aida Amini, Antoine Bosselut, Bhavana Dalvi Mishra +2
Procedural texts often describe processes (e.g., photosynthesis and cooking) that happen over entities (e.g., light, food). In this paper, we introduce an algorithm for procedural…
Pretrained Language Models for Sequential Sentence Classification
Arman Cohan, Iz Beltagy, Daniel King +2
As a step toward better document-level understanding, we explore classification of a sequence of sentences into their corresponding categories, a task that requires understanding s…
Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text
Bhavana Dalvi Mishra, Niket Tandon, Antoine Bosselut +2
Our goal is to better comprehend procedural text, e.g., a paragraph about photosynthesis, by not only predicting what happens, but why some actions need to happen before others. Ou…
WIQA: A dataset for "What if..." reasoning over procedural text
Niket Tandon, Bhavana Dalvi Mishra, Keisuke Sakaguchi +2
We introduce WIQA, the first large-scale dataset of "What if..." questions over procedural text. WIQA contains three parts: a collection of paragraphs each describing a process, e.…
From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project
Peter Clark, Oren Etzioni, Daniel Khashabi +11
AI has achieved remarkable mastery over games such as Chess, Go, and Poker, and even Jeopardy, but the rich variety of standardized exams has remained a landmark challenge. Even in…
Be Consistent! Improving Procedural Text Comprehension using Label Consistency
Xinya Du, Bhavana Dalvi Mishra, Niket Tandon +4
Our goal is procedural text comprehension, namely tracking how the properties of entities (e.g., their location) change with time given a procedural text (e.g., a paragraph about p…
Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension
Bhavana Dalvi Mishra, Lifu Huang, Niket Tandon +2
We present a new dataset and models for comprehending paragraphs about processes (e.g., photosynthesis), an important genre of text describing a dynamic world. The new dataset, Pro…
A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications
Dongyeop Kang, Waleed Ammar, Bhavana Dalvi +4
Peer reviewing is a central component in the scientific publishing process. We present the first public dataset of scientific peer reviews available for research purposes (PeerRead…
What Happened? Leveraging VerbNet to Predict the Effects of Actions in Procedural Text
Peter Clark, Bhavana Dalvi, Niket Tandon
Our goal is to answer questions about paragraphs describing processes (e.g., photosynthesis). Texts of this genre are challenging because the effects of actions are often implicit…