56 citations · 91 across the 13 of their papers we have counts for
6 papers · 1 filter
SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories
Ben Bogin, Kejuan Yang, Shashank Gupta +5
Given that Large Language Models (LLMs) have made significant progress in writing code, can they now be used to autonomously reproduce results from research repositories? Such a ca…
Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning
Joongwon Kim, Bhargavi Paranjape, Tushar Khot +1
Language agents perform complex tasks by using tools to execute each step precisely. However, most existing agents are based on proprietary models or designed to target specific ta…
DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents
Peter Jansen, Marc-Alexandre Côté, Tushar Khot +5
Automated scientific discovery promises to accelerate progress across scientific domains. However, developing and evaluating an AI agent's capacity for end-to-end scientific reason…
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
Peter Clark, Isaac Cowhey, Oren Etzioni +4
We present a new question set, text corpus, and baselines assembled to encourage AI research in advanced question answering. Together, these constitute the AI2 Reasoning Challenge…
Question Answering via Integer Programming over Semi-Structured Knowledge
Daniel Khashabi, Tushar Khot, Ashish Sabharwal +3
Answering science questions posed in natural language is an important AI challenge. Answering such questions often requires non-trivial inference and knowledge that goes beyond fac…
Markov Logic Networks for Natural Language Question Answering
Tushar Khot, Niranjan Balasubramanian, Eric Gribkoff +3
Our goal is to answer elementary-level science questions using knowledge extracted automatically from science textbooks, expressed in a subset of first-order logic. Given the incom…