69 citations · 113 across the 5 of their papers we have counts for
5 papers
Final Report on MITRE Evaluations for the DARPA Big Mechanism Program
Matthew Peterson, Tonia Korves, Christopher Garay +2
This report presents the evaluation approach developed for the DARPA Big Mechanism program, which aimed at developing computer systems that will read research papers, integrate the…
Hallmarks of Human-Machine Collaboration: A framework for assessment in the DARPA Communicating with Computers Program
Robyn Kozierok, John Aberdeen, Cheryl Clark +6
There is a growing desire to create computer systems that can communicate effectively to collaborate with humans on complex, open-ended activities. Assessing these systems presents…
Evaluation of text data mining for database curation: lessons learned from the KDD Challenge Cup
Alexander S. Yeh, Lynette Hirschman, Alexander A. Morgan
MOTIVATION: The biological literature is a major repository of knowledge. Many biological databases draw much of their content from a careful curation of this literature. However,…
How to Evaluate your Question Answering System Every Day and Still Get Real Work Done
Eric Breck, John D. Burger, Lisa Ferro +4
In this paper, we report on Qaviar, an experimental automated evaluation system for question answering applications. The goal of our research was to find an automatically calculate…
Automating Coreference: The Role of Annotated Training Data
Lynette Hirschman, Patricia Robinson, John Burger +1
We report here on a study of interannotator agreement in the coreference task as defined by the Message Understanding Conference (MUC-6 and MUC-7). Based on feedback from annotator…