Publications (19)
Memorization in In-Context Learning
Shahriar Golchin, Mihai Surdeanu, Steven Bethard +2
In-context learning (ICL) has proven to be an effective strategy for improving the performance of large language models (LLMs) with no additional training. However, the exact mecha…
Clinical TempEval
Steven Bethard, Leon Derczynski, James Pustejovsky +1
We describe the Clinical TempEval task which is currently in preparation for the SemEval-2015 evaluation exercise. This task involves identifying and describing events, times and t…
AlignSAE: Concept-Aligned Sparse Autoencoders
Minglai Yang, Xinyu Guo, Zhengliang Shi +4
Large Language Models (LLMs) encode factual knowledge within hidden parametric spaces that are difficult to inspect or control. While Sparse Autoencoders (SAEs) can decompose hidde…
TEAM-Atreides at SemEval-2022 Task 11: On leveraging data augmentation and ensemble to recognize complex Named Entities in Bangla
Nazia Tasnim, Md. Istiak Hossain Shihab, Asif Shahriyar Sushmit +2
Many areas, such as the biological and healthcare domain, artistic works, and organization names, have nested, overlapping, discontinuous entity mentions that may even be syntactic…
Improving Toponym Resolution with Better Candidate Generation, Transformer-based Reranking, and Two-Stage Resolution
Zeyu Zhang, Steven Bethard
Geocoding is the task of converting location mentions in text into structured data that encodes the geospatial semantics. We propose a new architecture for geocoding, GeoNorm. GeoN…
LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment
Lingyao Li, Deyi Li, Chen Chen +9
Large language models (LLMs) are increasingly deployed across healthcare applications, including clinical documentation, diagnostic reasoning, medicine recommendation, and medical…
Fusing Temporal Graphs into Transformers for Time-Sensitive Question Answering
Xin Su, Phillip Howard, Nagib Hakim +1
Answering time-sensitive questions from long documents requires temporal reasoning over the times in questions and documents. An important open question is whether large language m…
Explainable Verbal Reasoner Plus (EVR+): A Natural Language Reasoning Framework that Supports Diverse Compositional Reasoning
Zhengzhong Liang, Zeyu Zhang, Steven Bethard +1
Languages models have been successfully applied to a variety of reasoning tasks in NLP, yet the language models still suffer from compositional generalization. In this paper we pre…
Identifying Task Groupings for Multi-Task Learning Using Pointwise V-Usable Information
Yingya Li, Timothy Miller, Steven Bethard +1
The success of multi-task learning can depend heavily on which tasks are grouped together. Naively grouping all tasks or a random set of tasks can result in negative transfer, with…
Semi-Structured Chain-of-Thought: Integrating Multiple Sources of Knowledge for Improved Language Model Reasoning
Xin Su, Tiep Le, Steven Bethard +1
An important open question in the use of large language models for knowledge-intensive tasks is how to effectively integrate knowledge from three sources: the model's parametric me…
We need to talk about random seeds
Steven Bethard
Modern neural network libraries all take as a hyperparameter a random seed, typically used to determine the initial state of the model parameters. This opinion piece argues that th…
Better Retrieval May Not Lead to Better Question Answering
Zhengzhong Liang, Tushar Khot, Steven Bethard +2
Considerable progress has been made recently in open-domain question answering (QA) problems, which require Information Retrieval (IR) and Reading Comprehension (RC). A popular app…
Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering
Vikas Yadav, Steven Bethard, Mihai Surdeanu
We propose an unsupervised strategy for the selection of justification sentences for multi-hop question answering (QA) that (a) maximizes the relevance of the selected sentences, (…
A Semantic Parsing Framework for End-to-End Time Normalization
Xin Su, Sungduk Yu, Phillip Howard +1
Time normalization is the task of converting natural language temporal expressions into machine-readable representations. It underpins many downstream applications in information r…
A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
Vikas Yadav, Steven Bethard
Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc. NER systems have been studied and develope…
Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering
Vikas Yadav, Steven Bethard, Mihai Surdeanu
Evidence retrieval is a critical stage of question answering (QA), necessary not only to improve performance, but also to explain the decisions of the corresponding QA method. We i…
Transformer-Based Temporal Information Extraction and Application: A Review
Xin Su, Phillip Howard, Steven Bethard
Temporal information extraction (IE) aims to extract structured temporal information from unstructured text, thereby uncovering the implicit timelines within. This technique is app…
Predicting engagement in online social networks: Challenges and opportunities
Farig Sadeque, Steven Bethard
Since the introduction of social media, user participation or engagement has received little research attention. In this survey article, we establish the notion of participation in…
Improving Implicit Semantic Role Labeling by Predicting Semantic Frame Arguments
Quynh Ngoc Thi Do, Steven Bethard, Marie-Francine Moens
Implicit semantic role labeling (iSRL) is the task of predicting the semantic roles of a predicate that do not appear as explicit arguments, but rather regard common sense knowledg…