9 citations · 34 across the 71 of their papers we have counts for
24 papers · 1 filter
From Models to Microtheories: Distilling a Model's Topical Knowledge for Grounded Question Answering
Nathaniel Weir, Bhavana Dalvi Mishra, Orion Weller +6
Recent reasoning methods (e.g., chain-of-thought, entailment reasoning) help users understand how language models (LMs) answer a single question, but they do little to reveal the L…
DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generation
Miriam Wanner, Benjamin Van Durme, Mark Dredze
The decompose-then-verify strategy for verification of Large Language Model (LLM) generations decomposes claims that are then independently verified. Decontextualization augments t…
Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass
Tong Chen, Hao Fang, Patrick Xia +5
Large language models (LMs) are typically adapted to improve performance on new contexts (\eg text prompts that define new tasks or domains) through fine-tuning or prompting. Howev…
Grounding Partially-Defined Events in Multimodal Data
Kate Sanders, Reno Kriz, David Etter +5
How are we able to learn about complex current events just from short snippets of video? While natural language enables straightforward ways to represent under-specified, partially…
Multi-Field Adaptive Retrieval
Millicent Li, Tongfei Chen, Benjamin Van Durme +1
Document retrieval for tasks such as search and retrieval-augmented generation typically involves datasets that are unstructured: free-form text without explicit internal structure…
MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval
Reno Kriz, Kate Sanders, David Etter +10
Efficiently retrieving and synthesizing information from large-scale multimodal collections has become a critical challenge. However, existing video retrieval datasets suffer from…
Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements
Jingyu Zhang, Ahmed Elgohary, Ahmed Magooda +2
The current paradigm for safety alignment of large language models (LLMs) follows a one-size-fits-all approach: the model refuses to interact with any content deemed unsafe by the…
RATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoning
Dongwei Jiang, Guoxuan Wang, Yining Lu +5
The reasoning steps generated by LLMs might be incomplete, as they mimic logical leaps common in everyday communication found in their pre-training data: underlying rationales are…
Gaps or Hallucinations? Gazing into Machine-Generated Legal Analysis for Fine-grained Text Evaluations
Abe Bohan Hou, William Jurayj, Nils Holzenberger +2
Large Language Models (LLMs) show promise as a writing aid for professionals performing legal analyses. However, LLMs can often hallucinate in this setting, in ways difficult to re…
Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models
Orion Weller, Benjamin Van Durme, Dawn Lawrie +3
Instruction-tuned language models (LM) are able to respond to imperative commands, providing a more natural user interface compared to their base counterparts. In this work, we pre…
Baby Bear: Seeking a Just Right Rating Scale for Scalar Annotations
Xu Han, Felix Yu, Joao Sedoc +1
Our goal is a mechanism for efficiently assigning scalar ratings to each of a large set of elements. For example, "what percent positive or negative is this product review?" When s…
WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment
Jiefu Ou, Arda Uzunoglu, Benjamin Van Durme +1
AI systems make decisions in physical environments through primitive actions or affordances that are accessed via API calls. While deploying AI agents in the real world involves nu…
Core: Robust Factual Precision with Informative Sub-Claim Identification
Zhengping Jiang, Jingyu Zhang, Nathaniel Weir +6
Hallucinations pose a challenge to the application of large language models (LLMs) thereby motivating the development of metrics to evaluate factual precision. We observe that popu…
CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generation
Abe Bohan Hou, Orion Weller, Guanghui Qin +5
Legal professionals need to write analyses that rely on citations to relevant precedents, i.e., previous case decisions. Intelligent systems assisting legal professionals in writin…
RE-AdaptIR: Improving Information Retrieval through Reverse Engineered Adaptation
William Fleshman, Benjamin Van Durme
Large language models (LLMs) fine-tuned for text-retrieval have demonstrated state-of-the-art results across several information retrieval (IR) benchmarks. However, supervised trai…
Learning to Retrieve Iteratively for In-Context Learning
Yunmo Chen, Tongfei Chen, Harsh Jhamtani +4
We introduce iterative retrieval, a novel framework that empowers retrievers to make iterative decisions through policy optimization. Finding an optimal portfolio of retrieved item…
RE-Adapt: Reverse Engineered Adaptation of Large Language Models
William Fleshman, Benjamin Van Durme
We introduce RE-Adapt, an approach to fine-tuning large language models on new domains without degrading any pre-existing instruction-tuning. We reverse engineer an adapter which i…
AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees
William Fleshman, Aleem Khan, Marc Marone +1
Large language models (LLMs) are increasingly capable of completing knowledge intensive tasks by recalling information from a static pretraining corpus. Here we are concerned with…
SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses
Dongwei Jiang, Jingyu Zhang, Orion Weller +3
Can LLMs consistently improve their previous outputs for better results? For this to be true, LLMs would need to be better at discriminating among previously-generated alternatives…
Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data
Jingyu Zhang, Marc Marone, Tianjian Li +2
To trust the fluent generations of large language models (LLMs), humans must be able to verify their correctness against trusted, external sources. Recent efforts, such as providin…
A Closer Look at Claim Decomposition
Miriam Wanner, Seth Ebner, Zhengping Jiang +2
As generated text becomes more commonplace, it is increasingly important to evaluate how well-supported such text is by external knowledge sources. Many approaches for evaluating t…
FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions
Orion Weller, Benjamin Chang, Sean MacAvaney +5
Modern Language Models (LMs) are capable of following long and complex instructions that enable a large and diverse set of user requests. While Information Retrieval (IR) models us…
Dated Data: Tracing Knowledge Cutoffs in Large Language Models
Jeffrey Cheng, Marc Marone, Orion Weller +3
Released Large Language Models (LLMs) are often paired with a claimed knowledge cutoff date, or the dates at which training data was gathered. Such information is crucial for appli…
Tur[k]ingBench: A Challenge Benchmark for Web Agents
Kevin Xu, Yeganeh Kordi, Tanay Nayak +7
Can advanced multi-modal models effectively tackle complex web-based tasks? Such tasks are often found on crowdsourcing platforms, where crowdworkers engage in challenging micro-ta…