51 citations · 266 across the 32 of their papers we have counts for
7 papers · 1 filter
FinanceBench: A New Benchmark for Financial Question Answering
Pranab Islam, Anand Kannappan, Douwe Kiela +3
FinanceBench is a first-of-its-kind test suite for evaluating the performance of LLMs on open book financial question answering (QA). It comprises 10,231 questions about publicly t…
Leveraging Diffusion Perturbations for Measuring Fairness in Computer Vision
Nicholas Lui, Bryan Chia, William Berrios +2
Computer vision models have been known to encode harmful biases, leading to the potentially unfair treatment of historically marginalized groups, such as people of color. However,…
Anchor Points: Benchmarking Models with Much Fewer Examples
Rajan Vivek, Kawin Ethayarajh, Diyi Yang +1
Modern language models often exhibit powerful but brittle behavior, leading to the development of larger and more diverse benchmarks to reliably assess their behavior. Here, we sug…
Towards Language Models That Can See: Computer Vision Through the LENS of Natural Language
William Berrios, Gautam Mittal, Tristan Thrush +2
We propose LENS, a modular approach for tackling computer vision problems by leveraging the power of large language models (LLMs). Our system uses a language model to reason over o…
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Hugo Laurençon, Lucile Saulnier, Léo Tronchon +9
Large multimodal models trained on natural documents, which interleave images and text, outperform models trained on image-text pairs on various multimodal benchmarks. However, the…
AfroDigits: A Community-Driven Spoken Digit Dataset for African Languages
Chris Chinenye Emezue, Sanchit Gandhi, Lewis Tunstall +10
The advancement of speech technologies has been remarkable, yet its integration with African languages remains limited due to the scarcity of African speech corpora. To address thi…
Investigating Multi-source Active Learning for Natural Language Inference
Ard Snijders, Douwe Kiela, Katerina Margatina
In recent years, active learning has been successfully applied to an array of NLP tasks. However, prior work often assumes that training and test data are drawn from the same distr…