4 papers · 1 filter
DiversityMedQA: Assessing Demographic Biases in Medical Diagnosis using Large Language Models
Rajat Rawat, Hudson McBride, Dhiyaan Nirmal +5
As large language models (LLMs) gain traction in healthcare, concerns about their susceptibility to demographic biases are growing. We introduce {DiversityMedQA}, a novel benchmark…
AAVENUE: Detecting LLM Biases on NLU Tasks in AAVE via a Novel Benchmark
Abhay Gupta, Philip Meng, Ece Yurtseven +2
Detecting biases in natural language understanding (NLU) for African American Vernacular English (AAVE) is crucial to developing inclusive natural language processing (NLP) systems…
PathFinder: Guided Search over Multi-Step Reasoning Paths
Olga Golovneva, Sean O'Brien, Ramakanth Pasunuru +4
With recent advancements in large language models, methods like chain-of-thought prompting to elicit reasoning chains have been shown to improve results on reasoning tasks. However…
Contrastive Decoding Improves Reasoning in Large Language Models
Sean O'Brien, Mike Lewis
We demonstrate that Contrastive Decoding -- a simple, computationally light, and training-free text generation method proposed by Li et al 2022 -- achieves large out-of-the-box imp…