4 papers
Where to show Demos in Your Prompt: A Positional Bias of In-Context Learning
Kwesi Cobbina, Tianyi Zhou
In-context learning (ICL) is a critical emerging capability of large language models (LLMs), enabling few-shot learning during inference by including a few demonstrations (demos) i…
ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness
Yijun Liang, Ming Li, Chenrui Fan +7
Color plays an important role in human perception and usually provides critical clues in visual reasoning. However, it is unclear whether and how vision-language models (VLMs) can…
My LLM might Mimic AAE -- But When Should it?
Sandra C. Sandoval, Christabel Acquaye, Kwesi Cobbina +2
We examine the representation of African American English (AAE) in large language models (LLMs), exploring (a) the perceptions Black Americans have of how effective these technolog…
Evaluating the Semantic Profiling Abilities of LLMs for Natural Language Utterances in Data Visualization
Hannah K. Bako, Arshnoor Bhutani, Xinyi Liu +2
Automatically generating data visualizations in response to human utterances on datasets necessitates a deep semantic understanding of the data utterance, including implicit and ex…