3 citations · 5 across the 9 of their papers we have counts for
4 papers · 1 filter
GardenBench: A lightweight, daily evaluation of LLM capabilities
Benedict Cummins-Mburu, Mariana Garcia Mejia, Oscar Heath +4
We introduce GardenBench, a benchmark with 11 test suites that evaluate the capabilities of Large Language Models (LLMs) across a range of tasks. By systematically prompting LLMs w…
Correcting Mode Collapse in Silicon Sampling with Semantic Similarity Rating
Oscar Heath, Rohan Alexander
Silicon sampling refers to the use of Large Language Models (LLMs) to generate responses to surveys. It has shown promise, but tends to generate response distributions with unreali…
Same Prompt, Different Outcomes: Evaluating the Reproducibility of Data Analysis by LLMs
Jiaxin Cui, Rohan Alexander
We systematically evaluate the reproducibility of data analysis conducted by Large Language Models (LLMs). We evaluate two prompting strategies, six models, and four temperature se…
Benchmarking AI Performance on End-to-End Data Science Projects
Evelyn Hughes, Rohan Alexander
Data science is an integrated workflow of technical, analytical, communication, and ethical skills, but current AI benchmarks focus mostly on constituent parts. We test whether AI…