7 papers
Coresets Before Score Sets: Evaluation-Unsupervised Prompt Subset Selection for LLM Benchmarks
Jihan Yao, Gantavya Bhatt, Arnav Das +16
We study LLM benchmark coreset selection: selecting a small subset of prompts over multiple benchmarks whose induced model scores and rankings approximate those obtained from the f…
How Much Is a Dataset Worth? Scaling Laws, the Vendi Score, and Matrix Spectral Functions
Jeff A. Bilmes, Gantavya Bhatt, Arnav M. Das
Neural scaling laws appraise data through dataset size, while the Vendi Score uses quantum entropy to measure dataset value. We show both that common neural-scaling-law objectives…
How Many Images Does It Take? Estimating Imitation Thresholds in Text-to-Image Models
Sahil Verma, Royi Rassin, Arnav Das +6
Text-to-image models are trained using large datasets of image-text pairs collected from the internet. These datasets often include copyrighted and private images. Training models…
OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Languages and Modalities
Sahil Verma, Keegan Hines, Jeff Bilmes +4
The emerging capabilities of large language models (LLMs) have sparked concerns about their immediate potential for harmful misuse. The core approach to mitigate these concerns is…
Tilted Sharpness-Aware Minimization
Tian Li, Tianyi Zhou, Jeffrey A. Bilmes
Sharpness-Aware Minimization (SAM) has been demonstrated to improve the generalization performance of overparameterized models by seeking flat minima on the loss landscape through…
COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation
Arnav M. Das, Gantavya Bhatt, Lilly Kumari +2
Retrieval augmentation, the practice of retrieving additional data from large auxiliary pools, has emerged as an effective technique for enhancing model performance in the low-data…