4 papers · 1 filter
BenchBrowser: Retrieving Evidence for Evaluating Benchmark Validity
Harshita Diddee, Gregory Yauney, Swabha Swayamdipta +1
Do language model benchmarks actually measure what practitioners intend them to ? High-level metadata is too coarse to convey the granular reality of benchmarks: a "poetry" benchma…
CIE: Controlling Language Model Text Generations Using Continuous Signals
Vinay Samuel, Harshita Diddee, Yiming Zhang +1
Aligning language models (LMs) with user intent is becoming increasingly relevant to enhance user experience. This calls for designing methods that can allow users to control the p…
NoveltyBench: Evaluating Language Models for Humanlike Diversity
Yiming Zhang, Harshita Diddee, Susan Holm +5
Language models have demonstrated remarkable capabilities on standard benchmarks, yet they struggle increasingly from mode collapse, the inability to generate diverse and novel out…
Customizing Large Language Model Generation Style using Parameter-Efficient Finetuning
Xinyue Liu, Harshita Diddee, Daphne Ippolito
One-size-fits-all large language models (LLMs) are increasingly being used to help people with their writing. However, the style these models are trained to write in may not suit a…