10 papers
Uncovering Intervention Opportunities for Suicide Prevention with Language Model Assistants
Jaspreet Ranjit, Hyundong J. Cho, Claire J. Smerdon +5
Warning: This paper discusses topics of suicide and suicidal ideation, which may be distressing to some readers. The National Violent Death Reporting System (NVDRS) documents infor…
Are We Automating the Joy Out of Work? Designing AI to Augment Work, Not Meaning
Jaspreet Ranjit, Ke Zhou, Swabha Swayamdipta +1
Prior work has mapped which workplace tasks are exposed to AI, but less is known about whether workers perceive these tasks as meaningful or as busywork. We examined: (1) which dim…
How Reliable is Language Model Micro-Benchmarking?
Gregory Yauney, Shahzaib Saqib Warraich, Swabha Swayamdipta
Micro-benchmarking offers a solution to the often prohibitive time and cost of language model development: evaluate on a very small subset of existing benchmarks. Can these micro-b…
Sample, Align, Synthesize: Graph-Based Response Synthesis with ConGrs
Sayan Ghosh, Shahzaib Saqib Warraich, Dhruv Tarsadiya +2
Language models can be sampled multiple times to access the distribution underlying their responses, but existing methods cannot efficiently synthesize rich epistemic signals acros…
Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge
Xinyue Cui, Johnny Tian-Zheng Wei, Swabha Swayamdipta +1
Data watermarking in language models injects traceable signals, such as specific token sequences or stylistic patterns, into copyrighted text, allowing copyright holders to track a…
Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplification
Joseph Liu, Yoonsoo Nam, Xinyue Cui +1
Despite the successes of language models, their evaluation remains a daunting challenge for new and existing tasks. We consider the task of text simplification, commonly used to im…