6 papers · 1 filter
Perceptions of Linguistic Uncertainty by Language Models and Humans
Catarina G Belem, Markelle Kelly, Mark Steyvers +2
_Uncertainty expressions_ such as "probably" or "highly unlikely" are pervasive in human language. While prior work has established that there is population-level agreement in term…
Benchmark Data Repositories for Better Benchmarking
Rachel Longjohn, Markelle Kelly, Sameer Singh +1
In machine learning research, it is common to evaluate algorithms via their performance on standard benchmark datasets. While a growing body of work establishes guidelines for -- a…
Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills
Kolby Nottingham, Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra +3
Large language models (LLMs) have recently been used for sequential decision making in interactive environments. However, leveraging environment reward signals for continual LLM ac…
Are Models Biased on Text without Gender-related Language?
Catarina G Belém, Preethi Seshadri, Yasaman Razeghi +1
Gender bias research has been pivotal in revealing undesirable behaviors in large language models, exposing serious gender stereotypes associated with occupations, and emotions. A…
MisgenderMender: A Community-Informed Approach to Interventions for Misgendering
Tamanna Hossain, Sunipa Dev, Sameer Singh
Content Warning: This paper contains examples of misgendering and erasure that could be offensive and potentially triggering. Misgendering, the act of incorrectly addressing someon…
EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learning
Rajasekhar Reddy Mekala, Yasaman Razeghi, Sameer Singh
Language models are achieving impressive performance on various tasks by aggressively adopting inference-time prompting techniques, such as zero-shot and few-shot prompting. In thi…