4 papers · 1 filter
KoSBi: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Application
Hwaran Lee, Seokhee Hong, Joonsuk Park +3
Large language models (LLMs) learn not only natural text generation abilities but also social biases against different demographic groups from real-world data. This poses a critica…
SQuARe: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created Through Human-Machine Collaboration
Hwaran Lee, Seokhee Hong, Joonsuk Park +10
The potential social harms that large language models pose, such as generating offensive content and reinforcing biases, are steeply rising. Existing works focus on coping with thi…
Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation
Takyoung Kim, Jamin Shin, Young-Ho Kim +2
Most task-oriented dialogue (TOD) benchmarks assume users that know exactly how to use the system by constraining the user behaviors within the system's capabilities via strict use…
Mismatch between Multi-turn Dialogue and its Evaluation Metric in Dialogue State Tracking
Takyoung Kim, Hoonsang Yoon, Yukyung Lee +2
Dialogue state tracking (DST) aims to extract essential information from multi-turn dialogue situations and take appropriate actions. A belief state, one of the core pieces of info…