9 citations · 16 across the 8 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.CL2023★ 1 cited
Women Are Beautiful, Men Are Leaders: Gender Stereotypes in Machine Translation and Language Modeling
Matúš Pikuliak, Andrea Hrckova, Stefan Oresko +1
We present GEST -- a new manually created dataset designed to measure gender-stereotypical reasoning in language models and machine translation systems. GEST contains samples for 1…
cs.CL2023★ 1 cited
In-Depth Look at Word Filling Societal Bias Measures
Matúš Pikuliak, Ivana Beňová, Viktor Bachratý
Many measures of societal bias in language models have been proposed in recent years. A popular approach is to use a set of word filling prompts to evaluate the behavior of the lan…
cs.CL2023
Average Is Not Enough: Caveats of Multilingual Evaluation
Matúš Pikuliak, Marián Šimko
This position paper discusses the problem of multilingual evaluation. Using simple statistics, such as average language performance, might inject linguistic biases in favor of domi…