35 citations · 43 across the 9 of their papers we have counts for
11 papers
Confident Rankings with Fewer Items: Adaptive LLM Evaluation with Continuous Scores
Esma Balkır, Alice Pernthaller, Marco Basaldella +2
Computerized Adaptive Testing (CAT) has proven effective for efficient LLM evaluation on multiple-choice benchmarks, but modern LLM evaluation increasingly relies on generation tas…
Concept-Based Explanations to Test for False Causal Relationships Learned by Abusive Language Classifiers
Isar Nejadgholi, Svetlana Kiritchenko, Kathleen C. Fraser +1
Classifiers tend to learn a false causal relationship between an over-represented concept and a label, which can result in over-reliance on the concept and compromised classificati…
This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Models
Seraphina Goldfarb-Tarrant, Eddie Ungless, Esma Balkir +1
Bias research in NLP seeks to analyse models for social biases, thus helping NLP practitioners uncover, measure, and mitigate social harms. We analyse the body of work that uses pr…
Towards Procedural Fairness: Uncovering Biases in How a Toxic Language Classifier Uses Sentiment Information
Isar Nejadgholi, Esma Balkır, Kathleen C. Fraser +1
Previous works on the fairness of toxic language classifiers compare the output of models with different identity terms as input features but do not consider the impact of other im…
Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models
Esma Balkir, Svetlana Kiritchenko, Isar Nejadgholi +1
Motivations for methods in explainable artificial intelligence (XAI) often include detecting, quantifying and mitigating bias, and contributing to making machine learning models fa…
Does Moral Code Have a Moral Code? Probing Delphi's Moral Philosophy
Kathleen C. Fraser, Svetlana Kiritchenko, Esma Balkir
In an effort to guarantee that machine learning model outputs conform with human moral values, recent work has begun exploring the possibility of explicitly training models to lear…
Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detection
Esma Balkir, Isar Nejadgholi, Kathleen C. Fraser +1
We present a novel feature attribution method for explaining text classifiers, and analyze it in the context of hate speech detection. Although feature attribution models usually p…
Tensors over Semirings for Latent-Variable Weighted Logic Programs
Esma Balkir, Daniel Gildea, Shay Cohen
Semiring parsing is an elegant framework for describing parsers by using semiring weighted logic programs. In this paper we present a generalization of this concept: latent-variabl…
Using Pairwise Occurrence Information to Improve Knowledge Graph Completion on Large-Scale Datasets
Esma Balkir, Masha Naslidnyk, Dave Palfrey +1
Bilinear models such as DistMult and ComplEx are effective methods for knowledge graph (KG) completion. However, they require large batch sizes, which becomes a performance bottlen…
Sentence Entailment in Compositional Distributional Semantics
Esma Balkir, Dimitri Kartsaklis, Mehrnoosh Sadrzadeh
Distributional semantic models provide vector representations for words by gathering co-occurrence frequencies from corpora of text. Compositional distributional models extend thes…
Distributional Sentence Entailment Using Density Matrices
Esma Balkir, Mehrnoosh Sadrzadeh, Bob Coecke
Categorical compositional distributional model of Coecke et al. (2010) suggests a way to combine grammatical composition of the formal, type logical models with the corpus based, e…