71 citations · 102 across the 10 of their papers we have counts for
12 papers
Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning
Jinlong Liu, Mohammed Bahja, Venelin Kovatchev +1
Evaluating and optimising authorial style in long-form story generation remains challenging because style is often assessed with ad hoc prompting and is frequently conflated with o…
AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons
Shaona Ghosh, Heather Frase, Adina Williams +99
The rapid advancement and deployment of AI systems have created an urgent need for standard safety-evaluation frameworks. This paper introduces AILuminate v1.0, the first comprehen…
Benchmark Transparency: Measuring the Impact of Data on Evaluation
Venelin Kovatchev, Matthew Lease
In this paper we present an exploratory research on quantifying the impact that data distribution has on the performance and evaluation of NLP models. We propose an automated frame…
The State of Human-centered NLP Technology for Fact-checking
Anubrata Das, Houjiang Liu, Venelin Kovatchev +1
Misinformation threatens modern society by promoting distrust in science, changing narratives in public health, heightening social polarization, and disrupting democratic elections…
InferES : A Natural Language Inference Corpus for Spanish Featuring Negation-Based Contrastive and Adversarial Examples
Venelin Kovatchev, Mariona Taulé
In this paper, we present InferES - an original corpus for Natural Language Inference (NLI) in European Spanish. We propose, implement, and analyze a variety of corpus-creating str…
Paraphrasing, textual entailment, and semantic similarity above word level
Venelin Kovatchev
This dissertation explores the linguistic and computational aspects of the meaning relations that can hold between two or more complex linguistic expressions (phrases, clauses, sen…
longhorns at DADC 2022: How many linguists does it take to fool a Question Answering model? A systematic approach to adversarial attacks
Venelin Kovatchev, Trina Chatterjee, Venkata S Govindarajan +9
Developing methods to adversarially challenge NLP systems is a promising avenue for improving both model performance and interpretability. Here, we describe the approach of the tea…
ProtoTEx: Explaining Model Decisions with Prototype Tensors
Anubrata Das, Chitrank Gupta, Venelin Kovatchev +2
We present ProtoTEx, a novel white-box NLP classification architecture based on prototype networks. ProtoTEx faithfully explains model decisions based on prototype tensors that enc…
Finding Pareto Trade-offs in Fair and Accurate Detection of Toxic Speech
Soumyajit Gupta, Venelin Kovatchev, Anubrata Das +2
Optimizing NLP models for fairness poses many challenges. Lack of differentiable fairness measures prevents gradient-based loss training or requires surrogate losses that diverge f…
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann +122
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained…
Can vectors read minds better than experts? Comparing data augmentation strategies for the automated scoring of children's mindreading ability
Venelin Kovatchev, Phillip Smith, Mark Lee +1
In this paper we implement and compare 7 different data augmentation strategies for the task of automatic scoring of children's ability to understand others' thoughts, feelings, an…
"What is on your mind?" Automated Scoring of Mindreading in Childhood and Early Adolescence
Venelin Kovatchev, Phillip Smith, Mark Lee +3
In this paper we present the first work on the automated scoring of mindreading ability in middle childhood and early adolescence. We create MIND-CA, a new corpus of 11,311 questio…