72 citations · 200 across the 11 of their papers we have counts for
11 papers
The Nuts and Bolts of Natural Language to SQL Translation: A Systematic Analysis of Model Pipeline Optimisation Approaches and their Interactions
Filip Klubicka, Vasudevan Nedumpozhimana, Sneha Rautmare +3
In the age of large language models, Natural Language to SQL (NL2SQL) translation remains an open problem with many useful applications. We explore interactions between several NL2…
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
Anya Belz, Craig Thomson, Ehud Reiter +39
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/le…
Idioms, Probing and Dangerous Things: Towards Structural Probing for Idiomaticity in Vector Space
Filip Klubička, Vasudevan Nedumpozhimana, John D. Kelleher
The goal of this paper is to learn more about how idiomatic information is structurally encoded in embeddings, using a structural probing method. We repurpose an existing English v…
Queer In AI: A Case Study in Community-Led Participatory AI
Organizers Of QueerInAI, :, Anaelia Ovalle +48
We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community…
Probing Taxonomic and Thematic Embeddings for Taxonomic Information
Filip Klubička, John D. Kelleher
Modelling taxonomic and thematic relatedness is important for building AI with comprehensive natural language understanding. The goal of this paper is to learn more about how taxon…
Probing with Noise: Unpicking the Warp and Weft of Embeddings
Filip Klubička, John D. Kelleher
Improving our understanding of how information is encoded in vector space can yield valuable interpretability insights. Alongside vector dimensions, we argue that it is possible fo…
Semantic Relatedness and Taxonomic Word Embeddings
Magdalena Kacmajor, John D. Kelleher, Filip Klubicka +1
This paper connects a series of papers dealing with taxonomic word embeddings. It begins by noting that there are different types of semantic relatedness and that different lexical…
Is it worth it? Budget-related evaluation metrics for model selection
Filip Klubička, Giancarlo D. Salton, John D. Kelleher
Creating a linguistic resource is often done by using a machine learning model that filters the content that goes through to a human annotator, before going into the final resource…
Examining a hate speech corpus for hate speech detection and popularity prediction
Filip Klubička, Raquel Fernández
As research on hate speech becomes more and more relevant every day, most of it is still focused on hate speech detection. By attempting to replicate a hate speech detection experi…
Quantitative Fine-Grained Human Evaluation of Machine Translation Systems: a Case Study on English to Croatian
Filip Klubička, Antonio Toral, Víctor M. Sánchez-Cartagena
This paper presents a quantitative fine-grained manual evaluation approach to comparing the performance of different machine translation (MT) systems. We build upon the well-establ…
Fine-grained human evaluation of neural versus phrase-based machine translation
Filip Klubička, Antonio Toral, Víctor M. Sánchez-Cartagena
We compare three approaches to statistical machine translation (pure phrase-based, factored phrase-based and neural) by performing a fine-grained manual evaluation via error annota…