25 citations · 53 across the 24 of their papers we have counts for
26 papers · 1 filter
A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity Understanding
Dilara Torunoğlu-Selamet, Dogukan Arslan, Rodrigo Wilkens +75
Potentially idiomatic expressions (PIEs) construe meanings inherently tied to the everyday experience of a given language community. As such, they constitute an interesting challen…
Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languages
David Samuel, Lilja Øvrelid, Erik Velldal +1
We propose a post-training method for lower-resource languages that preserves the fluency of language models even when aligned by disfluent reward models. Preference optimization i…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Event-based evaluation of abstractive news summarization
Huiling You, Samia Touileb, Erik Velldal +1
An abstractive summary of a news article contains its most important information in a condensed version. The evaluation of automatically generated summaries by generative language…
NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark
Vladislav Mikhailov, Tita Enstad, David Samuel +4
This paper introduces NorEval, a new and comprehensive evaluation suite for large-scale standardized benchmarking of Norwegian generative language models (LMs). NorEval consists of…
Mixed Feelings: Cross-Domain Sentiment Classification of Patient Feedback
Egil Rønningstad, Lilja Charlotte Storset, Petter Mæhlum +2
Sentiment analysis of patient feedback from the public health domain can aid decision makers in evaluating the provided services. The current paper focuses on free-text comments in…
A Collection of Question Answering Datasets for Norwegian
Vladislav Mikhailov, Petter Mæhlum, Victoria Ovedie Chruickshank Langø +2
This paper introduces a new suite of question answering datasets for Norwegian; NorOpenBookQA, NorCommonSenseQA, NorTruthfulQA, and NRK-Quiz-QA. The data covers a wide range of ski…
Benchmarking Abstractive Summarisation: A Dataset of Human-authored Summaries of Norwegian News Articles
Samia Touileb, Vladislav Mikhailov, Marie Kroka +2
We introduce a dataset of high-quality human-authored summaries of news articles in Norwegian. The dataset is intended for benchmarking the abstractive summarisation capabilities o…
Small Languages, Big Models: A Study of Continual Training on Languages of Norway
David Samuel, Vladislav Mikhailov, Erik Velldal +4
Training large language models requires vast amounts of data, posing a challenge for less widely spoken languages like Norwegian and even more so for truly low-resource languages l…
Compositional Generalization with Grounded Language Models
Sondre Wold, Étienne Simon, Lucas Georges Gabriel Charpentier +3
Grounded language models use external sources of information, such as knowledge graphs, to meet some of the general challenges associated with pre-training. By extending previous w…
It's Difficult to be Neutral -- Human and LLM-based Sentiment Annotation of Patient Comments
Petter Mæhlum, David Samuel, Rebecka Maria Norman +4
Sentiment analysis is an important tool for aggregating patient voices, in order to provide targeted improvements in healthcare services. A prerequisite for this is the availabilit…
JSEEGraph: Joint Structured Event Extraction as Graph Parsing
Huiling You, Samia Touileb, Lilja Øvrelid
We propose a graph-based event extraction framework JSEEGraph that approaches the task of event extraction as general graph parsing in the tradition of Meaning Representation Parsi…
Text-To-KG Alignment: Comparing Current Methods on Classification Tasks
Sondre Wold, Lilja Øvrelid, Erik Velldal
In contrast to large text corpora, knowledge graphs (KG) provide dense and structured representations of factual information. This makes them attractive for systems that supplement…
NorBench -- A Benchmark for Norwegian Language Models
David Samuel, Andrey Kutuzov, Samia Touileb +5
We present NorBench: a streamlined suite of NLP tasks and probes for evaluating Norwegian language models (LMs) on standardized data splits and evaluation metrics. We also introduc…
NorQuAD: Norwegian Question Answering Dataset
Sardana Ivanova, Fredrik Aas Andreassen, Matias Jentoft +2
In this paper we present NorQuAD: the first Norwegian question answering dataset for machine reading comprehension. The dataset consists of 4,752 manually created question-answer p…
Measuring Normative and Descriptive Biases in Language Models Using Census Data
Samia Touileb, Lilja Øvrelid, Erik Velldal
We investigate in this paper how distributions of occupations with respect to gender is reflected in pre-trained language models. Such distributions are not always aligned to norma…
EventGraph at CASE 2021 Task 1: A General Graph-based Approach to Protest Event Extraction
Huiling You, David Samuel, Samia Touileb +1
This paper presents our submission to the 2022 edition of the CASE 2021 shared task 1, subtask 4. The EventGraph system adapts an end-to-end, graph-based semantic parser to the tas…
EventGraph: Event Extraction as Semantic Graph Parsing
Huiling You, David Samuel, Samia Touileb +1
Event extraction involves the detection and extraction of both the event triggers and corresponding event arguments. Existing systems often decompose event extraction into multiple…
Direct parsing to sentiment graphs
David Samuel, Jeremy Barnes, Robin Kurtz +3
This paper demonstrates how a graph-based semantic parser can be applied to the task of structured sentiment analysis, directly predicting sentiment graphs from text. We advance th…
The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization
Ildikó Pilán, Pierre Lison, Lilja Øvrelid +3
We present a novel benchmark and associated evaluation metrics for assessing the performance of text anonymization methods. Text anonymization, defined as the task of editing a tex…
Structured Sentiment Analysis as Dependency Graph Parsing
Jeremy Barnes, Robin Kurtz, Stephan Oepen +2
Structured sentiment analysis attempts to extract full opinion tuples from a text, but over time this task has been subdivided into smaller and smaller sub-tasks, e,g,, target extr…
Large-Scale Contextualised Language Modelling for Norwegian
Andrey Kutuzov, Jeremy Barnes, Erik Velldal +2
We present the ongoing NorLM initiative to support the creation and use of very large contextualised language models for Norwegian (and in principle other Nordic languages), includ…
If you've got it, flaunt it: Making the most of fine-grained sentiment annotations
Jeremy Barnes, Lilja Øvrelid, Erik Velldal
Fine-grained sentiment analysis attempts to extract sentiment holders, targets and polar expressions and resolve the relationship between them, but progress has been hampered by th…
A Systematic Comparison of Architectures for Document-Level Sentiment Classification
Jeremy Barnes, Vinit Ravishankar, Lilja Øvrelid +1
Documents are composed of smaller pieces - paragraphs, sentences, and tokens - that have complex relationships between one another. Sentiment classification models that take into a…