37 citations · 56 across the 21 of their papers we have counts for
24 papers
ReMova: Fine-tuning LLMs for English to Belarusian translation
Mikita Pilinka, Aliaksandr Kliujeŭ, David Samuel +1
This paper presents a Belarusian-specific data-cleaning pipeline and fine-tuning for English-Belarusian machine translation. Our cleaning pipeline distinguishes itself from others…
Dual-objective Language Models: Training Efficiency Without Overfitting
David Samuel, Lucas Georges Gabriel Charpentier
This paper combines autoregressive and masked-diffusion training objectives without any architectural modifications, resulting in flexible language models that outperform single-ob…
Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languages
David Samuel, Lilja Øvrelid, Erik Velldal +1
We propose a post-training method for lower-resource languages that preserves the fluency of language models even when aligned by disfluent reward models. Preference optimization i…
NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark
Vladislav Mikhailov, Tita Enstad, David Samuel +4
This paper introduces NorEval, a new and comprehensive evaluation suite for large-scale standardized benchmarking of Norwegian generative language models (LMs). NorEval consists of…
An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT)
Laurie Burchell, Ona de Gibert, Nikolay Arefyev +32
Training state-of-the-art large language models requires vast amounts of clean and diverse textual data. However, building suitable multilingual datasets remains a challenge. In th…
Multi-label Scandinavian Language Identification (SLIDE)
Mariia Fedorova, Jonas Sebulon Frydenberg, Victoria Handford +6
Identifying closely related languages at sentence level is difficult, in particular because it is often impossible to assign a sentence to a single language. In this paper, we focu…
GPT or BERT: why not both?
Lucas Georges Gabriel Charpentier, David Samuel
We present a simple way to merge masked language modeling with causal language modeling. This hybrid training objective results in a model that combines the strengths of both model…
BERTs are Generative In-Context Learners
David Samuel
While in-context learning is commonly associated with causal language models, such as GPT, we demonstrate that this capability also 'emerges' in masked language models. Through an…
It's Difficult to be Neutral -- Human and LLM-based Sentiment Annotation of Patient Comments
Petter Mæhlum, David Samuel, Rebecka Maria Norman +4
Sentiment analysis is an important tool for aggregating patient voices, in order to provide targeted improvements in healthcare services. A prerequisite for this is the availabilit…
More Room for Language: Investigating the Effect of Retrieval on Language Models
David Samuel, Lucas Georges Gabriel Charpentier, Sondre Wold
Retrieval-augmented language models pose a promising alternative to standard language modeling. During pretraining, these models search in a corpus of documents for contextually re…
Not all layers are equally as important: Every Layer Counts BERT
Lucas Georges Gabriel Charpentier, David Samuel
This paper introduces a novel modification of the transformer architecture, tailored for the data-efficient pretraining of language models. This aspect is evaluated by participatin…
Mean BERTs make erratic language teachers: the effectiveness of latent bootstrapping in low-resource settings
David Samuel
This paper explores the use of latent bootstrapping, an alternative self-supervision technique, for pretraining language models. Unlike the typical practice of using self-supervisi…
NoCoLA: The Norwegian Corpus of Linguistic Acceptability
Matias Jentoft, David Samuel
While there has been a surge of large language models for Norwegian in recent years, we lack any tool to evaluate their understanding of grammaticality. We present two new Norwegia…
Tokenization with Factorized Subword Encoding
David Samuel, Lilja Øvrelid
In recent years, language models have become increasingly larger and more complex. However, the input representations for these models continue to rely on simple and greedy subword…
NorBench -- A Benchmark for Norwegian Language Models
David Samuel, Andrey Kutuzov, Samia Touileb +5
We present NorBench: a streamlined suite of NLP tasks and probes for evaluating Norwegian language models (LMs) on standardized data splits and evaluation metrics. We also introduc…
BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer
Lucas Georges Gabriel Charpentier, Sondre Wold, David Samuel +1
Retrieval-based language models are increasingly employed in question-answering tasks. These models search in a corpus of documents for relevant information instead of having all f…
Trained on 100 million words and still in shape: BERT meets British National Corpus
David Samuel, Andrey Kutuzov, Lilja Øvrelid +1
While modern masked language models (LMs) are trained on ever larger corpora, we here explore the effects of down-scaling training to a modestly-sized but representative, well-bala…
EventGraph at CASE 2021 Task 1: A General Graph-based Approach to Protest Event Extraction
Huiling You, David Samuel, Samia Touileb +1
This paper presents our submission to the 2022 edition of the CASE 2021 shared task 1, subtask 4. The EventGraph system adapts an end-to-end, graph-based semantic parser to the tas…
EventGraph: Event Extraction as Semantic Graph Parsing
Huiling You, David Samuel, Samia Touileb +1
Event extraction involves the detection and extraction of both the event triggers and corresponding event arguments. Existing systems often decompose event extraction into multiple…
Direct parsing to sentiment graphs
David Samuel, Jeremy Barnes, Robin Kurtz +3
This paper demonstrates how a graph-based semantic parser can be applied to the task of structured sentiment analysis, directly predicting sentiment graphs from text. We advance th…
ÚFAL at MultiLexNorm 2021: Improving Multilingual Lexical Normalization by Fine-tuning ByT5
David Samuel, Milan Straka
We present the winning entry to the Multilingual Lexical Normalization (MultiLexNorm) shared task at W-NUT 2021 (van der Goot et al., 2021a), which evaluates lexical-normalization…
RobeCzech: Czech RoBERTa, a monolingual contextualized language representation model
Milan Straka, Jakub Náplava, Jana Straková +1
We present RobeCzech, a monolingual RoBERTa language representation model trained on Czech data. RoBERTa is a robustly optimized Transformer-based pretraining approach. We show tha…
ÚFAL at MRP 2020: Permutation-invariant Semantic Parsing in PERIN
David Samuel, Milan Straka
We present PERIN, a novel permutation-invariant approach to sentence-to-graph semantic parsing. PERIN is a versatile, cross-framework and language independent architecture for univ…
Meta-learning Extractors for Music Source Separation
David Samuel, Aditya Ganeshan, Jason Naradowsky
We propose a hierarchical meta-learning-inspired model for music source separation (Meta-TasNet) in which a generator model is used to predict the weights of individual extractor m…