12 citations · 46 across the 16 of their papers we have counts for
15 papers · 1 filter
It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs
Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois +7
Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--for instance, they contain litt…
ChocoLlama: Lessons Learned From Teaching Llamas Dutch
Matthieu Meeus, Anthony Rathé, François Remy +3
While Large Language Models (LLMs) have shown remarkable capabilities in natural language understanding and generation, their performance often lags in lower-resource, non-English…
Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLP
François Remy, Pieter Delobelle, Hayastan Avetisyan +3
The development of monolingual language models for low and mid-resource languages continues to be hindered by the difficulty in sourcing high-quality training data. In this study,…
OneLove beyond the field -- A few-shot pipeline for topic and sentiment analysis during the FIFA World Cup in Qatar
Christoph Rauchegger, Sonja Mei Wang, Pieter Delobelle
The FIFA World Cup in Qatar was discussed extensively in the news and on social media. Due to news reports with allegations of human rights violations, there were calls to boycott…
Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Models
Xavier Suau, Pieter Delobelle, Katherine Metcalf +4
An important issue with Large Language Models (LLMs) is their undesired ability to generate toxic language. In this work, we show that the neurons responsible for toxicity can be d…
Tik-to-Tok: Translating Language Models One Token at a Time: An Embedding Initialization Strategy for Efficient Language Adaptation
François Remy, Pieter Delobelle, Bettina Berendt +2
Training monolingual language models for low and mid-resource languages is made challenging by limited and often inadequate pretraining data. In this study, we propose a novel mode…
How Far Can It Go?: On Intrinsic Gender Bias Mitigation for Text Classification
Ewoenam Tokpo, Pieter Delobelle, Bettina Berendt +1
To mitigate gender bias in contextualized language models, different intrinsic mitigation strategies have been proposed, alongside many bias metrics. Considering that the end use o…
RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use
Pieter Delobelle, Thomas Winters, Bettina Berendt
Large transformer-based language models, e.g. BERT and GPT-3, outperform previous architectures on most natural language processing tasks. Such language models are first pre-traine…
FairDistillation: Mitigating Stereotyping in Language Models
Pieter Delobelle, Bettina Berendt
Large pre-trained language models are successfully being used in a variety of tasks, across many languages. With this ever-increasing usage, the risk of harmful side effects also r…
RobBERTje: a Distilled Dutch BERT Model
Pieter Delobelle, Thomas Winters, Bettina Berendt
Pre-trained large-scale language models such as BERT have gained a lot of attention thanks to their outstanding performance on a wide range of natural language tasks. However, due…
Measuring Fairness with Biased Rulers: A Survey on Quantifying Biases in Pretrained Language Models
Pieter Delobelle, Ewoenam Kwaku Tokpo, Toon Calders +1
An increasing awareness of biased patterns in natural language processing resources, like BERT, has motivated many metrics to quantify `bias' and `fairness'. But comparing the resu…
Measuring Shifts in Attitudes Towards COVID-19 Measures in Belgium Using Multilingual BERT
Kristen Scott, Pieter Delobelle, Bettina Berendt
We classify seven months' worth of Belgian COVID-related Tweets using multilingual BERT and relate them to their governments' COVID measures. We classify Tweets by their stated opi…
Dutch Humor Detection by Generating Negative Examples
Thomas Winters, Pieter Delobelle
Detecting if a text is humorous is a hard task to do computationally, as it usually requires linguistic and common sense insights. In machine learning, humor detection is usually m…
RobBERT: a Dutch RoBERTa-based Language Model
Pieter Delobelle, Thomas Winters, Bettina Berendt
Pre-trained language models have been dominating the field of natural language processing in recent years, and have led to significant performance gains for various complex natural…
Time to Take Emoji Seriously: They Vastly Improve Casual Conversational Models
Pieter Delobelle, Bettina Berendt
Graphical emoji are ubiquitous in modern-day online conversations. So is a single thumbs-up emoji able to signify an agreement, without any words. We argue that the current state-o…