752 citations · 820 across the 21 of their papers we have counts for
23 papers
From Text to Source: Results in Detecting Large Language Model-Generated Content
Wissam Antoun, Benoît Sagot, Djamé Seddah
The widespread use of Large Language Models (LLMs), celebrated for their ability to generate human-like text, has raised concerns about misinformation and ethical implications. Add…
Enriching the NArabizi Treebank: A Multifaceted Approach to Supporting an Under-Resourced Language
Arij Riabi, Menel Mahamdi, Djamé Seddah
In this paper we address the scarcity of annotated data for NArabizi, a Romanized form of North African Arabic used mostly on social media, which poses challenges for Natural Langu…
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect?
Wissam Antoun, Virginie Mouilleron, Benoît Sagot +1
Recent advances in natural language processing (NLP) have led to the development of large language models (LLMs) such as ChatGPT. This paper proposes a methodology for developing a…
Data-Efficient French Language Modeling with CamemBERTa
Wissam Antoun, Benoît Sagot, Djamé Seddah
Recent advances in NLP have significantly improved the performance of language models on a variety of tasks. While these advances are largely driven by the availability of large am…
Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models
Syrielle Montariol, Arij Riabi, Djamé Seddah
Zero-shot cross-lingual transfer learning has been shown to be highly challenging for tasks involving a lot of linguistic specificities or when a cultural gap is present between la…
Towards Unsupervised Content Disentanglement in Sentence Representations via Syntactic Roles
Ghazi Felhi, Joseph Le Roux, Djamé Seddah
Linking neural representations to linguistic factors is crucial in order to build and analyze NLP models interpretable by humans. Among these factors, syntactic roles (e.g. subject…
Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs
Ghazi Felhi, Joseph Le Roux, Djamé Seddah
We propose a generative model for text generation, which exhibits disentangled latent representations of syntax and semantics. Contrary to previous work, this model does not need s…
Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora
Hila Gonen, Ganesh Jawahar, Djamé Seddah +1
The problem of comparing two bodies of text and searching for words that differ in their usage between them arises often in digital humanities and computational social science. Thi…
Noisy UGC Translation at the Character Level: Revisiting Open-Vocabulary Capabilities and Robustness of Char-Based Models
José Carlos Rosales Núñez, Guillaume Wisniewski, Djamé Seddah
This work explores the capacities of character-based Neural Machine Translation to translate noisy User-Generated Content (UGC) with a strong focus on exploring the limits of such…
Understanding the Impact of UGC Specificities on Translation Quality
José Carlos Rosales Núñez, Djamé Seddah, Guillaume Wisniewski
This work takes a critical look at the evaluation of user-generated content automatic translation, the well-known specificities of which raise many challenges for MT. Our analyses…
PAGnol: An Extra-Large French Generative Model
Julien Launay, Elena Tommasone, Baptiste Pannier +5
Access to large pre-trained models of varied architectures, in many different languages, is central to the democratization of NLP. We introduce PAGnol, a collection of French GPT m…
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?
Arij Riabi, Benoît Sagot, Djamé Seddah
Recent impressive improvements in NLP, largely based on the success of contextual neural language models, have been mostly demonstrated on at most a couple dozen high-resource lang…
Challenging the Semi-Supervised VAE Framework for Text Classification
Ghazi Felhi, Joseph Le Roux, Djamé Seddah
Semi-Supervised Variational Autoencoders (SSVAEs) are widely used models for data efficient learning. In this paper, we question the adequacy of the standard design of sequence SSV…
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT
Benjamin Muller, Yanai Elazar, Benoît Sagot +1
Multilingual pretrained language models have demonstrated remarkable zero-shot cross-lingual transfer capabilities. Such transfer emerges by fine-tuning on a task of interest in on…
Disentangling semantics in language through VAEs and a certain architectural choice
Ghazi Felhi, Joseph Le Roux, Djamé Seddah
We present an unsupervised method to obtain disentangled representations of sentences that single out semantic content. Using modified Transformers as building blocks, we train a V…
QUACKIE: A NLP Classification Task With Ground Truth Explanations
Yves Rychener, Xavier Renard, Djamé Seddah +2
NLP Interpretability aims to increase trust in model predictions. This makes evaluating interpretability approaches a pressing issue. There are multiple datasets for evaluating NLP…
On the Granularity of Explanations in Model Agnostic NLP Interpretability
Yves Rychener, Xavier Renard, Djamé Seddah +2
Current methods for Black-Box NLP interpretability, like LIME or SHAP, are based on altering the text to interpret by removing words and modeling the Black-Box response. In this pa…
Treebanking User-Generated Content: a UD Based Overview of Guidelines, Corpora and Unified Recommendations
Manuela Sanguinetti, Lauren Cassidy, Cristina Bosco +7
This article presents a discussion on the main linguistic phenomena which cause difficulties in the analysis of user-generated texts found on the web and in social media, and propo…
Controlling the Interaction Between Generation and Inference in Semi-Supervised Variational Autoencoders Using Importance Weighting
Ghazi Felhi, Joseph Leroux, Djamé Seddah
Even though Variational Autoencoders (VAEs) are widely used for semi-supervised learning, the reason why they work remains unclear. In fact, the addition of the unsupervised object…
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models
Benjamin Muller, Antonis Anastasopoulos, Benoît Sagot +1
Transfer learning based on pretraining language models on a large amount of raw data has become a new norm to reach state-of-the-art performance in NLP. Still, it remains unclear h…
Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering
Arij Riabi, Thomas Scialom, Rachel Keraron +3
Coupled with the availability of large scale datasets, deep learning architectures have enabled rapid progress on the Question Answering task. However, most of those datasets are i…
Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi
Benjamin Muller, Benoit Sagot, Djamé Seddah
Building natural language processing systems for non standardized and low resource languages is a difficult challenge. The recent success of large-scale multilingual pretrained lan…
CamemBERT: a Tasty French Language Model
Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez +5
Pretrained language models are now ubiquitous in Natural Language Processing. Despite their success, most available models have either been trained on English data or on the concat…