65 citations · 176 across the 25 of their papers we have counts for
28 papers
A Functional Data Perspective and Baseline On Multi-Layer Out-of-Distribution Detection
Eduardo Dadalto, Pierre Colombo, Guillaume Staerman +2
A key feature of out-of-distribution (OOD) detection is to exploit a trained neural network by extracting statistical patterns and relationships through the multi-layer classifier…
Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks
Anas Himmi, Ekhine Irurozki, Nathan Noiry +2
The evaluation of natural language processing (NLP) systems is crucial for advancing the field, but current benchmarking approaches often assume that all systems have scores availa…
Hallucinations in Large Multilingual Translation Models
Nuno M. Guerreiro, Duarte Alves, Jonas Waldendorf +4
Large-scale multilingual machine translation systems have demonstrated remarkable ability to translate directly between numerous languages, making them increasingly appealing for r…
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Hugo Laurençon, Lucile Saulnier, Thomas Wang +51
As language models grow ever larger, the need for large-scale high-quality text datasets has never been more pressing, especially in multilingual settings. The BigScience workshop,…
Unsupervised Layer-wise Score Aggregation for Textual OOD Detection
Maxime Darrin, Guillaume Staerman, Eduardo Dadalto Câmara Gomes +3
Out-of-distribution (OOD) detection is a rapidly growing field due to new robustness and security requirements driven by an increased number of AI-based systems. Existing OOD textu…
Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation
Nuno M. Guerreiro, Pierre Colombo, Pablo Piantanida +1
Neural machine translation (NMT) has become the de-facto standard in real-world machine translation applications. However, NMT models can unpredictably produce severely pathologica…
Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data
Maxime Darrin, Pablo Piantanida, Pierre Colombo
Implementing effective control mechanisms to ensure the proper functioning and security of deployed NLP models, from translation to chatbots, is essential. A key ingredient to ensu…
Beyond Mahalanobis-Based Scores for Textual OOD Detection
Pierre Colombo, Eduardo D. C. Gomes, Guillaume Staerman +2
Deep learning methods have boosted the adoption of NLP systems in real-life applications. However, they turn out to be vulnerable to distribution shifts over time which may cause s…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation
Cyril Chhun, Pierre Colombo, Chloé Clavel +1
Research on Automatic Story Generation (ASG) relies heavily on human and automatic evaluation. However, there is no consensus on which human evaluation criteria to use, and no anal…
The Glass Ceiling of Automatic Evaluation in Natural Language Generation
Pierre Colombo, Maxime Peyrard, Nathan Noiry +2
Automatic evaluation metrics capable of replacing human judgments are critical to allowing fast development of new methods. Thus, numerous research efforts have focused on crafting…
Learning Disentangled Textual Representations via Statistical Measures of Similarity
Pierre Colombo, Guillaume Staerman, Nathan Noiry +1
When working with textual data, a natural application of disentangled representations is fair classification where the goal is to make predictions without being biased (or influenc…
A Differential Entropy Estimator for Training Neural Networks
Georg Pichler, Pierre Colombo, Malik Boudiaf +2
Mutual Information (MI) has been widely used as a loss regularizer for training neural networks. This has been particularly effective when learn disentangled or compressed represen…
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann +122
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained…
InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation
Pierre Colombo, Chloe Clavel, Pablo Piantanida
Assessing the quality of natural language generation systems through human annotation is very expensive. Additionally, human annotation campaigns are time-consuming and include non…
Beam Search with Bidirectional Strategies for Neural Response Generation
Pierre Colombo, Chouchang Yang, Giovanna Varni +1
Sequence-to-sequence neural networks have been widely used in language-based applications as they have flexible capabilities to learn various language models. However, when seeking…
Improving Multimodal fusion via Mutual Dependency Maximisation
Pierre Colombo, Emile Chapuis, Matthieu Labeau +1
Multimodal sentiment analysis is a trending area of research, and the multimodal fusion is one of its most active topic. Acknowledging humans communicate through a variety of chann…
Code-switched inspired losses for generic spoken dialog representations
Emile Chapuis, Pierre Colombo, Matthieu Labeau +1
Spoken dialog systems need to be able to handle both multiple languages and multilinguality inside a conversation (\textit{e.g} in case of code-switching). In this work, we introdu…
Automatic Text Evaluation through the Lens of Wasserstein Barycenters
Pierre Colombo, Guillaume Staerman, Chloe Clavel +1
A new metric \texttt{BaryScore} to evaluate text generation based on deep contextualized embeddings e.g., BERT, Roberta, ELMo) is introduced. This metric is motivated by a new fram…
A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations
Pierre Colombo, Chloe Clavel, Pablo Piantanida
Learning disentangled representations of textual data is essential for many natural language tasks such as fair classification, style transfer and sentence generation, among others…
A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions
Guillaume Staerman, Pavlo Mozharovskyi, Pierre Colombo +2
The design of a metric between probability distributions is a longstanding problem motivated by numerous applications in Machine Learning. Focusing on continuous probability distri…
The importance of fillers for text representations of speech transcripts
Tanvi Dinkar, Pierre Colombo, Matthieu Labeau +1
While being an essential component of spoken language, fillers (e.g."um" or "uh") often remain overlooked in Spoken Language Understanding (SLU) tasks. We explore the possibility o…
Hierarchical Pre-training for Sequence Labelling in Spoken Dialog
Emile Chapuis, Pierre Colombo, Matteo Manica +2
Sequence labelling tasks like Dialog Act and Emotion/Sentiment identification are a key component of spoken dialog systems. In this work, we propose a new approach to learn generic…
Heavy-tailed Representations, Text Polarity Classification & Data Augmentation
Hamid Jalalzai, Pierre Colombo, Chloé Clavel +4
The dominant approaches to text representation in natural language rely on learning embeddings on massive corpora which have convenient properties such as compositionality and dist…