12 citations · 46 across the 25 of their papers we have counts for
30 papers
Retrieval-Augmented LLM Agents: Learning to Learn from Experience
Thomas Palmeira Ferraz, Romain Deffayet, Vassilina Nikoulina +2
While large language models (LLMs) have advanced the development of general-purpose agents, robust generalization to unseen tasks remains challenging. Two common approaches are sup…
DiffLoRA: Differential Low-Rank Adapters for Large Language Models
Alexandre Misrahi, Nadezhda Chirkova, Maxime Louis +1
Differential Transformer has recently been proposed to improve performance in Transformer models by canceling out noise through a denoiser attention mechanism. In this work, we int…
Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generation
Alexandre Misrahi, Nadezhda Chirkova, Maxime Louis +1
Retrieval-Augmented Generation (RAG) enhances LLM factuality, but multi-domain applications face challenges like lack of diverse benchmarks and poor out-of-domain generalization. T…
Provence: efficient and robust context pruning for retrieval-augmented generation
Nadezhda Chirkova, Thibault Formal, Vassilina Nikoulina +1
Retrieval-augmented generation improves various aspects of large language models (LLMs) generation, but suffers from computational overhead caused by long contexts as well as the p…
Retrieval-augmented generation in multilingual settings
Nadezhda Chirkova, David Rau, Hervé Déjean +3
Retrieval-augmented generation (RAG) has recently emerged as a promising solution for incorporating up-to-date or domain-specific knowledge into large language models (LLMs) and im…
Investigating the potential of Sparse Mixtures-of-Experts for multi-domain neural machine translation
Nadezhda Chirkova, Vassilina Nikoulina, Jean-Luc Meunier +1
We focus on multi-domain Neural Machine Translation, with the goal of developing efficient models which can handle data from various domains seen during training and are robust to…
BERGEN: A Benchmarking Library for Retrieval-Augmented Generation
David Rau, Hervé Déjean, Nadezhda Chirkova +4
Retrieval-Augmented Generation allows to enhance Large Language Models with external knowledge. In response to the recent popularity of generative LLMs, many RAG approaches have be…
FrenchToxicityPrompts: a Large Benchmark for Evaluating and Mitigating Toxicity in French Texts
Caroline Brun, Vassilina Nikoulina
Large language models (LLMs) are increasingly popular but are also prone to generating bias, toxic or harmful language, which can have detrimental effects on individuals and commun…
Zero-shot cross-lingual transfer in instruction tuning of large language models
Nadezhda Chirkova, Vassilina Nikoulina
Instruction tuning (IT) is widely used to teach pretrained large language models (LLMs) to follow arbitrary instructions, but is under-studied in multilingual settings. In this wor…
Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative tasks
Nadezhda Chirkova, Vassilina Nikoulina
Zero-shot cross-lingual knowledge transfer enables a multilingual pretrained language model, finetuned on a task in one language, make predictions for this task in other languages.…
Multilingual DistilWhisper: Efficient Distillation of Multi-task Speech Models via Language-Specific Experts
Thomas Palmeira Ferraz, Marcely Zanon Boito, Caroline Brun +1
Whisper is a multitask and multilingual speech model covering 99 languages. It yields commendable automatic speech recognition (ASR) results in a subset of its covered languages, b…
Empirical study of pretrained multilingual language models for zero-shot cross-lingual knowledge transfer in generation
Nadezhda Chirkova, Sheng Liang, Vassilina Nikoulina
Zero-shot cross-lingual knowledge transfer enables the multilingual pretrained language model (mPLM), finetuned on a task in one language, make predictions for this task in other l…
Long-Tail Theory under Gaussian Mixtures
Arman Bolatov, Maxat Tezekbayev, Igor Melnykov +3
We suggest a simple Gaussian mixture model for data generation that complies with Feldman's long tail theory (2020). We demonstrate that a linear classifier cannot decrease the gen…
Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model
Yeskendir Koishekenov, Alexandre Berard, Vassilina Nikoulina
The recently released NLLB-200 is a set of multilingual Neural Machine Translation models that cover 202 languages. The largest model is based on a Mixture of Experts architecture…
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting
Zheng-Xin Yong, Hailey Schoelkopf, Niklas Muennighoff +12
The BLOOM model is a large publicly available multilingual language model, but its pretraining was limited to 46 languages. To extend the benefits of BLOOM to other languages witho…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages
Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard +3
In recent years, multilingual machine translation models have achieved promising performance on low-resource language pairs by sharing information between similar languages, thus e…
What Do Compressed Multilingual Machine Translation Models Forget?
Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard +3
Recently, very large pre-trained models achieve state-of-the-art results in various natural language processing (NLP) tasks, but their size makes it more challenging to apply them…
DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation
Cheonbok Park, Hantae Kim, Ioan Calapodescu +2
Domain Adaptation (DA) of Neural Machine Translation (NMT) model often relies on a pre-trained general NMT model which is adapted to the new domain on a sample of in-domain paralle…
Adapting BigScience Multilingual Model to Unseen Languages
Zheng-Xin Yong, Vassilina Nikoulina
We benchmark different strategies of adding new languages (German and Korean) into the BigScience's pretrained multilingual language model with 1.3 billion parameters that currentl…
Zero-Shot and Few-Shot Classification of Biomedical Articles in Context of the COVID-19 Pandemic
Simon Lupart, Benoit Favre, Vassilina Nikoulina +1
MeSH (Medical Subject Headings) is a large thesaurus created by the National Library of Medicine and used for fine-grained indexing of publications in the biomedical domain. In the…
Speeding Up Entmax
Maxat Tezekbayev, Vassilina Nikoulina, Matthias Gallé +1
Softmax is the de facto standard in modern neural networks for language processing when it comes to normalizing logits. However, by producing a dense probability distribution each…
Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters
Asa Cooper Stickland, Alexandre Bérard, Vassilina Nikoulina
Adapter layers are lightweight, learnable units inserted between transformer layers. Recent work explores using such layers for neural machine translation (NMT), to adapt pre-train…
Efficient Inference for Multilingual Neural Machine Translation
Alexandre Berard, Dain Lee, Stéphane Clinchant +2
Multilingual NMT has become an attractive solution for MT deployment in production. But to match bilingual quality, it comes at the cost of larger and slower models. In this work,…