9 papers · 1 filter
DiffLoRA: Differential Low-Rank Adapters for Large Language Models
Alexandre Misrahi, Nadezhda Chirkova, Maxime Louis +1
Differential Transformer has recently been proposed to improve performance in Transformer models by canceling out noise through a denoiser attention mechanism. In this work, we int…
Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generation
Alexandre Misrahi, Nadezhda Chirkova, Maxime Louis +1
Retrieval-Augmented Generation (RAG) enhances LLM factuality, but multi-domain applications face challenges like lack of diverse benchmarks and poor out-of-domain generalization. T…
Provence: efficient and robust context pruning for retrieval-augmented generation
Nadezhda Chirkova, Thibault Formal, Vassilina Nikoulina +1
Retrieval-augmented generation improves various aspects of large language models (LLMs) generation, but suffers from computational overhead caused by long contexts as well as the p…
Retrieval-augmented generation in multilingual settings
Nadezhda Chirkova, David Rau, Hervé Déjean +3
Retrieval-augmented generation (RAG) has recently emerged as a promising solution for incorporating up-to-date or domain-specific knowledge into large language models (LLMs) and im…
Investigating the potential of Sparse Mixtures-of-Experts for multi-domain neural machine translation
Nadezhda Chirkova, Vassilina Nikoulina, Jean-Luc Meunier +1
We focus on multi-domain Neural Machine Translation, with the goal of developing efficient models which can handle data from various domains seen during training and are robust to…
BERGEN: A Benchmarking Library for Retrieval-Augmented Generation
David Rau, Hervé Déjean, Nadezhda Chirkova +4
Retrieval-Augmented Generation allows to enhance Large Language Models with external knowledge. In response to the recent popularity of generative LLMs, many RAG approaches have be…