140 citations · 165 across the 8 of their papers we have counts for
8 papers
SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain
Pierre Colombo, Telmo Pires, Malik Boudiaf +7
In this paper, we introduce SaulLM-54B and SaulLM-141B, two large language models (LLMs) tailored for the legal sector. These models, which feature architectures of 54 billion and…
SaulLM-7B: A pioneering Large Language Model for Law
Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf +8
In this paper, we introduce SaulLM-7B, a large language model (LLM) tailored for the legal domain. With 7 billion parameters, SaulLM-7B is the first LLM designed explicitly for leg…
One Wide Feedforward is All You Need
Telmo Pessoa Pires, António V. Lopes, Yannick Assogba +1
The Transformer architecture has two main non-embedding components: Attention and the Feed Forward Network (FFN). Attention captures interdependencies between words regardless of t…
Learning Language-Specific Layers for Multilingual Machine Translation
Telmo Pessoa Pires, Robin M. Schmidt, Yi-Hsiu Liao +1
Multilingual Machine Translation promises to improve translation quality between non-English languages. This is advantageous for several reasons, namely lower latency (no need to t…
State Spaces Aren't Enough: Machine Translation Needs Attention
Ali Vardasbi, Telmo Pessoa Pires, Robin M. Schmidt +1
Structured State Spaces for Sequences (S4) is a recently proposed sequence model with successful applications in various tasks, e.g. vision, language modeling, and audio. Thanks to…
Non-Autoregressive Neural Machine Translation: A Call for Clarity
Robin M. Schmidt, Telmo Pires, Stephan Peitz +1
Non-autoregressive approaches aim to improve the inference speed of translation models by only requiring a single forward pass to generate the output sequence instead of iterativel…