200 citations · 332 across the 13 of their papers we have counts for
13 papers
Translation Heads: Disentangling meaning from language in LLM-based machine translation
Théo Lasnier, Armel Zebaze, Djamé Seddah +2
Mechanistic Interpretability (MI) seeks to explain how neural networks implement their capabilities, but the scale of Large Language Models (LLMs) has limited prior MI work in Mach…
LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens
Armel Zebaze, Rachel Bawden, Benoît Sagot
Large reasoning models (LRMs) have led to new possibilities in terms of problem-solving, through the devising of a natural language thought process prior to answering a query. Whil…
TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation
Armel Zebaze, Benoît Sagot, Rachel Bawden
LLMs have been shown to perform well in machine translation (MT) with the use of in-context learning (ICL), rivaling supervised models when translating into high-resource languages…
Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation
Armel Zebaze, Benoît Sagot, Rachel Bawden
The ability of generative large language models (LLMs) to perform in-context learning has given rise to a large body of research into how best to prompt models for various natural…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1168
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
Tree of Problems: Improving structured problem solving with compositionality
Armel Zebaze, Benoît Sagot, Rachel Bawden
Large Language Models (LLMs) have demonstrated remarkable performance across multiple tasks through in-context learning. For complex reasoning tasks that require step-by-step think…
In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation
Armel Zebaze, Benoît Sagot, Rachel Bawden
The ability of generative large language models (LLMs) to perform in-context learning has given rise to a large body of research into how best to prompt models for various natural…
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
Terry Yue Zhuo, Minh Chien Vu, Jenny Chim +30
Task automation has been greatly empowered by the recent advances in Large Language Models (LLMs) via Python code, where the tasks ranging from software engineering development to…
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus
Matthieu Futeral, Armel Zebaze, Pedro Ortiz Suarez +5
Multimodal Large Language Models (mLLMs) are trained on a large amount of text-image data. While most mLLMs are trained on caption-like data only, Alayrac et al. (2022) showed that…
StarCoder 2 and The Stack v2: The Next Generation
Anton Lozhkov, Raymond Li, Loubna Ben Allal +63
The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder2. In partnership wi…
Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
Terry Yue Zhuo, Armel Zebaze, Nitchakarn Suppattarachai +4
The high cost of full-parameter fine-tuning (FFT) of Large Language Models (LLMs) has led to a series of parameter-efficient fine-tuning (PEFT) methods. However, it remains unclear…
OctoPack: Instruction Tuning Code Large Language Models
Niklas Muennighoff, Qian Liu, Armel Zebaze +7
Finetuning large language models (LLMs) on instructions leads to vast performance improvements on natural language tasks. We apply instruction tuning using code, leveraging the nat…
StarCoder: may the source be with you!
Raymond Li, Loubna Ben Allal, Yangtian Zi +64
The BigCode community, an open-scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder and StarCoderBase…