6 citations · 17 across the 21 of their papers we have counts for
24 papers
Invent a Dataset: Measuring Dataset Generation Abilities with Zero Seed Data
Shivalika Singh, Andrija Djurisic, Gbemileke Onilude +2
Building datasets remains one of the most manual and brittle parts of AI development. In this technical report, we focus on the most extreme but also most prevalent setting real wo…
Tiny Aya: Bridging Scale and Multilingual Depth
Alejandro R. Salamanca, Diana Abagyan, Daniel D'souza +23
Tiny Aya redefines what a small multilingual language model can achieve. Trained on 70 languages and refined through region-aware posttraining, it delivers state-of-the-art in tran…
SimMerge: Learning to Select Merge Operators from Similarity Signals
Oliver Bolton, Aakanksha, Arash Ahmadian +3
Model merging combines multiple models into a single model with aggregated capabilities, making it a powerful tool for large language model (LLM) development. However, scaling mode…
The Art of Asking: Multilingual Prompt Optimization for Synthetic Data
David Mora, Viraat Aryabumi, Wei-Yin Ko +3
Synthetic data has become a cornerstone for scaling large language models, yet its multilingual use remains bottlenecked by translation-based prompts. This strategy inherits Englis…
Verification Limits Code LLM Training
Srishti Gureja, Elena Tommasone, Jingyi He +3
Large language models for code generation increasingly rely on synthetic data, where both problem solutions and verification tests are generated by models. While this enables scala…
When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs
Ammar Khairi, Daniel D'souza, Ye Shen +2
Recent advancements in large language models (LLMs) have shifted focus toward scaling inference-time compute, improving performance without retraining the model. A common approach…
Language Models can perform Single-Utterance Self-Correction of Perturbed Reasoning
Sam Silver, Jimin Sun, Ivan Zhang +2
Large Language Models (LLMs) have demonstrated impressive mathematical reasoning capabilities, yet their performance remains brittle to minor variations in problem description and…
The Multilingual Divide and Its Impact on Global AI Safety
Aidan Peppin, Julia Kreutzer, Alice Schoenauer Sebag +13
Despite advances in large language model capabilities in recent years, a large gap remains in their capabilities and safety performance for many languages beyond a relatively small…
Aya Vision: Advancing the Frontier of Multilingual Multimodality
Saurabh Dash, Yiyang Nan, John Dang +22
Building multimodal language models is fundamentally challenging: it requires aligning vision and language modalities, curating high-quality instruction data, and avoiding the degr…
The Leaderboard Illusion
Shivalika Singh, Yiyang Nan, Alex Wang +10
Measuring progress is fundamental to the advancement of any scientific field. As benchmarks play an increasingly central role, they also grow more susceptible to distortion. Chatbo…
Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
Israfel Salazar, Manuel Fernández Burda, Shayekh Bin Islam +42
The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While mu…
Command A: An Enterprise-Ready Large Language Model
Team Cohere, :, Aakanksha +227
In this report we describe the development of Command A, a powerful large language model purpose-built to excel at real-world enterprise use cases. Command A is an agent-optimised…
Fairness of Deep Ensembles: On the interplay between per-group task difficulty and under-representation
Estanislao Claucich, Sara Hooker, Diego H. Milone +2
Ensembling is commonly regarded as an effective way to improve the general performance of models in machine learning, while also increasing the robustness of predictions. When it c…
Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier
John Dang, Shivalika Singh, Daniel D'souza +42
We introduce the Aya Expanse model family, a new generation of 8B and 32B parameter multilingual language models, aiming to address the critical challenge of developing highly perf…
Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
Shivalika Singh, Angelika Romanou, Clémentine Fourrier +21
Cultural biases in multilingual datasets pose significant challenges for their effectiveness as global benchmarks. These biases stem not only from differences in language but also…
Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning
Aakanksha, Arash Ahmadian, Seraphina Goldfarb-Tarrant +3
Large Language Models (LLMs) have been adopted and deployed worldwide for a broad variety of applications. However, ensuring their safe use remains a significant challenge. Prefere…
M-RewardBench: Evaluating Reward Models in Multilingual Settings
Srishti Gureja, Lester James V. Miranda, Shayekh Bin Islam +7
Reward models (RMs) have driven the state-of-the-art performance of LLMs today by enabling the integration of human feedback into the language modeling process. However, RMs are pr…
Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progress
Ayomide Odumakinde, Daniel D'souza, Pat Verga +2
The use of synthetic data has played a critical role in recent state-of-art breakthroughs. However, overly relying on a single oracle teacher model to generate data has been shown…
LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives
Luísa Shimabucoro, Sebastian Ruder, Julia Kreutzer +2
The widespread adoption of synthetic data raises new questions about how models generating the data can influence other large language models (LLMs) via distilled data. To start, o…
The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm
Aakanksha, Arash Ahmadian, Beyza Ermis +4
A key concern with the concept of "alignment" is the implicit question of "alignment to what?". AI systems are increasingly used across the world, yet safety alignment is often foc…
From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models
Luiza Pozzobon, Patrick Lewis, Sara Hooker +1
To date, toxicity mitigation in language models has almost entirely been focused on single-language settings. As language models embrace multilingual capabilities, it's crucial our…
Elo Uncovered: Robustness and Best Practices in Language Model Evaluation
Meriem Boubdir, Edward Kim, Beyza Ermis +2
In Natural Language Processing (NLP), the Elo rating system, originally designed for ranking players in dynamic games such as chess, is increasingly being used to evaluate Large La…
Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation
Meriem Boubdir, Edward Kim, Beyza Ermis +2
Human evaluation is increasingly critical for assessing large language models, capturing linguistic nuances, and reflecting user preferences more accurately than traditional automa…
Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models
Luiza Pozzobon, Beyza Ermis, Patrick Lewis +1
Considerable effort has been dedicated to mitigating toxicity, but existing methods often require drastic modifications to model parameters or the use of computationally intensive…