activity
20232026
most citedGlobal MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation

6 citations · 17 across the 21 of their papers we have counts for

collaborators
Showing 2025Show all

10 papers · 1 filter

cs.CL2025

The Art of Asking: Multilingual Prompt Optimization for Synthetic Data

David Mora, Viraat Aryabumi, Wei-Yin Ko +3

Synthetic data has become a cornerstone for scaling large language models, yet its multilingual use remains bottlenecked by translation-based prompts. This strategy inherits Englis…

cs.SE2025

Verification Limits Code LLM Training

Srishti Gureja, Elena Tommasone, Jingyi He +3

Large language models for code generation increasingly rely on synthetic data, where both problem solutions and verification tests are generated by models. While this enables scala…

cs.CL2025

When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs

Ammar Khairi, Daniel D'souza, Ye Shen +2

Recent advancements in large language models (LLMs) have shifted focus toward scaling inference-time compute, improving performance without retraining the model. A common approach…

cs.CL2025

Language Models can perform Single-Utterance Self-Correction of Perturbed Reasoning

Sam Silver, Jimin Sun, Ivan Zhang +2

Large Language Models (LLMs) have demonstrated impressive mathematical reasoning capabilities, yet their performance remains brittle to minor variations in problem description and…

cs.AI2025

The Multilingual Divide and Its Impact on Global AI Safety

Aidan Peppin, Julia Kreutzer, Alice Schoenauer Sebag +13

Despite advances in large language model capabilities in recent years, a large gap remains in their capabilities and safety performance for many languages beyond a relatively small…

cs.CL2025

Aya Vision: Advancing the Frontier of Multilingual Multimodality

Saurabh Dash, Yiyang Nan, John Dang +22

Building multimodal language models is fundamentally challenging: it requires aligning vision and language modalities, curating high-quality instruction data, and avoiding the degr…

cs.AI2025

The Leaderboard Illusion

Shivalika Singh, Yiyang Nan, Alex Wang +10

Measuring progress is fundamental to the advancement of any scientific field. As benchmarks play an increasingly central role, they also grow more susceptible to distortion. Chatbo…

cs.CL2025

Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation

Israfel Salazar, Manuel Fernández Burda, Shayekh Bin Islam +42

The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While mu…

cs.CL2025

Command A: An Enterprise-Ready Large Language Model

Team Cohere, :, Aakanksha +227

In this report we describe the development of Command A, a powerful large language model purpose-built to excel at real-world enterprise use cases. Command A is an agent-optimised…

cs.LG2025

Fairness of Deep Ensembles: On the interplay between per-group task difficulty and under-representation

Estanislao Claucich, Sara Hooker, Diego H. Milone +2

Ensembling is commonly regarded as an effective way to improve the general performance of models in machine learning, while also increasing the robustness of predictions. When it c…