8 papers
PolyFact: Comparing Consistency-Driven Post-training Methods for Cross-Lingual Factual Recall
Jonathan von Rad, Louis Arts, George Burgess +6
Large language models (LLMs) trained predominantly on English data encode substantial world knowledge, yet often fail to express it reliably in other languages, a phenomenon known…
ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws
Xuyang Cao, Qianying Liu, Chuan Xiao +7
In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, namely the \textit{language mixtu…
AIRA_2: Overcoming Bottlenecks in AI Research Agents
Karen Hambardzumyan, Nicolas Baldwin, Edan Toledo +22
Existing research has identified three structural performance bottlenecks in AI research agents: (1) synchronous single-GPU execution constrains sample throughput, limiting the ben…
The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining
Jiandong Shao, Raphael Tang, Crystina Zhang +4
Multilingual large language models achieve impressive cross-lingual performance despite largely monolingual pretraining. While bilingual data in pretraining corpora is widely belie…
AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench
Edan Toledo, Karen Hambardzumyan, Martin Josifoski +22
AI research agents are demonstrating great potential to accelerate scientific progress by automating the design, implementation, and training of machine learning models. We focus o…
SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages?
Senyu Li, Jiayi Wang, Felermino D. M. A. Ali +7
Evaluating machine translation (MT) quality for under-resourced African languages remains a significant challenge, as existing metrics often suffer from limited language coverage a…