works on
bayesian methods 1data mixture optimization 1domain weighting 1large language models 1pretraining data selection 1

From the 1 of 1 linked paper with an AI index.

collaborators

1 paper

cs.LG2026

Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting

Xiang Yuan, Kaiqing Lei, Zhenyu Jin +3

The paper proposes a Bayesian method that learns optimal domain weights for multi‑domain pre‑training of large language models by inferring a Dirichlet distribution with Gamma prio…