1 paper · 1 filter
Kimia Hamidieh, Lester Mackey, David Alvarez-Melis
The paper defines and measures how mixing data from different domains during language model pretraining can produce synergistic or interfering effects, and shows that accounting fo…