Aligning the Query Space: Greedy Information Projection for Language Model Data Selection

Victor Ye Dong, Kuan-Yun Lee, Jiamei Shuai +3

Data selection for language models is often framed as balancing example quality and diversity. We argue that both are consequences of a more fundamental principle: selected example…