4 papers
Can Small Training Runs Reliably Guide Data Curation? Rethinking Proxy-Model Practice
Jiachen T. Wang, Tong Wu, Kaifeng Lyu +4
Data teams at frontier AI companies routinely train small proxy models to make critical decisions about pretraining data recipes for full-scale training runs. However, the communit…
Proper Dataset Valuation by Pointwise Mutual Information
Shuran Zheng, Xuan Qi, Rui Ray Chen +2
Data plays a central role in advancements in modern artificial intelligence, with high-quality data emerging as a key driver of model performance. This has prompted the development…
Capturing the Temporal Dependence of Training Data Influence
Jiachen T. Wang, Dawn Song, James Zou +2
Traditional data influence estimation methods, like influence function, assume that learning algorithms are permutation-invariant with respect to training data. However, modern tra…
A Survey on Data Markets
Jiayao Zhang, Yuran Bi, Mengye Cheng +15
Data is the new oil of the 21st century. The growing trend of trading data for greater welfare has led to the emergence of data markets. A data market is any mechanism whereby the…