3 papers
cs.LG2024
SoftSRV: Learn to Generate Targeted Synthetic Data
Giulia DeSalvo, Jean-Fracois Kagy, Lazaros Karydas +2
We present a novel framework, SoftSRV, that is used to generate targeted synthetic fine-tuning data for improving task-specific model performance. Given a sample from a target dist…
cs.DS2024
GIST: Greedy Independent Set Thresholding for Max-Min Diversification with Submodular Utility
Matthew Fahrbach, Srikumar Ramalingam, Morteza Zadimoghaddam +3
This work studies a novel subset selection problem called max-min diversification with monotone submodular utility (), which has a wide range of applications in mach…
cs.LG2024
SpacTor-T5: Pre-training T5 Models with Span Corruption and Replaced Token Detection
Ke Ye, Heinrich Jiang, Afshin Rostamizadeh +6
Pre-training large language models is known to be extremely resource intensive and often times inefficient, under-utilizing the information encapsulated in the training text sequen…