Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
SoftSRV: Learn to Generate Targeted Synthetic Data
Giulia DeSalvo, Jean-Fracois Kagy, Lazaros Karydas +2
We present a novel framework, SoftSRV, that is used to generate targeted synthetic fine-tuning data for improving task-specific model performance. Given a sample from a target dist…
cs.LG2024
SpacTor-T5: Pre-training T5 Models with Span Corruption and Replaced Token Detection
Ke Ye, Heinrich Jiang, Afshin Rostamizadeh +6
Pre-training large language models is known to be extremely resource intensive and often times inefficient, under-utilizing the information encapsulated in the training text sequen…