2 papers
cs.LG2026
Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining
Shuqi Ke, Giulia Fanti
Continued pretraining is optimized with fixed self-supervised tasks but selected by downstream performance, creating a coarse feedback loop in which practitioners evaluate checkpoi…
cs.LG2025
Characterizing the Training Dynamics of Private Fine-tuning with Langevin diffusion
Shuqi Ke, Charlie Hou, Sewoong Oh +1
We show that differentially private full fine-tuning (DP-FFT) can distort pre-trained backbone features based on both theoretical and empirical results. We identify the cause of th…