2 papers
cs.LG2025
Understanding the Gain from Data Filtering in Multimodal Contrastive Learning
Divyansh Pareek, Sewoong Oh, Simon S. Du
The success of modern multimodal representation learning relies on internet-scale datasets. Due to the low quality of a large fraction of raw web data, data curation has become a c…
cs.LG2024
Understanding the Gains from Repeated Self-Distillation
Divyansh Pareek, Simon S. Du, Sewoong Oh
Self-Distillation is a special type of knowledge distillation where the student model has the same architecture as the teacher model. Despite using the same architecture and the sa…