3 citations · 3 across the 3 of their papers we have counts for
4 papers · 1 filter
The Magic Correlations: Understanding Knowledge Transfer from Pretraining to Supervised Fine-Tuning
Simin Fan, Dimitris Paparas, Natasha Noy +3
Understanding how language model capabilities transfer from pretraining to supervised fine-tuning (SFT) is fundamental to efficient model development and data curation. In this wor…
On Fairness of Low-Rank Adaptation of Large Models
Zhoujie Ding, Ken Ziyu Liu, Pura Peetathawatchai +2
Low-rank adaptation of large models, particularly LoRA, has gained traction due to its computational efficiency. This efficiency, contrasted with the prohibitive costs of full-mode…
Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations
Rylan Schaeffer, Victor Lecomte, Dhruv Bhandarkar Pai +10
Maximum Manifold Capacity Representations (MMCR) is a recent multi-view self-supervised learning (MVSSL) method that matches or surpasses other leading MVSSL methods. MMCR is intri…
Adaptive Compression in Federated Learning via Side Information
Berivan Isik, Francesco Pase, Deniz Gunduz +3
The high communication cost of sending model updates from the clients to the server is a significant bottleneck for scalable federated learning (FL). Among existing approaches, sta…