3 papers
cs.LG2024
SFT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
Xinyu Yang, Jixuan Leng, Geyang Guo +5
Current PEFT methods for LLMs can achieve either high quality, efficient training, or scalable serving, but not all three simultaneously. To address this limitation, we investigate…
stat.ML2024
Differentially Private Federated Learning: Servers Trustworthiness, Estimation, and Statistical Inference
Zhe Zhang, Ryumei Nakada, Linjun Zhang
Differentially private federated learning is crucial for maintaining privacy in distributed environments. This paper investigates the challenges of high-dimensional estimation and…
cs.LG2023
Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data
Ryumei Nakada, Halil Ibrahim Gulluk, Zhun Deng +3
Language-supervised vision models have recently attracted great attention in computer vision. A common approach to build such models is to use contrastive learning on paired data a…