14 citations · 49 across the 26 of their papers we have counts for
6 papers · 1 filter
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning
Kaustubh Ponkshe, Raghav Singhal, Eduard Gorbunov +3
Low-rank adapters have become standard for efficiently fine-tuning large language models, but they often fall short of achieving the performance of full fine-tuning. We propose a m…
FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Foundation Models
Raghav Singhal, Kaustubh Ponkshe, Praneeth Vepakomma
Low-Rank Adaptation (LoRA) is a popular technique for efficient fine-tuning of foundation models. However, applying LoRA in federated learning environments, where data is distribut…
Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMix
Seungeun Oh, Sihun Baek, Jihong Park +5
In computer vision, the vision transformer (ViT) has increasingly superseded the convolutional neural network (CNN) for improved accuracy and robustness. However, ViT's large model…
SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
Zheng Lin, Xuanjie Hu, Yuxin Zhang +6
The scalability of large language models (LLMs) in handling high-complexity models and large-scale datasets has led to tremendous successes in pivotal domains. While there is an ur…
DecentNeRFs: Decentralized Neural Radiance Fields from Crowdsourced Images
Zaid Tasneem, Akshat Dave, Abhishek Singh +4
Neural radiance fields (NeRFs) show potential for transforming images captured worldwide into immersive 3D visual experiences. However, most of this captured visual data remains si…
DAVED: Data Acquisition via Experimental Design for Data Markets
Charles Lu, Baihe Huang, Sai Praneeth Karimireddy +3
The acquisition of training data is crucial for machine learning applications. Data markets can increase the supply of data, particularly in data-scarce domains such as healthcare,…