8 citations · 16 across the 15 of their papers we have counts for
12 papers · 1 filter
Modulated learning for private and distributed regression with just a single sample per client device
Praneeth Vepakomma, Amirhossein Reisizadeh, Samuel Horváth +1
This work focuses on the question of learning from a large number of devices with each device holding only a single sample of data. Several real-world applications exist to this on…
Learning in the Null Space: Small Singular Values for Continual Learning
Cuong Anh Pham, Praneeth Vepakomma, Samuel Horváth
Alleviating catastrophic forgetting while enabling further learning is a primary challenge in continual learning (CL). Orthogonal-based training methods have gained attention for t…
Offline and Online KL-Regularized RLHF under Differential Privacy
Yulian Wu, Rushil Thareja, Praneeth Vepakomma +1
In this paper, we study the offline and online settings of reinforcement learning from human feedback (RLHF) with KL-regularization -- a widely used objective function in large lan…
Power Mechanism: Private Tabular Representation Release for Model Agnostic Consumption
Praneeth Vepakomma, Kaustubh Ponkshe
Traditional collaborative learning approaches are based on sharing of model weights between clients and a server. However, there are advantages to resource efficiency through schem…
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
Zheng Lin, Yuxin Zhang, Zhe Chen +6
Recently, large language models (LLMs) have achieved remarkable breakthroughs, revolutionizing the natural language processing domain and beyond. Due to immense parameter sizes, fi…
Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Study
Kaustubh Ponkshe, Shaan Shah, Raghav Singhal +1
Large Language Models (LLMs) rely on safety alignment to produce socially acceptable responses. However, this behavior is known to be brittle: further fine-tuning, even on benign o…