13 papers
Modulated learning for private and distributed regression with just a single sample per client device
Praneeth Vepakomma, Amirhossein Reisizadeh, Samuel Horváth +1
This work focuses on the question of learning from a large number of devices with each device holding only a single sample of data. Several real-world applications exist to this on…
Combinatorial Privacy: Private Multi-Party Bitstream Grand Sum by Hiding in Birkhoff Polytopes
Praneeth Vepakomma
We introduce PolyVeil, a protocol for private Boolean summation across clients that encodes private bits as permutation matrices in the Birkhoff polytope. A two-layer architect…
Learning in the Null Space: Small Singular Values for Continual Learning
Cuong Anh Pham, Praneeth Vepakomma, Samuel Horváth
Alleviating catastrophic forgetting while enabling further learning is a primary challenge in continual learning (CL). Orthogonal-based training methods have gained attention for t…
ABBA-Adapters: Efficient and Expressive Fine-Tuning of Foundation Models
Raghav Singhal, Kaustubh Ponkshe, Rohit Vartak +1
Large Language Models have demonstrated strong performance across a wide range of tasks, but adapting them efficiently to new domains remains a key challenge. Parameter-Efficient F…
Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Study
Kaustubh Ponkshe, Shaan Shah, Raghav Singhal +1
Large Language Models (LLMs) rely on safety alignment to produce socially acceptable responses. However, this behavior is known to be brittle: further fine-tuning, even on benign o…
DP-Fusion: Token-Level Differentially Private Inference for Large Language Models
Rushil Thareja, Preslav Nakov, Praneeth Vepakomma +1
Large language models (LLMs) do not preserve privacy at inference-time. The LLM's outputs can inadvertently reveal information about the model's context, which presents a privacy c…