6 papers
Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts
Jiayuan Ye, Vitaly Feldman, Kunal Talwar
Large language models (LLMs) can struggle to memorize factual knowledge in their parameters, often leading to hallucinations and poor performance on knowledge-intensive tasks. In t…
Privacy amplification by random allocation
Vitaly Feldman, Moshe Shenfeld
We consider the privacy amplification properties of a sampling scheme in which a user's data is used in k steps chosen randomly and uniformly from a sequence (or set) of t steps. T…
Enabling Differentially Private Federated Learning for Speech Recognition: Benchmarks, Adaptive Optimizers and Gradient Clipping
Martin Pelikan, Sheikh Shams Azam, Vitaly Feldman +4
While federated learning (FL) and differential privacy (DP) have been extensively studied, their application to automatic speech recognition (ASR) remains largely unexplored due to…
PREAMBLE: Private and Efficient Aggregation via Block Sparse Vectors
Hilal Asi, Vitaly Feldman, Hannah Keller +2
We revisit the problem of secure aggregation of high-dimensional vectors in a two-server system such as Prio. These systems are typically used to aggregate vectors such as gradient…
Instance-Optimality for Private KL Distribution Estimation
Jiayuan Ye, Vitaly Feldman, Kunal Talwar
We study the fundamental problem of estimating an unknown discrete distribution over symbols, given i.i.d. samples from the distribution. We are interested in minimizin…
Local Pan-Privacy for Federated Analytics
Vitaly Feldman, Audra McMillan, Guy N. Rothblum +1
Pan-privacy was proposed by Dwork et al. as an approach to designing a private analytics system that retains its privacy properties in the face of intrusions that expose the system…