368 citations · 1.5k across the 66 of their papers we have counts for
74 papers
SHIP: Structured and Hierarchical Insight with Privacy
Kai Chen, Guisi Ni, Albert Cheu +4
Extracting insights from textual data while protecting user privacy is increasingly important. Existing methods produce private insights in diverse forms for different analytical p…
Advancing the State-of-the-Art in Empirical Privacy Auditing
Nicole Mitchell, Galen Andrew, Arun Ganesh +2
Parameter-efficient fine-tuning of large language models (LLMs) can exhibit problematic memorization of individual training examples. Empirical privacy auditing (EPA) quantifies th…
CLIOPATRA: Extracting Private Information from LLM Insights
Meenatchi Sundaram Muthu Selva Annamalai, Emiliano De Cristofaro, Peter Kairouz
The widespread adoption of AI assistants has encouraged the development of privacy-aware platforms designed to extract insights from real-world usage. Their privacy protections pri…
How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy
Natalia Ponomareva, Zheng Xu, H. Brendan McMahan +12
High quality data is needed to unlock the full potential of AI for end users. However finding new sources of such data is getting harder: most publicly-available human generated da…
Toward provably private analytics and insights into GenAI use
Albert Cheu, Artem Lagzdin, Brett McLarnon +8
Large-scale systems that compute analytics over a fleet of devices must achieve high privacy and security standards while also meeting data quality, usability, and resource efficie…
ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Control
Yuzheng Hu, Ryan McKenna, Da Yu +4
Generating high-quality synthetic text under differential privacy (DP) is critical for training and evaluating language models without compromising user privacy. Prior work on synt…
Urania: Differentially Private Insights into AI Use
Daogao Liu, Edith Cohen, Badih Ghazi +8
We introduce , a novel framework for generating insights about LLM chatbot interactions with rigorous differential privacy (DP) guarantees. The framework employs a private…
Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?
Marika Swanberg, Ryan McKenna, Edo Roth +2
Differentially private (DP) synthetic data is a versatile tool for enabling the analysis of private data. Recent advancements in large language models (LLMs) have inspired a number…
Mayfly: Private Aggregate Insights from Ephemeral Streams of On-Device User Data
Christopher Bian, Albert Cheu, Stanislav Chiknavaryan +12
This paper introduces Mayfly, a federated analytics approach enabling aggregate queries over ephemeral on-device data streams without central persistence of sensitive user data. Ma…
Secure Stateful Aggregation: A Practical Protocol with Applications in Differentially-Private Federated Learning
Marshall Ball, James Bell-Clark, Adria Gascon +3
Recent advances in differentially private federated learning (DPFL) algorithms have found that using correlated noise across the rounds of federated learning (DP-FTRL) yields prova…
Federated Learning in Practice: Reflections and Projections
Katharine Daly, Hubert Eichner, Peter Kairouz +3
Federated Learning (FL) is a machine learning technique that enables multiple entities to collaboratively learn a shared model without exchanging their local data. Over the past de…
Randomization Techniques to Mitigate the Risk of Copyright Infringement
Wei-Ning Chen, Peter Kairouz, Sewoong Oh +1
In this paper, we investigate potential randomization approaches that can complement current practices of input-based methods (such as licensing data and prompt filtering) and outp…
Releasing Large-Scale Human Mobility Histograms with Differential Privacy
Christopher Bian, Albert Cheu, Yannis Guzman +4
Environmental Insights Explorer (EIE) is a Google product that reports aggregate statistics about human mobility, including various methods of transit used by people across roughly…
Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition
Eleni Triantafillou, Peter Kairouz, Fabian Pedregosa +12
We present the findings of the first NeurIPS competition on unlearning, which sought to stimulate the development of novel algorithms and initiate discussions on formal and robust…
Improved Communication-Privacy Trade-offs in Mean Estimation under Streaming Differential Privacy
Wei-Ning Chen, Berivan Isik, Peter Kairouz +3
We study mean estimation under central differential privacy and communication constraints, and address two key challenges: firstly, existing mean estimation schemes that simu…
AirGapAgent: Protecting Privacy-Conscious Conversational Agents
Eugene Bagdasarian, Ren Yi, Sahra Ghalebikesabi +5
The growing use of large language model (LLM)-based conversational agents to manage sensitive user data raises significant privacy concerns. While these agents excel at understandi…
Private federated discovery of out-of-vocabulary words for Gboard
Ziteng Sun, Peter Kairouz, Haicheng Sun +2
The vocabulary of language models in Gboard, Google's keyboard application, plays a crucial role for improving user experience. One way to improve the vocabulary is to discover fre…
Can LLMs get help from other LLMs without revealing private information?
Florian Hartmann, Duc-Hieu Tran, Peter Kairouz +2
Cascades are a common type of machine learning systems in which a large, remote model can be queried if a local model is not able to accurately label a user's data by itself. Servi…
Confidential Federated Computations
Hubert Eichner, Daniel Ramage, Kallista Bonawitz +11
Federated Learning and Analytics (FLA) have seen widespread adoption by technology platforms for processing sensitive on-device data. However, basic FLA systems have privacy limita…
Privacy-Preserving Instructions for Aligning Large Language Models
Da Yu, Peter Kairouz, Sewoong Oh +1
Service providers of large language model (LLM) applications collect user instructions in the wild and use them in further aligning LLMs with users' intentions. These instructions,…
User Inference Attacks on Large Language Models
Nikhil Kandpal, Krishna Pillutla, Alina Oprea +3
Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications. In this paper, we study the privacy implications of f…
Federated Heavy Hitter Recovery under Linear Sketching
Adria Gascon, Peter Kairouz, Ziteng Sun +1
Motivated by real-life deployments of multi-round federated analytics with secure aggregation, we investigate the fundamental communication-accuracy tradeoffs of the heavy hitter d…
Private Federated Learning with Autotuned Compression
Enayat Ullah, Christopher A. Choquette-Choo, Peter Kairouz +1
We propose new techniques for reducing communication in private federated learning without the need for setting or tuning compression rates. Our on-the-fly methods automatically ad…
Private Federated Learning in Gboard
Yuanbo Zhang, Daniel Ramage, Zheng Xu +3
This white paper describes recent advances in Gboard(Google Keyboard)'s use of federated learning, DP-Follow-the-Regularized-Leader (DP-FTRL) algorithm, and secure aggregation tech…