173 citations · 449 across the 24 of their papers we have counts for
26 papers
Unmasking the Secret Code: Using Canaries to Show How Masked Context Controls Memorization in SFT
Avi Schwarzschild, Om Thakkar, Sahra Ghalebikesabi +2
Canaries, or synthetic training records containing known secrets, are widely used to assess whether language models reveal information from their training data. But how much do the…
Model Card for OpenAI Privacy Filter
Charles de Bourcy, Sahra Ghalebikesabi, Avi Schwarzschild +22
OpenAI Privacy Filter is a compact, bidirectional token-classification model for detecting and redacting personally identifiable information (PII) and secrets in unstructured text.…
Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices
Congzheng Song, Xinyu Tang
Fine-tuning large language models (LLMs) with backpropagation\textemdash even for a subset of parameters such as LoRA\textemdash can be much more memory-consuming than inference an…
Private Federated Learning In Real World Application -- A Case Study
An Ji, Bortik Bandyopadhyay, Congzheng Song +7
This paper presents an implementation of machine learning model training using private federated learning (PFL) on edge devices. We introduce a novel framework that uses PFL to add…
pfl-research: simulation framework for accelerating research in Private Federated Learning
Filip Granqvist, Congzheng Song, Áine Cahill +7
Federated learning (FL) is an emerging machine learning (ML) training paradigm where clients own their data and collaborate to train a global model, without revealing any data to t…
Momentum Approximation in Asynchronous Private Federated Learning
Tao Yu, Congzheng Song, Jianyu Wang +1
Asynchronous protocols have been shown to improve the scalability of federated learning (FL) with a massive number of clients. Meanwhile, momentum-based methods can achieve the bes…
Population Expansion for Training Language Models with Private Federated Learning
Tatsuki Koga, Congzheng Song, Martin Pelikan +1
Federated learning (FL) combined with differential privacy (DP) offers machine learning (ML) training with distributed devices and with a formal privacy guarantee. With a large pop…
Samplable Anonymous Aggregation for Private Federated Data Analysis
Kunal Talwar, Shan Wang, Audra McMillan +34
We revisit the problem of designing scalable protocols for private statistics and private federated learning when each device holds its private data. Locally differentially private…
Private Federated Statistics in an Interactive Setting
Audra McMillan, Omid Javidbakht, Kunal Talwar +21
Privately learning statistics of events on devices can enable improved user experience. Differentially private algorithms for such problems can benefit significantly from interacti…
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices
Mingbin Xu, Congzheng Song, Ye Tian +10
Federated Learning (FL) is a technique to train models using data distributed across devices. Differential Privacy (DP) provides a formal privacy guarantee for sensitive data. Our…
FLAIR: Federated Learning Annotated Image Repository
Congzheng Song, Filip Granqvist, Kunal Talwar
Cross-device federated learning is an emerging machine learning (ML) paradigm where a large population of devices collectively train an ML model while the data remains on the devic…
Training a Tokenizer for Free with Private Federated Learning
Eugene Bagdasaryan, Congzheng Song, Rogier van Dalen +2
Federated learning with differential privacy, i.e. private federated learning (PFL), makes it possible to train models on private data distributed across users' devices without har…
Adversarial Semantic Collisions
Congzheng Song, Alexander M. Rush, Vitaly Shmatikov
We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and…
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
Roei Schuster, Congzheng Song, Eran Tromer +1
Code autocompletion is an integral feature of modern code editors and IDEs. The latest generation of autocompleters uses neural language models, trained on public open-source code…
Information Leakage in Embedding Models
Congzheng Song, Ananth Raghunathan
Embeddings are functions that map raw input data to low-dimensional vector representations, while preserving important semantic information about the inputs. Pre-training embedding…
Generalized Zero-shot ICD Coding
Congzheng Song, Shanghang Zhang, Najmeh Sadoughi +2
The International Classification of Diseases (ICD) is a list of classification codes for the diagnoses. Automatic ICD coding is in high demand as the manual coding can be labor-int…
Robust Membership Encoding: Inference Attacks and Copyright Protection for Deep Learning
Congzheng Song, Reza Shokri
Machine learning as a service (MLaaS), and algorithm marketplaces are on a rise. Data holders can easily train complex models on their data using third party provided learning code…
Overlearning Reveals Sensitive Attributes
Congzheng Song, Vitaly Shmatikov
"Overlearning" means that a model trained for a seemingly simple objective implicitly learns to recognize attributes and concepts that are (1) not part of the learning objective, a…
Auditing Data Provenance in Text-Generation Models
Congzheng Song, Vitaly Shmatikov
To help enforce data-protection regulations such as GDPR and detect unauthorized uses of personal data, we develop a new \emph{model auditing} technique that helps users check if t…
Exploiting Unintended Feature Leakage in Collaborative Learning
Luca Melis, Congzheng Song, Emiliano De Cristofaro +1
Collaborative machine learning and related techniques such as federated learning allow multiple participants, each with his own training dataset, to build a joint model by training…
Chiron: Privacy-preserving Machine Learning as a Service
Tyler Hunt, Congzheng Song, Reza Shokri +2
Major cloud operators offer machine learning (ML) as a service, enabling customers who have the data but not ML expertise or infrastructure to train predictive models on this data.…
Fooling OCR Systems with Adversarial Text Images
Congzheng Song, Vitaly Shmatikov
We demonstrate that state-of-the-art optical character recognition (OCR) based on deep learning is vulnerable to adversarial images. Minor modifications to images of printed text,…
Kernel Distillation for Fast Gaussian Processes Prediction
Congzheng Song, Yiming Sun
Gaussian processes (GPs) are flexible models that can capture complex structure in large-scale dataset due to their non-parametric nature. However, the usage of GPs in real-world a…
Machine Learning Models that Remember Too Much
Congzheng Song, Thomas Ristenpart, Vitaly Shmatikov
Machine learning (ML) is becoming a commodity. Numerous ML frameworks and services are available to data holders who are not ML experts but want to train predictive models on their…