Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12

Abstract

In June 2016, Apple announced that it will deploy differential privacy for some user data collection in order to ensure privacy of user data, even from Apple. The details of Apple's approach remained sparse. Although several patents have since appeared hinting at the algorithms that may be used to achieve differential privacy, they did not include a precise explanation of the approach taken to privacy parameter choice. Such choice and the overall approach to privacy budget use and management are key questions for understanding the privacy protections provided by any deployment of differential privacy. In this work, through a combination of experiments, static and dynamic code analysis of macOS Sierra (Version 10.12) implementation, we shed light on the choices Apple made for privacy budget management. We discover and describe Apple's set-up for differentially private data processing, including the overall data pipeline, the parameters used for differentially private perturbation of each piece of data, and the frequency with which such data is sent to Apple's servers. We find that although Apple's deployment ensures that the (differential) privacy loss per each datum submitted to its servers is $1$ or $2$, the overall privacy loss permitted by the system is significantly higher, as high as $16$ per day for the four initially announced applications of Emojis, New words, Deeplinks and Lookup Hints. Furthermore, Apple renews the privacy budget available every day, which leads to a possible privacy loss of 16 times the number of days since user opt-in to differentially private data collection for those four applications. We advocate that in order to claim the full benefits of differentially private data collection, Apple must give full transparency of its implementation, enable user choice in areas related to privacy loss, and set meaningful defaults on the privacy loss permitted.

References in corpus (1)

• Privacy in Information-Rich Intelligent Infrastructure

Cited by in corpus (26)

• Efficient privacy preservation of big data for accurate data mining
• LDP-IDS: Local Differential Privacy for Infinite Data Streams
• Collecting and Analyzing Multidimensional Data with Local Differential Privacy
• Encode, Shuffle, Analyze Privacy Revisited: Formalizations and Empirical Evaluation
• Pointwise Maximal Leakage
• Reviewing and Improving the Gaussian Mechanism for Differential Privacy
• Differential Privacy for Government Agencies -- Are We There Yet?
• Information-theoretic metrics for Local Differential Privacy protocols
• Robust and Differentially Private Mean Estimation
• Privacy-Preserving Blockchain Based Federated Learning with Differential Data Sharing
• Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings
• An Incentive Mechanism for Trading Personal Data in Data Markets
• LIA: Privacy-Preserving Data Quality Evaluation in Federated Learning Using a Lazy Influence Approximation
• An Analysis of the Deployment of Models Trained on Private Tabular Synthetic Data: Unexpected Surprises
• Differentially Private High Dimensional Sparse Covariance Matrix Estimation
• Privately Publishable Per-instance Privacy
• Effective and Privacy preserving Tabular Data Synthesizing
• Local Differential Privacy in Decentralized Optimization
• Locally private online change point detection
• Noise Variance Optimization in Differential Privacy: A Game-Theoretic Approach Through Per-Instance Differential Privacy
• Nebula: Efficient, Private and Accurate Histogram Estimation
• Privacy-Aware Rejection Sampling
• "I inherently just trust that it works": Investigating Mental Models of Open-Source Libraries for Differential Privacy
• Differential Privacy for Evolving Almost-Periodic Datasets with Continual Linear Queries: Application to Energy Data Privacy
• Privacy-Preserving Public Release of Datasets for Support Vector Machine Classification
• Revenue Attribution on iOS 14 using Conversion Values in F2P Games