ON THE EMPIRICAL POINT-WISE PRIVACY DYNAMICS OF DEEP LEARNING MODELS

Abstract

Evaluating privacy risks of a machine learning model’s training data is crucial to understand its susceptibility to leakage. Recent literature shows two ways of doing so: either by training the model using DP-SGD and perform individual privacy accounting, or by performing Membership Inference Attacks that provide empirical lower bounds of privacy risk. However, while privacy accounting is limited to the realm of models trained on DP-SGD, MIAs commonly provides either model-specific metrics such as the area under the ROC curve (AUC), or example-specific metrics such as the Attack Success Rate (ASR), but not a metric with both. To address this, we propose a fine-grained privacy metric - the α-Score, which captures model and example-specific privacy risk simultaneously derived from the p-value of the hypothesis test. This fine-grained metric allows us to study the point-wise privacy dynamics of a training algorithm and observe under a new light multiple privacy effects recently discovered in the literature such as privacy amplification by iteration, and the privacy onion effect. Using such fine-grained metric, we first introduce a private learning algorithm that improves overall privacy while preserving utility, but also a more efficient way to determine empirically the inherent individual vulnerability of training examples.