TOWARD A BETTER UNDERSTANDING OF PRIVACY LEAKAGE IN MACHINE LEARNING, USING DATASET PRUNING ATTACK

Abstract

While membership inference attacks in machine learning setups have been studied for several years, the exact mechanisms leading to privacy leakage are only partially understood. We developed a dataset pruning attack, which is similar to a dataset poisoning attack, but that differs in that it removes points instead of adding them. The effects of this attack help us get insights as to how privacy leakage occurs. This dataset pruning attack utilizes local behaviors in the latent space of a trained model, to isolate target points from other same class samples. It is very efficient, as it increases the AUC of the membership inference attack on the targeted points by up to 91 &37; . It also produces side effects by affecting non targeted points. Furthermore, we developed a python library called privacy-meter, to help non-experts gain insights regarding privacy leakage that their models and datasets are exposed to.