Adversarial Sampling for Fairness Testing in Deep Neural Network

Abstract

In this research, we focus on the usage of adversarial sampling to test for the fairness in the prediction of deep neural network model across different classes of image in a given dataset. While several framework had been proposed to ensure robustness of machine learning model against adversarial attack, some of which includes adversarial training algorithm. There is still the pitfall that adversarial training algorithm tends to cause disparity in accuracy and robustness among different group. Our research is aimed at using adversarial sampling to test for fairness in the prediction of deep neural network model across different classes or categories of image in a given dataset. We successfully demonstrated a new method of ensuring fairness across various group of input in deep neural network classifier. We trained our neural network model on the original image, and without training our model on the perturbed or attacked image. When we feed the adversarial samplings to our model, it was able to predict the original category/ class of the image the adversarial sample belongs to. We also introduced and used the separation of concern concept from software engineering whereby there is an additional standalone filter layer that filters perturbed image by heavily removing the noise or attack before automatically passing it to the network for classification, we were able to have accuracy of 93.3%. Cifar-10 dataset have ten categories of dataset, and so, in order to account for fairness, we applied our hypothesis across each categories of dataset and were able to get a consistent result and accuracy.

arXiv comment7 pages, 5 figures, International Journal of Advanced Computer Science and Application

References in corpus (12)

• Explaining and Harnessing Adversarial Examples
• Stealing Machine Learning Models via Prediction APIs
• Synthesizing Robust Adversarial Examples
• Fairness Testing: Testing Software for Discrimination
• Ex Machina: Personal Attacks Seen at Scale
• Automated Directed Fairness Testing
• APE-GAN: Adversarial Perturbation Elimination with GAN
• Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching
• Bypassing Feature Squeezing by Increasing Adversary Strength
• FairTest: Discovering Unwarranted Associations in Data-Driven Applications
• Information Laundering for Model Privacy
• Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)