False Sense of Security: Leveraging XAI to Analyze the Reasoning and True Performance of Context-less DGA Classifiers

Abstract

The problem of revealing botnet activity through Domain Generation Algorithm (DGA) detection seems to be solved, considering that available deep learning classifiers achieve accuracies of over 99.9%. However, these classifiers provide a false sense of security as they are heavily biased and allow for trivial detection bypass. In this work, we leverage explainable artificial intelligence (XAI) methods to analyze the reasoning of deep learning classifiers and to systematically reveal such biases. We show that eliminating these biases from DGA classifiers considerably deteriorates their performance. Nevertheless we are able to design a context-aware detection system that is free of the identified biases and maintains the detection rate of state-of-the art deep learning classifiers. In this context, we propose a visual analysis system that helps to better understand a classifier's reasoning, thereby increasing trust in and transparency of detection methods and facilitating decision-making.

Accepted at The 26th International Symposium on Research in Attacks, Intrusions and Defenses (RAID '23)

References in corpus (6)

• Predicting Domain Generation Algorithms with Long Short-Term Memory Networks
• An Investigation of Why Overparameterization Exacerbates Spurious Correlations
• Analyzing the Real-World Applicability of DGA Classifiers
• First Step Towards EXPLAINable DGA Multiclass Classification
• Making Use of NXt to Nothing: The Effect of Class Imbalances on DGA Detection Classifiers
• Explaining Machine Learning DGA Detectors from DNS Traffic Data

Cited by in corpus (2)

• Towards Robust Domain Generation Algorithm Classification
• DRIFT: Drift-Resilient Invariant-Feature Transformer for DGA Detection