Mind the Gap: On Bridging the Semantic Gap between Machine Learning and Information Security
arXiv:2005.01800
Abstract
Despite the potential of Machine learning (ML) to learn the behavior of malware, detect novel malware samples, and significantly improve information security (InfoSec) we see few, if any, high-impact ML techniques in deployed systems, notwithstanding multiple reported successes in open literature. We hypothesize that the failure of ML in making high-impacts in InfoSec are rooted in a disconnect between the two communities as evidenced by a semantic gap---a difference in how executables are described (e.g. the data and features extracted from the data). Specifically, current datasets and representations used by ML are not suitable for learning the behaviors of an executable and differ significantly from those used by the InfoSec community. In this paper, we survey existing datasets used for classifying malware by ML algorithms and the features that are extracted from the data. We observe that: 1) the current set of extracted features are primarily syntactic, not behavioral, 2) datasets generally contain extreme exemplars producing a dataset in which it is easy to discriminate classes, and 3) the datasets provide significantly different representations of the data encountered in real-world systems. For ML to make more of an impact in the InfoSec community requires a change in the data (including the features and labels) that is used to bridge the current semantic gap. As a first step in enabling more behavioral analyses, we label existing malware datasets with behavioral features using open-source threat reports associated with malware families. This behavioral labeling alters the analysis from identifying intent (e.g. good vs bad) or malware family membership to an analysis of which behaviors are exhibited by an executable. We offer the annotations with the hope of inspiring future improvements in the data that will further bridge the semantic gap between the ML and InfoSec communities.
14 pages, 2 Figures, 6 tables
References in corpus (9)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
- EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models
- Adversarial Deep Learning for Robust Detection of Binary Encoded Malware
- Generation & Evaluation of Adversarial Examples for Malware Obfuscation
- A Benchmark API Call Dataset for Windows PE Malware Classification
- Scan2Plan: Efficient Floorplan Generation from 3D Scans of Indoor Scenes
- ALOHA: Auxiliary Loss Optimization for Hypothesis Augmentation
- Automatic Malware Description via Attribute Tagging and Similarity Embedding