A Survey of Machine Learning Methods and Challenges for Windows Malware Classification

Abstract

Malware classification is a difficult problem, to which machine learning methods have been applied for decades. Yet progress has often been slow, in part due to a number of unique difficulties with the task that occur through all stages of the developing a machine learning system: data collection, labeling, feature creation and selection, model selection, and evaluation. In this survey we will review a number of the current methods and challenges related to malware classification, including data collection, feature extraction, and model construction, and evaluation. Our discussion will include thoughts on the constraints that must be considered for machine learning based solutions in this domain, and yet to be tackled problems for which machine learning could also provide a solution. This survey aims to be useful both to cybersecurity practitioners who wish to learn more about how machine learning can be applied to the malware problem, and to give data scientists the necessary background into the challenges in this uniquely complicated space.

arXiv commentTo appear in NeurIPS 2020 Workshop: ML Retrospectives, Surveys & Meta-Analyses (ML-RSA)

References in corpus (14)

• Scikit-learn: Machine Learning in Python
• SMOTE: Synthetic Minority Over-sampling Technique
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
• WaveNet: A Generative Model for Raw Audio
• A Critical Review of Recurrent Neural Networks for Sequence Learning
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models
• Learning the PE Header, Malware Detection with Minimal Domain Knowledge
• A Divide-and-Conquer Solver for Kernel Support Vector Machines
• Exploiting compositionality to explore a large space of model structures
• Non-Negative Networks Against Adversarial Attacks
• Learning from Context: Exploiting and Interpreting File Path Information for Better Malware Detection

Cited by in corpus (6)

• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• Adversarial Attacks against Windows PE Malware Detection: A Survey of the State-of-the-Art
• A Framework for Cluster and Classifier Evaluation in the Absence of Reference Labels
• Adversarial Transfer Attacks With Unknown Data and Class Overlap
• Towards an Automated Pipeline for Detecting and Classifying Malware through Machine Learning
• Practical Cross-modal Manifold Alignment for Grounded Language