Representation learning with function call graph transformations for malware open set recognition

Jingyun Jia, Philip K. Chan

Open set recognition (OSR) problem has been a challenge in many machine learning (ML) applications, such as security. As new/unknown malware families occur regularly, it is difficu…