Representation learning for malware open set recognition

Philip Chan

Florida Tech

Abstract

Open set recognition (OSR) problem has been a challenge in many machine learning (ML) applications, such as security. As new/unknown malware families occur regularly, it is difficult to exhaust samples that cover all the classes for the training process in ML systems. An advanced malware classification system should classify the known classes correctly while sensitive to the unknown class. In this paper, we introduce a self-supervised pre-training approach for the OSR problem in malware classification. We propose two transformations for the function call graph (FCG) based malware representations to facilitate the pretext task. Also, we present a statistical thresholding approach to find the optimal threshold for the unknown class. Moreover, the experiment results indicate that our proposed pretraining process can improve different performances of different downstream loss functions for the OSR problem.

About the Speaker

Philip Chan is an Associate Professor of Computer Science at Florida Institute of Technology. His main research interests include machine learning, data mining, and distributed computing. He received his PhD, MS, and BS in Computer Science from Columbia University, Vanderbilt University, and Texas State University respectively. He is an Associate Editor for ACM Transactions on Knowledge Discovery from Data (TKDD), Transactions on Machine Learning Research (TMLR), and the Knowledge and Information Systems (KAIS) journal.