Malicious URL Detection using Machine Learning: A Survey

Abstract

Malicious URL, a.k.a. malicious website, is a common and serious threat to cybersecurity. Malicious URLs host unsolicited content (spam, phishing, drive-by exploits, etc.) and lure unsuspecting users to become victims of scams (monetary loss, theft of private information, and malware installation), and cause losses of billions of dollars every year. It is imperative to detect and act on such threats in a timely manner. Traditionally, this detection is done mostly through the usage of blacklists. However, blacklists cannot be exhaustive, and lack the ability to detect newly generated malicious URLs. To improve the generality of malicious URL detectors, machine learning techniques have been explored with increasing attention in recent years. This article aims to provide a comprehensive survey and a structural understanding of Malicious URL Detection techniques using machine learning. We present the formal formulation of Malicious URL Detection as a machine learning task, and categorize and review the contributions of literature studies that addresses different dimensions of this problem (feature representation, algorithm design, etc.). Further, this article provides a timely and comprehensive survey for a range of different audiences, not only for machine learning researchers and engineers in academia, but also for professionals and practitioners in cybersecurity industry, to help them understand the state of the art and facilitate their own research and practical applications. We also discuss practical issues in system design, open research challenges, and point out some important directions for future research.

References in corpus (7)

• URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection
• Online Learning: A Comprehensive Survey
• eXpose: A Character-Level Convolutional Neural Network with Embeddings For Detecting Malicious URLs, File Paths and Registry Keys
• Identifying Malicious Web Domains Using Machine Learning Techniques with Online Credibility and Performance Data
• Breaking Bad: Detecting malicious domains using word segmentation
• Know Your Phish: Novel Techniques for Detecting Phishing Sites and their Targets
• SOL: A Library for Scalable Online Learning Algorithms

Cited by in corpus (14)

• URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection
• A Survey on Malicious Domains Detection through DNS Data Analysis
• Real-Time COVID-19 Diagnosis from X-Ray Images Using Deep CNN and Extreme Learning Machines Stabilized by Chimp Optimization Algorithm
• Malicious Web Domain Identification using Online Credibility and Performance Data by Considering the Class Imbalance Issue
• Detecting Phishing Sites -- An Overview
• Using Lexical Features for Malicious URL Detection -- A Machine Learning Approach
• Transformers for End-to-End InfoSec Tasks: A Feasibility Study
• Challenges in Combating COVID-19 Infodemic -- Data, Tools, and Ethics
• Detecting Malicious URLs of COVID-19 Pandemic using ML technologies
• Less is More: Robust and Novel Features for Malicious Domain Detection
• Training Transformers for Information Security Tasks: A Case Study on Malicious URL Prediction
• Ten AI Stepping Stones for Cybersecurity
• Deep Learning based Frameworks for Handling Imbalance in DGA, Email, and URL Data Analysis
• Detection of cybersecurity attacks through analysis of web browsing activities using principal component analysis