URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection

Abstract

Malicious URLs host unsolicited content and are used to perpetrate cybercrimes. It is imperative to detect them in a timely manner. Traditionally, this is done through the usage of blacklists, which cannot be exhaustive, and cannot detect newly generated malicious URLs. To address this, recent years have witnessed several efforts to perform Malicious URL Detection using Machine Learning. The most popular and scalable approaches use lexical properties of the URL string by extracting Bag-of-words like features, followed by applying machine learning models such as SVMs. There are also other features designed by experts to improve the prediction performance of the model. These approaches suffer from several limitations: (i) Inability to effectively capture semantic meaning and sequential patterns in URL strings; (ii) Requiring substantial manual feature engineering; and (iii) Inability to handle unseen features and generalize to test data. To address these challenges, we propose URLNet, an end-to-end deep learning framework to learn a nonlinear URL embedding for Malicious URL Detection directly from the URL. Specifically, we apply Convolutional Neural Networks to both characters and words of the URL String to learn the URL embedding in a jointly optimized framework. This approach allows the model to capture several types of semantic information, which was not possible by the existing models. We also propose advanced word-embeddings to solve the problem of too many rare words observed in this task. We conduct extensive experiments on a large-scale dataset and show a significant performance gain over existing methods. We also conduct ablation studies to evaluate the performance of various components of URLNet.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Dynamic Memory Networks for Visual and Textual Question Answering
• Malicious URL Detection using Machine Learning: A Survey
• PhishDef: URL Names Say It All
• eXpose: A Character-Level Convolutional Neural Network with Embeddings For Detecting Malicious URLs, File Paths and Registry Keys

Cited by in corpus (11)

• Malicious URL Detection using Machine Learning: A Survey
• Look Before You Leap: Detecting Phishing Web Pages by Exploiting Raw URL And HTML Characteristics
• HTMLPhish: Enabling Phishing Web Page Detection by Applying Deep Learning Techniques on HTML Analysis
• Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• To Ensemble or Not: Assessing Majority Voting Strategies for Phishing Detection with Large Language Models
• Transformers for End-to-End InfoSec Tasks: A Feasibility Study
• Predicting conversions in display advertising based on URL embeddings
• Training Transformers for Information Security Tasks: A Case Study on Malicious URL Prediction
• Deep Learning based Frameworks for Handling Imbalance in DGA, Email, and URL Data Analysis
• Characterizing Malicious URL Campaigns