Upload File

learning based malicious web sites detection using...

  • Home
  • Documents
  • Learning based Malicious Web Sites Detection Using ...users.eecs.northwestern.edu/~hlc720/349/HTXPZYQ_poster.pdf · Learning based Malicious Web Sites Detection Using Suspicious URLs!
1
Learning based Malicious Web Sites Detection Using Suspicious URLs Haotian Liu, Xiang Pan, Zhengyang Qu {haotianliu2011, xiangpan2011,zhengyangqu2017}@u.northwestern.edu EECS 349: Machine Learning, Bryan Pardo (instructor), Northwestern University 1.Introduction Problem statement Web applications become popular and bring people convenience, while there is a rapid growth in the number of attacks from various criminal enterprises, such as financial fraud and spam-advertised commerce. The common thread among those attacks is the requirement that unsuspecting users visiting the sites, clicking the target Uniform Resource Locator (URL). If we can be informed of the properties of the target URL in advance, in other words, whether it is dangerous or not, such problems will be largely resolved. Project Goal This project is to present a learning based system that is capable of classifying URL intelligently without accessing the Web site, which removes the run-time latency and protects user from being exposed to browser-based vulnerabilities. Thus, our approach is complementary to existing blacklisting service and the systems based on evaluating site content and behavior. With the effective extraction of URL features and the resulting classification model, our system labels whether the Web site of the input URL is benign, phishing, or malware. 2. System 3. Approach 5. Conclusions Classifier Comparison J48 Decision Tree (DT) Support Vector Machine (SVM) Feature Extraction & Selection Given the input URL, our system first extracts the features in 3 groups and performs transformation on the features with String type by referring to a Map constructed by us. With the sample and classification model, our approach can classify the URL and output the classification result. URL Lexical Features Result Host-based Features Gathering The Data We randomly collect 22,190 benign URLs from DMOZ Open Directory Project. DMOZ is one of the largest human-edited directory of the world. It classifies URLs into different categories. As for phishing URLs, we collect 5,703 samples from PhishTank, a collaborative site where people can submit and verify phishing URLs. Besides, we select 9,220 URLs from DNS-BH project, a site creating and maintaining a list of domains known to be used to propagate malware and spyware. Baseline Feature Comparison We adopt ZeroR algorithm as baseline method. ZeroR is a simple rule-based classifier and it assigns every value to the most common class. In our dataset, we have 37113 instances, including 5703 phishing URLs, 22190 benign URLs and 9220 malware URLs. Therefore, ZeroR will simply classify all the instances as benign URLs and in our case, the accuracy of ZeroR can be calculated by using the number of benign URLs divided by the size of dataset. The baseline is 59.79%. Additional thanks to Zhiyao Duan, Mark Cartwright We propose a learning based approach to separating Web sites into 3 classes: benign, phishing, and malware. The analysis is only based on URL itself without accessing the target website, which removes the run-time latency and protects user from being exposed to browser-based vulnerabilities. We argue that this approach is complementary to both blacklisting and the systems based on evaluating site content and behavior. By carefully selecting features and learning algorithms, our system achieves 97.53% accuracy on detecting malicious Web sites. •Evaluate how much each group of features increases/decreases the classification accuracy •Host-based features contribute the most to the performance. •The effect of lexical features on classification accuracy will be largely reduced when LR is utilized. Logistic Regression (LR) Learning Algorithm Method Host-based Feature We separate the features of URL into 3 classes: Lexical Feature, Site popularity Feature, and Host-based Feature. Feature Type Length of hostname Integer Length of entire URL Integer Number of dots Integer Top-level domain Integer Domain token count Integer Path token count Integer Average domain token length Real Average path token length Real Longest domain token length Integer Longest path token length Integer Brand name presence Binary IP address presence Binary Security sensitive word presence Binary 4. Evaluation Site Popularity Features Classifier Lexical Feature 6. Web Site Classifier Accuracy (%) Runtime (s) DT 97.53 1.9 SVM 86.26 579.4 LR 95.84 5.7 We evaluate each algorithm in classification accuracy, model building time, and true positive rate (TPR) and false positive rate (FPR). http://users.eecs.northwestern.edu/~hlc720/349/index.html Training URLs in the dataset are labeled. We use 5-fold method to train our systems. Before training, we preprocess the features not consistent with others. For example, the range of traffic feature is much larger than that of other features. We map the feature into a much smaller range and it turns out to significantly increase the accuracy. We also use Chi-Square test and virtualization tool in Weka to select most informative features. Then, we use three learning algorithms-J48 Decision Tree, Support Vector Machine (SVM) and Logistic Regression to construct the model. Target Web Site Preprocessing Lexical Feature Extraction Host-based Feature Extraction Site Popularity Feature Extraction Feature Selection & Combination Feature Type Number of External Links Integer Real traffic rank Integer Domain in a reputable sites list Binary Site Popularity Feature DT SVM LR 0 10 20 30 40 50 60 70 80 90 100 Learning Algorithms Classification Accuracy (%) Lexical Popularity Host−based 30 times test (randomly select 75% samples) DT SVM LR Benign Malware Phishing TPR (%) 98.3 99.9 90.7 FPR (%) 3.6 0.2 1.1 Benign Malware Phishing TPR (%) 98.9 84.8 44.0 FPR (%) 30.6 0.7 0.3 Benign Malware Phishing TPR (%) 97.5 100 83.2 FPR (%) 6.4 0 1.8 Feature Type Autonomous system number Integer IP country Integer Number of registration information Integer Number of resolved IPs Integer Domain contains valid PTR record Binary Redirect to new site Binary All IPs are consistent Binary 5 10 15 20 25 30 84 86 88 90 92 94 96 98 100 Number of training times Classification Accuracy (%) DT SVM LR

Upload: dinhnhi

Post on 14-Jul-2018

222 views

Category:

Documents


1 download

Report

TRANSCRIPT

Page 1: Learning based Malicious Web Sites Detection Using ...users.eecs.northwestern.edu/~hlc720/349/HTXPZYQ_poster.pdf · Learning based Malicious Web Sites Detection Using Suspicious URLs!

Learning based Malicious Web Sites Detection Using Suspicious URLs!Haotian Liu, Xiang Pan, Zhengyang Qu {haotianliu2011, xiangpan2011,zhengyangqu2017}@u.northwestern.edu!

EECS 349: Machine Learning, Bryan Pardo (instructor), Northwestern University!

1. Introduction"Problem statement!Web applications become popular and bring people convenience, while there is a rapid growth in the number of attacks from various criminal enterprises, such as financial fraud and spam-advertised commerce. The common thread among those attacks is the requirement that unsuspecting users visiting the sites, clicking the target Uniform Resource Locator (URL). If we can be informed of the properties of the target URL in advance, in other words, whether it is dangerous or not, such problems will be largely resolved."Project Goal!This project is to present a learning based system that is capable of classifying URL intelligently without accessing the Web site, which removes the run-time latency and protects user from being exposed to browser-based vulnerabilities. Thus, our approach is complementary to existing blacklisting service and the systems based on evaluating site content and behavior. With the effective extraction of URL features and the resulting classification model, our system labels whether the Web site of the input URL is benign, phishing, or malware.!

2. System"

3. Approach"

5. Conclusions"

Classifier Comparison"

J48 Decision Tree (DT)"

Support Vector Machine (SVM)"

!!!!

Feature Extraction & Selection"

Given the input URL, our system first extracts the features in 3 groups and performs transformation on the features with String type by referring to a Map constructed by us. With the sample and classification model, our approach can classify the URL and output the classification result. !

URL!

Lexical Features !

Result

Host-based Features!

Gathering The Data"We randomly collect 22,190 benign URLs from DMOZ Open Directory Project. DMOZ is one of the largest human-edited directory of the world. It classifies URLs into different categories. As for phishing URLs, we collect 5,703 samples from PhishTank, a collaborative site where people can submit and verify phishing URLs. Besides, we select 9,220 URLs from DNS-BH project, a site creating and maintaining a list of domains known to be used to propagate malware and spyware. !

Baseline"

Feature Comparison"

We adopt ZeroR algorithm as baseline method. ZeroR is a simple rule-based classifier and it assigns every value to the most common class. In our dataset, we have 37113 instances, including 5703 phishing URLs, 22190 benign URLs and 9220 malware URLs. Therefore, ZeroR will simply classify all the instances as benign URLs and in our case, the accuracy of ZeroR can be calculated by using the number of benign URLs divided by the size of dataset. The baseline is 59.79%.!

Additional thanks to Zhiyao Duan, Mark Cartwright!

We propose a learning based approach to separating Web sites into 3 classes: benign, phishing, and malware. The analysis is only based on URL itself without accessing the target website, which removes the run-time latency and protects user from being exposed to browser-based vulnerabilities. We argue that this approach is complementary to both blacklisting and the systems based on evaluating site content and behavior. By carefully selecting features and learning algorithms, our system achieves 97.53% accuracy on detecting malicious Web sites.

• Evaluate how much each group of features increases/decreases the classification accuracy !

• Host-based features contribute the most to the performance. !!• The effect of lexical features on classification accuracy will be largely reduced when LR is utilized.

Logistic Regression (LR)"

Learning Algorithm"

Method"

Host-based Feature!

We separate the features of URL into 3 classes: Lexical Feature, Site popularity Feature, and Host-based Feature.!

Feature" Type"Length of hostname! Integer!Length of entire URL! Integer!Number of dots! Integer!Top-level domain! Integer!Domain token count! Integer!Path token count! Integer!Average domain token length! Real!Average path token length! Real!Longest domain token length! Integer!Longest path token length! Integer!Brand name presence! Binary!IP address presence! Binary!Security sensitive word presence! Binary!

4. Evaluation"

Site Popularity Features!

Classifier!

Lexical Feature!

6. Web Site"

Classifier" Accuracy (%)" Runtime (s)"DT! 97.53! 1.9!

SVM! 86.26! 579.4!LR! 95.84! 5.7!

We evaluate each algorithm in classification accuracy, model building time, and true positive rate (TPR) and false positive rate (FPR).!

http://users.eecs.northwestern.edu/~hlc720/349/index.html!

Training"URLs in the dataset are labeled. We use 5-fold method to train our systems. Before training, we preprocess the features not consistent with others. For example, the range of traffic feature is much larger than that of other features. We map the feature into a much smaller range and it turns out to significantly increase the accuracy. We also use Chi-Square test and virtualization tool in Weka to select most informative features. Then, we use three learning algorithms-J48 Decision Tree, Support Vector Machine (SVM) and Logistic Regression to construct the model.!

Target Web Site"

Preprocessing

Lexical Feature Extraction

Host-based Feature Extraction

Site Popularity Feature Extraction

Feature Selection & Combination

Feature" Type"Number of External Links! Integer!Real traffic rank! Integer!Domain in a reputable sites list! Binary!

Site Popularity Feature!DT SVM LR

0

10

20

30

40

50

60

70

80

90

100

Learning Algorithms

Cla

ssifi

catio

n A

ccur

acy

(%)

LexicalPopularityHost−based

30 times test (randomly select 75% samples)"

DT!

SVM!

LR!

Benign" Malware" Phishing"TPR (%)! 98.3! 99.9! 90.7!FPR (%)! 3.6! 0.2! 1.1!

Benign" Malware" Phishing"TPR (%)! 98.9! 84.8! 44.0!FPR (%)! 30.6! 0.7! 0.3!

Benign" Malware" Phishing"TPR (%)! 97.5! 100! 83.2!FPR (%)! 6.4! 0! 1.8!

Feature" Type"Autonomous system number! Integer!IP country! Integer!Number of registration information! Integer!Number of resolved IPs! Integer!Domain contains valid PTR record! Binary!Redirect to new site! Binary!All IPs are consistent! Binary!

5 10 15 20 25 3084

86

88

90

92

94

96

98

100

Number of training times

Cla

ssifi

catio

n Ac

cura

cy (%

)

DTSVMLR

Weighted trust evaluation-based malicious node detection ...bingweb.binghamton.edu/~hhu1/paper/Hu2009weighted.pdf · Weighted trust evaluation-based malicious node detection for WSNs

Popularity-Based Detection of Malicious Content in …stalukder.cs.edinboro.edu/publications/ICTSCI_2019.pdfPopularity-Based Detection of Malicious Content in Facebook Using Machine

Man vs. Machine: Adversarial Detection of Malicious Crowdsourcing Workers

Malicious URL Detection by Dynamically Mining Patterns ...jpei/publications/URLPatternMining_ · Malicious URL Detection by Dynamically Mining Patterns without Pre ... in network

DeepPhish Simulating Malicious AI - Black Hat Briefings · •Phishing URL Detection using Machine Learning •Malicious Cert Detection using Deep Learning •DeepPhish: Simulating

DROIDETEC: Android Malware Detection and Malicious Code

Malicious Client Detection using Machine learning

Intrusion Detection System for Detecting Malicious Nodes in …dspace.nitrkl.ac.in/dspace/bitstream/2080/1540/1/SKJENA...Intrusion Detection System for Detecting Malicious Nodes in

Dynamic Detection of Malicious Behavior

Fast Detection of New Malicious Domains using DNS

Malicious Website Detection Based on Honeypot Systems

Detection Driven Adaptive Multi-cue Integration for ...users.eecs.northwestern.edu/~mya671/mypapers/ICCV09_Yang_Lv_Xu... · Detection Driven Adaptive Multi-cue Integration for Multiple

Malicious node detection in vanet

Intelligent Detection of Malicious Script Code Kamron Farrokh

Adaptive Intrusion Detection of Malicious Unmanned Air

Anomaly-based detection of malicious activity in in

Jarhead Analysis and Detection of Malicious Java Applets

Detection of malicious Encrypted Web Traffic Using Machine

iOS Threats - Malicious Configuration Profiles, Threat, Detection & Mitigation

Intrusion Detection System for Detecting Malicious Nodes in

Malicious url detection using machine learning

Testing Malicious Code Detection Tools

Optimal Inspection Points for Malicious Attack Detection

HinDom: A Robust Malicious Domain Detection System based on ... · HinDom: A Robust Malicious Domain Detection System based on Heterogeneous Information Network with Transductive

Joint detection of malicious domains and infected clients · Detection of malware-infected computers and detection of malicious web domains based on ... and apply several several

Malicious JavaScript Detection by Features Extraction · Malicious JavaScript Detection by Features Extraction 67 fact, change frequently the IP addresses espe-ciallywhentheyareblacklisted

Malicious Nodes Detection in AODV-Based Mobile Ad Hoc … · 50 Malicious Nodes Detection in AODV-Based Mobile Ad Hoc GESTS- Oct.2005 node and a malicious node can access the wireless

Image analysis for malicious advertisement detection

Malicious Node Detection in WSN Using WTE

Malicious Accounts Detection based on Short URLs in Twitter

Detection of Unknown Malicious Code via Machine Learning

th Intrusion Detection System to Detect Malicious ......Intrusion Detection System to Detect Malicious Misbehaviour Nodes in Manet M.Vijay1, R.Sujatha2 P.G Scholar (CSE), M.Kumarasamy

A Hybrid Malicious Code Detection Method based on Deep ... · detection rate and detection accuracy, and reduces the time complexity of the hybrid model. 2. Hybrid Malicious Code

Malicious Code Detection - ISACA · Malicious Code Detection SCR FOR MALICIOUS INTENTION, NOT JUST ... Script Kiddie AppSec Pro Organized ... - Environment modi¿FDtion Inversion

Detection v2 Malicious Process TENABLE NETWORK SECURITY…static.tenable.com/oldsite/blog/files/report---malicious-process... · Tenable Network Security 2 Malicious Software Detection