detection of web-based attackspralab.diee.unica.it/sites/default/files/corona-phd_slides.pdf ·...

64
Detection of Web-based attacks Detection of Web-based attacks PhD Thesis - DIEE University of Cagliari, Italy Igino Corona March 4, 2010

Upload: others

Post on 02-Aug-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Detection of Web-based attacksPhD Thesis - DIEE University of Cagliari, Italy

Igino Corona

March 4, 2010

Page 2: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

1 Research outline

2 Current Internet ThreatsWorld Wide WebCommon Gateway InterfaceClient-side web securityServer-side web security

3 Our Contribution to Client-side Web SecurityFlux Buster

4 Our Contribution to Server-side Web SecurityWeb Guardian

5 Research Contributions - summary

6 Limitations - summary

Page 3: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Research outline

Intrusion Detection and Adversarial Environment - criticalreview

I. Corona , G. Giacinto, F. Roli, Intrusion detection in computer systems as apattern recognition task in adversarial environment: a cri tical review ,Workshop on Neural Information Processing Systems (NIPS), Whistler, BritishColumbia, Canada, 08/12/2007

Detailed work on the PhD thesis (it is going to be submitted soon to animportant Journal)

Intrusion Detection and Multiple Classifier SystemsI. Corona , G. Giacinto, F. Roli, Intrusion Detection in Computer Systemsusing Multiple Classifer Systems , Supervised and Unsupervised EnsembleMethods and Their Applications, O. Okun and G. Valentini, no. 126:Springer-Verlag, Berlin/Heidelberg, pp. 91-114, 2008

Page 4: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Research outline

Intrusion Detection and Information FusionI. Corona , G. Giacinto, C. Mazzariello, F. Roli, C. Sansone, Information fusionfor computer security: State of the art and open issues , Information Fusion,vol. 10, pp. 274-284, 2009

Intrusion Detection and Web SecurityI. Corona , D. Ariu, G. Giacinto , HMM-Web: a framework for the detection ofattacks against Web applications , IEEE ICC 2009, Dresden, Germany,14/06/2009

HMM-Web → Web Guardian Detailed work on the PhD Thesis (it is going to besubmitted soon to a relevant conference)

R. Perdisci, I. Corona , D. Dagon, W. Lee, Detecting Malicious Flux ServiceNetworks through Passive Analysis of Recursive DNS Traces , AnnualComputer Security Applications Conference (ACSAC), Honolulu, Hawaii, USA,07/12/2009

Page 5: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

World Wide Web

The weak point in the chain: World Wide Web

Nowadays, most of Internet threats are due to Web-basedvulnerabilities [SANS (2009), Cenzic (2009)]

World Wide Web

easyinformation

sharing

businessoppor-tunities

highexpositionof services

developerswith littlesecuritytraining

strict timedevelopmentconstraints

complexapplications

Page 6: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Common Gateway Interface

web browser Internet web server

Page 7: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Common Gateway Interface

web browser Internet web serverrequest

Page 8: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Common Gateway Interface

web browser Internet web serverrequest request

web application

CGIinput query

Page 9: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Common Gateway Interface

web browser Internet web serverrequest request

web application

CGIinput query content

Page 10: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Common Gateway Interface

web browser Internet web serverrequest request

web application

CGIinput query content

response [content]response [content]

Page 11: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Client-side web security

web browser web server[malicious content/scams]

attackerweb user (victim)

Client-side problem : malicious (or infect) websites

Malicious websites routinely exploit vulnerabilities on browsers(e.g. Internet Explorer, Firefox) and their plugins (e.g.Javascript, Adobe Reader, Flash player) to execute arbitrary(unauthorized) instructions at client-side. Compromisedcomputers may take part in a botnet. In addition, maliciouswebsites may support a wide range of scams (e.g. Phishingscams, Fake Job proposals, Fake lotteries).

Page 12: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Client-side web security

Malicious Fast Flux NetworksMalicious websites are increasingly hosted through maliciousFast Flux Service Networks. These networks are composed bymalware infected computers that can be remotely controlled bymiscreants. Each computer typically acts as a HTTP proxy, i.e.retrieve malicious content from a central node calledmothership. These illegal networks are very robust, pervasiveand inherently difficult to block.

Page 13: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Client-side web security

Page 14: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Client-side web security

Page 15: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Client-side web security

Page 16: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Client-side web security

Page 17: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Client-side web security

Page 18: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Server-side web security

web browser web servermalicious request

legitimate web serviceattacker

Server-side problem : malicious web requests

Legitimate web services are routinely compromised byexploiting vulnerabilities on web servers and web applications.For example, miscreants may steal confidential information orinject malicious code on web pages, in order to attack usersthat will further access to the web services.

Page 19: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Current Internet Threats

Server-side web security

Example: Joomla Hotel Booking SystemComponent

SQL Injectionhttp://www.vulnerablehotel.com/components/com_hbssearch/longDesc.php?h_id=1&id=-2%20union%20select%20concat%28username,0x3a,password%29%20from%20jos_users--

Cross-site scriptinghttp://www.vulnerablehotel.com/index.php?option=com_hbssearch&task=showhoteldetails&id=118&adult=2<script%20src=http://www.dbrgf.ru/script.js>

Page 20: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Our Contribution to Client-side Web Security

Flux Buster

Page 21: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Key observationsIn large networks (i.e. serving millions of users), it is very likely that some userswill (unfortunately) fall victims of malicious web content, and will therefore “click”on (and initiate DNS queries about) fast flux domain names.

Passive analysis of real users’ activities allows us to stealthily detect and collectinformation about “popular” malicious flux networks on the Internet, regardless ofthe method used by miscreants to advertise websites hosted through thesenetworks.

Thousands of new domain names per day. In general, during the time, so manydifferent (but equivalent ) domain names may resolve to the same flux network.Thus, an IP-based clustering of domain names is really useful to (a) identify therelationship between domain names, (b) accurately characterize different fastflux networks, (c) obtain a lower number of objects (domain clusters vs domains)that must be classified.

Page 22: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Passive RDNS data collection

Page 23: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Architecture

Page 24: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Architecture

Very conservative (but effective) prefiltering rules

F1: stateless rules, e.g. TTL ≥ 3 hoursF2: stateful rules, e.g. for each domain name resolved at least100 times: (a) it is associated to only 5 (or less) distinct IPaddresses and (b) there is no DNS reply which returns morethan 2 new IP addresses.

Page 25: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Preprocessing phase

Page 26: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Preprocessing phase

↓ F1+F2

Page 27: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Hierarchical single linkage Clustering

sim(α, β) =|R(α) ∩ R(β)|

|R(α) ∪ R(β)|·

1

1 + eγ−min(|R(α)|,|R(β)|)∈ [0, 1]

Page 28: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Hierarchical single linkage Clustering

sim(α, β) =|R(α) ∩ R(β)|

|R(α) ∪ R(β)|·

1

1 + eγ−min(|R(α)|,|R(β)|)∈ [0, 1]

0.0 0.2 0.4 0.6 0.8 1.0

020

0040

0060

0080

00

cut height (h)

num

. of c

lust

ers

Figure: Cluster Analysis,Sensor 1.

0.0 0.2 0.4 0.6 0.8 1.0

010

0030

0050

0070

00

cut height (h)

num

. of c

lust

ers

Figure: Cluster Analysis,Sensor 2.

Page 29: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Service Classifier

Cluster statistical features

Passive : φ1 Number of resolved IPs, φ2 Number of do-mains, φ3 Avg. TTL per domain, φ4 Networkprefix diversity, φ5 Number of domains per net-work, φ6 IP Growth Ratio

Active : φ7 Autonomous System (AS) diversity, φ8 BGPprefix diversity, φ9 Organization diversity, φ10Country Code diversity, φ11 Dynamic IP ratio,φ12 Average Uptime Index.

Page 30: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Service Classifier

Cluster ID Cluster Nickname Use Labell1 cdne.gearsofwar.xbox.com CDN Legitimatel2 fotf.cdnetworks.net CDN Legitimatel3 3.europe.ntp.org NTP pool Legitimatel4 opendht.nyuld.net OASIS Legitimate

m1 50b0f40526956b85.saidthesestory.com Adult Content/Malware Malicious Fluxm2 paypal.database-confirmation.com Phishing Malicious Fluxm3 hqdvrp.flagacai.com Pharmacy Scam Malicious Flux

l1 l2 l3 l4 m1 m2 m3

IP Growth Ratio (φ6) 0.028 0.016 0.039 0.021 0.932 0.374 0.56Number of domains per network (φ5) 488 165 57 54 42000 228 1632Avg. TTL per domain (φ3) 22 20 1402 7421 300 180 180

Page 31: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Service Classifier

Labeled DatasetTime Interval 1march / 14april 2009Users Over 4 millionsDNS queries 2.5 · 109 per dayCandidate flux domains ∼ 105 per dayDomain Clusters ∼ 310 clusters per day1

Fast Flux Clusters ∼ 23 clusters per dayFast Flux domain names 61,710Flux Agents 17,332

1We consider only clusters (networks) having at least 10 IP addresses

Page 32: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Service Classifier

Page 33: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Service Classifier - accuracy

Decision tree Accuracy - C4.5 algorithm -5 fold cross validation: 60%training, 40%test

Features AUC DR FPAll 0.992 (0.003) 99.7% (0.36) 0.3% (0.36)

Passive 0.993 (0.005) 99.4% (0.53) 0.6% (0.53)φ6, φ3, φ5 0.989 (0.006) 99.3% (0.49) 0.7% (0.49)

Page 34: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - domain name Blacklisting

adult content0711afafa7803d51.nugentcelticdonnell.com, 088683b12777d475.ghostsbarredrental.com,

08f15257a0ea7ee5.spreadnettingcleanly.com, 09ad518ad726e193.squadsvariousembryos.com,

09ae7f81efa7faa2.fraserlibraryshabby.com, 0a1a7c2792c461ed.nugentcelticdonnell.com,

0b53caa4e8a9edb5.fraserlibraryshabby.com, 0bc0dd7f7773c50c.nugentcelticdonnell.com,

0bfd3365dca2c45b.nugentcelticdonnell.com, 0c9328f675b1b931.ghostsbarredrental.com,

0d565d437fb5869d.ghostsbarredrental.com, 0d9d81f5e70761d2.squadsvariousembryos.com,

0dfde08e68ca8358.ghostsbarredrental.com, 0e294041c5d3d17c.developleftcity.com,

0e3fe6f42143105b.squadsvariousembryos.com, 0f255699977f3a81.ghostsbarredrental.com,

0fde9565dad27a33.nugentcelticdonnell.com, 100d83dcb74219a6.fraserlibraryshabby.com,

14cc04d937dd090f.fraserlibraryshabby.com, 163f3db2671f9703.fraserlibraryshabby.com,

189dda5b6c51569e.squadsvariousembryos.com, 18ad145ae37d4318.ghostsbarredrental.com,

191ab3abf627f482.nugentcelticdonnell.com, 1a3a25badc9819c5.nugentcelticdonnell.com

[· · · many more]

Page 35: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - domain name Blacklisting

facebook phishingfacebook.shared.accessservlet.personalid-fbhmod8j9.processlogon.344session.com,

facebook.shared.accessservlet.personalid-kd0vb3bjj.ceptservlet.8345server.com,

facebook.shared.accessservlet.personalid-mct6meeyi.alternative.8345server.com,

facebook.shared.accessservlet.personalid-xm4f9y8xa.emberuiweb.344session.com,

facebook.shared.accountholder.personalid-0ip00okut.mixed.5435core.com,

facebook.shared.accountholder.personalid-3vj54osat.accountholder.344session.com,

facebook.shared.accountverify.personalid-4z37tsrz9.usermanage.344session.com,

facebook.shared.accountverify.personalid-sa3vts29i.serveronline.8345server.com [· · ·

many more]

Page 36: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - domain name Blacklisting

myspace phishingaccounts.myspace.com.tteszk.org.uk, accounts.myspace.com.tteszk.me.uk,

accounts.myspace.com.tteszk.co.uk, accounts.myspace.com.tteszg.org.uk,

accounts.myspace.com.tteszg.me.uk, accounts.myspace.com.tteszg.co.uk,

accounts.myspace.com.tteszf.co.uk, accounts.myspace.com.ttesza.org.uk,

accounts.myspace.com.ttesza.me.uk, accounts.myspace.com.ttesza.co.uk,

accounts.myspace.com.terhhoq.org.uk, accounts.myspace.com.terhhoq.me.uk,

accounts.myspace.com.terhhoq.co.uk, accounts.myspace.com.terhhol.org.uk,

accounts.myspace.com.terhhol.me.uk, accounts.myspace.com.terhhol.eu,

accounts.myspace.com.terhhol.co.uk, accounts.myspace.com.terhhok.org.uk,

accounts.myspace.com.terhhok.me.uk, accounts.myspace.com.terhhok.eu,

accounts.myspace.com.iuuuujer.me.uk, accounts.myspace.com.iuuuujer.eu,

accounts.myspace.com.iuuuujer.co.uk, accounts.myspace.com.iuuuujek.org.uk,

accounts.myspace.com.iuuuujek.me.uk [· · · many more]

Page 37: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - domain name Blacklisting

ebay phishingcgi.ebay.com.fvdssrt.com, cgi.ebay.com.idservertff.net, cgi.ebay.com.idsrvtttr.com,

cgi.ebay.com.modefst10.mobi, cgi.ebay.com.msdrvffg.net, cgi.ebay.com.msdrvt1.bz,

cgi.ebay.com.msfddre.com, cgi.ebay.com.mtdfggs.com, cgi.ebay.com.sdlserverts.com,

cgi.ebay.com.trffdsl.com, cgi.ebay.com.vfrres.com, cgi.ebay.com.vsdfggg.net,

cgi.ebay.com.vvssldr.com, cgi.ebay.com.vvssldr.net, cgi.ebay.com.vzdfff1.com,

cgi.ebay.com.dllmsdrv.net

Page 38: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - domain name Blacklisting

bank/irs.gov phishingchaseonline.chase.com.omersw.com, chaseonline.chase.com.omersr.net,

chaseonline.chase.com.omersr.com, chaseonline.chase.com.omersf.net,

chaseonline.chase.com.omersf.com, chaseonline.chase.com.omersd.net,

chaseonline.chase.com.nyterdasq.net, chaseonline.chase.com.nyterdasq.com,

chaseonline.chase.com.omersx.net, chaseonline.chase.com.omersx.com, fwd.omersf.net,

chaseonline.chase.com.nyterdasp.net, 02fgu145501.cn,

chaseonline.chase.com.nyterdasp.com, chaseonline.chase.com.omersw.net, ger11zr.com,

c.omersx.com, www.irs.gov.ger11zh.net, www.irs.gov.yh1ferz.info,

www.irs.gov.yh1ferz.com, www.irs.gov.ger11zr.com, www.irs.gov.merfaslo.com,

www.irs.gov.ger11zh.com, www.irs.gov.ger11zx.eu, gshipagc.com, gshipagc.net,

www.ger11zf.net, grph.omersf.net [· · · many more]

Page 39: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - domain name Blacklisting

on-line pharmacy scamfiweixg.cn, fshioiwg.cn, fsieoowf.cn, galn.sfoioiiw.cn, gba.sdigwpd.cn,

gdao.sfoioiiw.cn, gdap.sdigwpd.cn, gdou.sdigwpd.cn, gdq.sfoioiiw.cn, gff.fsieoowf.cn,

gfnt.fsieoowf.cn, ggq.fieooief.cn, ggq.sdigwpd.cn, gguf.ssmmmwp.cn, gh.dipmmeig.cn,

gib.fsieoowf.cn, gib.igemmpi.cn, giew.igemmpi.cn, gii.fsieoowf.cn, gjhn.dipmmeig.cn,

gkah.sdigwpd.cn, glhh.sfoioiiw.cn, glqu.sfoioiiw.cn, gmb.sdigwpd.cn, gnum.sdigwpd.cn,

gnvq.fshioiwg.cn, gpb.sdigwpd.cn, gpq.fieooief.cn, gpwc.sdigwpd.cn, gqk.sfoioiiw.cn,

grd.sfoioiiw.cn, grx.sfoioiiw.cn, gsew.fieooief.cn, gsvg.fsieoowf.cn,

gtf.dipmmeig.cn, gtr.dipmmeig.cn, gtse.fshioiwg.cn, gudl.sfoioiiw.cn,

guo.bssigrpi.cn, gvhd.sfoioiiw.cn, gvxl.fsieoowf.cn, gvy.fsieoowf.cn,

gwc.sfoioiiw.cn, gwgz.sdigwpd.cn, gwz.fshioiwg.cn [· · · many more]

Page 40: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - domain name Blacklisting

Time interval: November, 3, 2009 - February, 2, 2010. Fluxagents: 21,108 IP addresses. Fast flux domain names: 16,375.

Total Visited Malicious0

2000

4000

6000

8000

10000

12000

14000

16000

18000

Num

ber

of

uniq

ue f

ast

flu

x d

om

ain

nam

es

Analysis of flux domain names through Google safebrowsing

Page 41: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - domain name Blacklisting

Interpretation

We speculate that most of flux domain names are advertized bywebpages not indexed by Google, or by means ofnon-web-based forms of advertisement. In fact, during ourexperiments we came accross several compromised websiteswhose injected HTML code was in the form:<META NAME="ROBOTS" CONTENT="NOFOLLOW"><script src=http://fast-flux-domain-name1/script.js><script src=http://fast-flux-domain-name2/script.js>...

<script src=http://fast-flux-domain-nameN/script.js> </META>

Page 42: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - real time detection and spam filtering

Real time detection of suspicious websites

We may detect in real time suspicious domain names, i.e.domain names whose resolved IPs are among the pool ofknown flux agents (detected through our system).

Page 43: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Client-side Web Security

Flux Buster

Application - real time detection and spam filtering

0.0 0.0002 0.0004 0.0006 0.0008 0.001 0.0012 0.0014 0.0016 0.0018False Positive Rate % (Alexa TOP domains)

0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

90

95

100D

ete

cti

on

Rate

%

Day 2009-03-04, 33697 spam domainsDay 2009-03-06, 105608 spam domainsDay 2009-03-10, 103554 spam domainsDay 2009-03-15, 168298 spam domains

Interpretation

We spot almost all domain names inside spam emails. It isworth noting that some of them do not have a “fluxy” behavior,but resolve to flux agents characterized by high uptime.

Page 44: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Our Contribution to Server-side Web Security

Web Guardian

Page 45: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Anomaly-based approach

ProblemWe would like to detect either known or unknown attacksagainst web services. Also, we’d like to provide for automaticcounteractions against such attacks, to protect web services inreal time.

Our Approach

Given a sample of requests on the web server, we modelthe normal (legitimate) web traffic profile

We detect web traffic that does not reflect the legitimateprofile (i.e. web attacks)

We may provide for well-suited real-time counteractions,depending on the detected anomalies

Page 46: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Architecture

Page 47: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Learning framework

Problem

We cannot assume anattack-free training set! Knownoutlier detection techniques maybe not suitable for our task.

Automatic noise filtering

Each model is (re)trainedexcluding some samples fromthe training set.

Page 48: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

General models

General-purpose models

Feature ModelSequence of symbols Hidden Markov Model (model-a) -

Baum Welch Algorithm, states=avgn. of distinct symbols in a se-quence, random init state transitionand symbol emission matrix

Numeric Value p[x|model-b] = σ2

(x−µ)2 if x > µ + σ

Discrete Value p[x|model-c] = count(x)

total n. samples

Page 49: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Modeled features

model-a sequence of: headers; web app. attributes;attribute inputs (generalization of numbers andletters);

model-b ratio between rejected and successful requests,and frequency of requests on each webapplication, per source IP address; for eachheader, its input lenght;

model-c method; http version; for each header, thefollowing flags: has-alphabetic-input,has-digit-input; for each header: list ofnon-alphanumeric input characters.

Page 50: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Experiments

Dataset Λ = Σ ∪ T

time interval 27 November - 3December, 2009

number of web requests 447,178distinct IP addresses 1,703bad requests 5,507web application queries 98,900number of web applications 217

Dataset Σ and T

Σ contains the first 200,000 requests in Λ, and it is employedfor training the system. T contains the remaining 247,178requests, and it is used for performance evaluation.

Page 51: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Experiments

Training phase

CPU Intel CoreDuo2 T8100 2.1Ghz, 2GBytes of RAM, andLinux (Ubuntu 8.04) Operating System. Training time: 2 hoursand 53 minutes RAM max 1.6GBytes.

OK, but what about attacks inside dataset Λ?

We identify attacks inside Λ with the help of Web Guardian. For each model, wemanually inspect the training samples receiving lower probability. This is justified since:(a) we may assume that attack samples are in lower number w.r.t. legitimate samples,(b) attacks are characterized by patterns significantly different from legitimate patterns.Furthermore, this process is not expensive, because we need to inspect only a smallprotion of training samples for each model.

Page 52: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Experiments

Attack dataset Φ

Target Details Attack Type References Attacks

web applica-tion queries

90 distinct web applicationsand 372 attributes

cross-site scripting,sql injection, remotecode execution, re-mote file inclusion,information gathering

[Spett (2002)][Admin (2002)][Mac Vittie (2007)][Hansen (2009)][Pastor (2009)][Auger (2010)][L0t3k]

412

headers Accept,Accept-Language,Referer, Content-Type,Accept-Encoding,User-Agent, Host,Content-Length,Connection,Cache-Control, Cookie,Via, X-Forwarded-For,If-Modified-Since

generic buffer over-flow, cross-sitescripting, sql injec-tion, http requestsmuggling, CRLFinjection

[Bellamy (2002)][PSS (2002)][Linhart et al. (2005)][Symantec (2006)][CAPEC (2007)][Bajpai (2009)][Mac Vittie (2010)]

78

method PROPFIND, OPTIONS,TRACE and bad strings

buffer overflow,cross-site scripting,information gathering

[Donaldson (2002)][Juniper (2002)][Manion (2003)][Shah (2004)]

12

http version bad format string buffer overflow, infor-mation gathering

[Donaldson (2002)][Shah (2004)]

5

Page 53: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Experiments

Results

Parameter Dataset Value

detection rateΛ = Σ ∪ T 232/232 100% ∼39alerts/dayΦ 505/507 99.6%

false alarm rateΛ 1,252/447,178 0.28% ∼209alerts/dayΣ 450/200,000 0.22% ∼150alerts/dayT 802/247,178 0.32% ∼267alerts/day

response time Λ 1.2 milliseconds

Page 54: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Experiments

Observation

It is worth to note that a significantly lower false positive rate may be attained bymanually verifying false alarms on our web interface. Using such a interface we may:

group anomalies depending on their type: i.e. what is the model which raised theanomaly, common traits of the anomaly (e.g. a suspect non-alphanumericcharacter), source IP address, targeted web application/header

adjust model thresholds, so that attacks may be still reliably evidenced and falsealarms are reduced

(re)train models using some samples which have been erroneously discarded bythe learning framework (e.g. because there were no attacks in the set of trainingsamples)

Page 55: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Our Contribution to Server-side Web Security

Web Guardian

Implementation

Page 56: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Research Contributions - summary

Flux Buster

novel, passive approach for detecting and tracking malicious flux servicenetworks.

we detect fast flux domain names, regardless the way they are advertised

active probing proposed so far is expensive, requires a distributed architecture,and may be detected and blocked/influenced by miscreants. Contrary, we do notinteract ourselves with the flux network and our approach is stealthy.

we accurately characterize and detect flux networks. By means of Flux Busterwe may substantially enhance the state-of-the-art protection of web users andspam filtering applications.

Page 57: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Research Contributions - summary

Web Guardian

unsupervised training which effectively handles the presence of attacks in thetraining set

accurate detection both known and unknown attacks against web services. Thiscomplements the rule-based approach of modsecurity.

low false positive rate

ability to counteract in real time, and thus protect web services

multiple, specific anomaly detectors allow to (a) infer the typology of an attack,(b) further reduce false positives by grouping similar anomalies, (c) provide forwell-suited counteractions

easy to extend with new models/features

the host-based approach allows us to limit evasive attacks (e.g.desynchronization) and monitor both HTTP and HTTPS traffic

Page 58: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Limitations - summary

Flux Buster

the approach is effective only if applied in large computer networks

some flux domain names may be erroneously prefiltered. To this end, a detailedevaluation is required. For example, we could select filtered domain nameswhose patterns are placed near the decision surface of our prefiltering stage.Then, we may analyze them using other fast flux detection tools (e.g.abuse.ch).

due to the massive amount of data Flux Buster has to process, theresponsiveness of Flux Buster is slow. However, this limitation may be reducedby employing the detection approach proposed for spam filtering.

in principle, fast flux operators may deliberately inject some legitimate IP addressin the pool of flux agents. However, they have to pay a reduced effectiveness offlux domain names. In order to cope with this issue, we may filterknown-as-legitimate IP addresses from the pool of flux agents, e.g. by extractingall IP addresses used by most popular websites according to legitimateorganizations such as Alexa.

Page 59: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Limitations - summary

Web Guardian

it is fundamentally limited to the detection of input validation attacks. In order todetect web attacks exploiting logical vulnerabilities, we must add new featuresand models.

actually we do not have a description of attacks. We are working on theautomatic inference of the attack class, given an anomaly.

false alarm injection: automatic counteractions may still prevent successfulattacks. However, as matter of fact, the false alarm injection attacks are notcurrently addressed by Web Guardian. As future work we intend to researchsolutions to this issue.

Page 60: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Thank you!

Thank you for your attention!

Any question?

Page 61: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Thank you!

SANS Institute (2009). The Top Cyber Security Risks -september 2009. ⇒ web link (accessed January 2010)

Cenzic, Inc. (2009). Web Application Security TrendsReport ⇒ web link (accessed January 2010)

Spett, K. (2002). SQL Injection: Are Your Web ApplicationsVulnerable?, A White Paper from SPI Dynamics ⇒ web link(accessed January 2010)

[email protected] (2002). The Cross Site ScriptingFAQ, Packet storm security ⇒ web link (accessed February2010)

Mac Vittie, L. (2007). SQL Injection Evasion Detection, F5Whitepaper ⇒ web link (accessed January 2010)

Hansen, R. (2009). XSS (Cross Site Scripting) Cheat Sheetfor filter evasion, ha.ckers.org ⇒ web link (accessedJanuary 2010)

Page 62: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Thank you!

Pastor, A. (2009). CVE-2009-1151: phpMyAdmin RemoteCode Execution Proof of Concept, GNUCitizen ⇒ web link(accessed February 2010)

Auger, R. (2010). Remote File Inclusion, The WebApplication Security Consortium ⇒ web link (accessedFebruary 2010)

L0t3k, SQL Injection: The Complete Documentation ⇒web link (accessed January 2010)

Bellamy, W. (2002). HyperText Transfer Protocol (HTTP)Header Exploitation, Advanced Incident Handling andHacker Exploits, SANS GIAC GCIH Practical Assignmentv2.1 ⇒ web link (accessed January 2010)

Packet Storm Security (2002). Apache 2.0 Cross-SiteScripting Vulnerability, ⇒ web link (accessed February2010)

Page 63: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Thank you!

Linhart, C., Klein, A., Heled, R., Orrin, S. (2005). HTTPRequest Smuggling, Watchfire ⇒ web link (accessedJanuary 2010).

Symantec (2006). HTTP Smuggle Get Content Length,attack signature ⇒ web link (accessed January 2010)

Common Attack Pattern Enumeration and Classification(CAPEC)-86: Embedding Script (XSS) in HTTP Headers,MITRE Corporation, ⇒ web link (accessed February 2010)

Bajpai, G. (2009). HP OpenView NNM HTTPAccept-Language header Buffer Overflow Vulnerability,iPolicy Networks Security Advisory ⇒ web link (accessedFebruary 2010)

Mac Vittie, L. (2007). I am in your HTTP headers, attackingyour application, F5 Whitepaper ⇒ web link (accessedJanuary 2010)

Page 64: Detection of Web-based attackspralab.diee.unica.it/sites/default/files/Corona-PhD_Slides.pdf · websites may support a wide range of scams (e.g. Phishing scams, Fake Job proposals,

Detection of Web-based attacks

Thank you!

Donaldson, M.E. (2002). Inside the Buffer Overflow Attack:Mechanism, Method, &amp; Prevention, SANS InstituteInfoSec Reading Room, SANS Whitepaper ⇒ web link(accessed January 2010)

Juniper Networks (2002). HTTP: Apache WebDavPROPFIND Directory Disclosure ⇒ web link (accessedJanuary 2010)

Manion, A. (2003). Web servers enable HTTP TRACEmethod by default, Vulnerability Note VU#867593,US-CERT ⇒ web link (accessed January 2010)

Shah, S. (2004). An Introduction to HTTP fingerprinting,Net square ⇒ web link (accessed January 2010)