Pa#ern  Recogni-on    and  Applica-ons  Lab  

                               

 University  

of  Cagliari,  Italy  

 

Department  of  Electrical  and  Electronic  

Engineering  

1.5-class MCSs for Secure Learning against Evasion Attacks at Test Time

Ba#sta  Biggio1,  Igino  Corona1,  Zhi-­‐min  He2,  Patrick  P.K.  Chan2,  Giorgio  Giacinto1,  Daniel  Yeung2,  Fabio  Roli1  

 (1)  Dept.  Of  Electrical  and  Electronic  Engineering,  University  of  Cagliari,  Italy  

(2)  School  of  Computer  Science  and  Eng.,  South  China  University  of  Technology,  China      

Guenzburg,  Germany,  Jun  29  -­‐  Jul  1,  2015  MCS  2015  

 

http://pralab.diee.unica.it

Machine Learning in Adversarial Settings

•  Pattern recognition in security applications –  spam filtering, malware detection, biometrics

•  Attackers manipulate data to evade detection at test time

2  

legitimate malicious

x1  

x2   f(x)

…cheap…

…che4p…

a(x)

 

http://pralab.diee.unica.it

a(x)

Simplified Risk Analysis under Attack

•  Malicious data distribution is not stationary (TR/TS)

3  

f

Rts ( f )− Rtr ( f ) = Ex,y l(y, f (a(x)))− l(y, f (x)){ }

x

p(x, y)

 

http://pralab.diee.unica.it

a(x)

Simplified Risk Analysis under Attack

•  Malicious data distribution is not stationary (TR/TS)

4  

Rts ( f )− Rts ( f*) = Ex,y l(y, f (a(x)))− l(y, f *(a(x))){ }

x

p(x, y)

f *

Be+er  enclosing  legi4mate  data  in  feature  space  may  improve  classifier  security  …  at  the  expense  of  more  false  alarms  

 

http://pralab.diee.unica.it

1.5-class Classification The Rationale Behind

5  

2−class classification

−5 0 5−5

0

5

1−class classification (legitimate)

−5 0 5−5

0

5

•  2-­‐class  classifica-on  is  usually  more  accurate  in  the  absence  of  a#ack  •  …  but  poten-ally  more  vulnerable  under  a#ack  (not  enclosing  legi-mate  data)  

1.5C classification (MCS)

−5 0 5−5

0

5

1.5-­‐class  classifica4on  aims  at  retaining  high  accuracy  and  security  under  a+ack    

 

http://pralab.diee.unica.it

Secure 1.5-class Classification with MCSs

•  Heuristic approach to 1.5-class classification

•  Base classifiers –  2-class classifier: good accuracy in the absence of attacks –  1-class classifiers: detect anomalous patterns (no support in TR)

•  Combiner –  1-class classifier on legitimate data to improve classifier security

6  

data 1C Classifier (malicious)

Feature Extraction

malicious

1C Classifier (legitimate)

2C Classifier

1C Classifier (legitimate)

legitimate

x

g1(x)

g2(x)

g3(x)

g(x) ≥ t g(x)

true

false

 

http://pralab.diee.unica.it

Classifier Security against Evasion Attacks

7  

•  How to evaluate classifier security against evasion attacks?

•  Attack strategy:

•  Non-linear, constrained optimization –  Gradient descent: approximate

solution for smooth functions

•  Gradients of g(x) can be analytically computed in many cases –  SVMs, Neural networks

−2−1.5

−1−0.5

00.5

11.5

x

f (x) = sign g(x)( ) =+1, malicious−1, legitimate

"#$

%$

minx 'g(x ')

s.t. d(x, x ') ≤ dmax

x '

B. Biggio et al., Evasion attacks against machine learning at test time, ECML-PKDD 2013

 

http://pralab.diee.unica.it

Computing Descent Directions

Support vector machines

1.5-class MCS

g(x) = αi yik(x,i∑ xi )+ b, ∇g(x) = αi yi∇k(x, xi )

i∑

RBF kernel gradient: ∇k(x,xi ) = −2γ exp −γ || x − xi ||2{ }(x − xi )

8  

data 1C Classifier (malicious)

Feature Extraction

malicious

1C Classifier (legitimate)

2C Classifier

1C Classifier (legitimate)

legitimate

x

g1(x)

g2(x)

g3(x)

g(x) ≥ t g(x)

true

false

z(x) = g1(x), g2 (x), g3(x)!" #$T

∇g(x) = −2γ αi exp −γ z(x)− z(xi )2{ }i∑ z(x)− z(xi )( )

Τ δzδx

B. Biggio et al., Evasion attacks against machine learning at test time, ECML-PKDD 2013

 

http://pralab.diee.unica.it

Bounding the Adversary’s Knowledge Limited-knowledge attacks

•  Only feature representation and learning algorithm are known •  Surrogate data sampled from the same distribution as the

classifier’s training data •  Classifier’s feedback to label surrogate data

PD(X,Y) data  

Surrogate training data

f(x)

Send queries

Get labels Learn surrogate classifier

f’(x)

9  B. Biggio et al., Evasion attacks against machine learning at test time, ECML-PKDD 2013

 

http://pralab.diee.unica.it

Experimental Analysis

•  Two case studies: spam and PDF malware detection –  Perfect-knowledge (PK) and limited-knowledge (LK) attacks

•  Spam data (TREC ’07) –  25,220 ham and 50,199 spam emails

•  we used the first 5,000 emails in chronological order

–  2-class linear SVM, 1-class RBF SVMs

•  PDF data –  2,000 samples collected from the web and public malware

databases (e.g., Contagio) –  2-class RBF SVM, 1-class RBF SVMs

•  Experimental setup –  50% TR/TS splits, 20% TR for surrogate learning –  5-fold cross-validation to tune

10  

C,γ ∈ 2−10 ,2−9 ,...,2+10{ }

 

http://pralab.diee.unica.it

Spam Filtering

•  Features: presence/absence of words •  Attacks: bad word obfuscation / good word insertion

•  Attack strategy:

11  

Start 2007 with a bang! Make WBFS YOUR PORTFOLIO’s first winner of the year ...

startbang portfolio winneryear ... university campus

1 1111...00

x   x’  

St4rt 2007 with a b4ng! Make WBFS YOUR PORTFOLIO’s first winner of the year ... campus

startbang portfolio winneryear ... university campus

0 0111...01

minx 'g(x ')

s.t. d(x, x ') ≤ dmaxL1-­‐distance  counts  the  number  of  modified  words  in  each  spam  

 

http://pralab.diee.unica.it

Experiments on PDF Malware Detection

•  PDF: hierarchy of interconnected objects (keyword/value pairs)

•  Attack strategy

–  adding up to dmax objects to the PDF –  removing objects may

compromise the PDF file (and embedded malware code)!

/Type    2  /Page    1  /Encoding  1  …  

13  0  obj  <<  /Kids  [  1  0  R  11  0  R  ]  /Type  /Page  ...  >>  end  obj  17  0  obj  <<  /Type  /Encoding  /Differences  [  0  /C0032  ]  >>  endobj    

Features:  keyword  count  

minx 'g(x ')

s.t. d(x, x ') ≤ dmax

x ≤ x '

12  

 

http://pralab.diee.unica.it

Experimental Results

13  

0 5 10 15 20 25 300

0.2

0.4

0.6

0.8

1

maximum number of modified words

AUC 1%

(PK)

2C SVM1C SVM (L)1C SVM (M)1.5C MCS

0 5 10 15 20 25 300

0.2

0.4

0.6

0.8

1

maximum number of modified words

AUC 1%

(LK)

2C SVM1C SVM (L)1C SVM (M)1.5C MCS

Spam

 filte

ring  

0 5 10 15 20 25 300

0.2

0.4

0.6

0.8

1

maximum number of added keywords

AUC 1%

(PK)

2C SVM1C SVM (L)1C SVM (M)1.5C MCS

0 5 10 15 20 25 300

0.2

0.4

0.6

0.8

1

maximum number of added keywords

AUC 1%

(LK)

2C SVM1C SVM (L)1C SVM (M)1.5C MCSPD

F  Malware  De

tec-on

 

 

http://pralab.diee.unica.it

Conclusions and Future Work

•  1.5-class MCSs –  to improve classifier security under attack (enclosing legitimate data) –  to retain good accuracy in the absence of attack

•  General approach –  Suitable for any learning/classification algorithm (in principle) –  No specific assumption on adversarial data manipulation

•  Future work –  Formal characterization of trade-off between security and accuracy –  Robustness to poisoning attacks (training data contamination)

14  

 

http://pralab.diee.unica.it

?   Any questions Thanks  for  your  a#en-on!  

15