troguard - acsac 2017 · 3 rank name ... gcalctool, edenmath,speedcrunch, kcalc, keurocalc,...

36
TROGUARD: ContextAware Protec6on Against WebBased Socially Engineered Trojans Rui Han, Alejandro Mesa, University of Miami Mihai Christodorescu, QualComm Research Saman Zonouz, Rutgers University

Upload: lethien

Post on 15-Jun-2018

218 views

Category:

Documents


0 download

TRANSCRIPT

TROGUARD:    Context-­‐Aware  Protec6on  Against    Web-­‐Based  Socially  Engineered  Trojans    Rui  Han,  Alejandro  Mesa,    University  of  Miami  Mihai  Christodorescu,                QualComm  Research  Saman  Zonouz,                                        Rutgers  University          

Mo#va#on    •  Waterfall  screen  saver  Trojan  

2  

Mac  OS  threats  

3  

Rank   Name   Percentage  

1   Trojan.OSX.FakeCo.a   52%  2   Trojan-­‐Downloader.OSX.Jahlav.d   8%  3   Trojan-­‐Downloader.OSX.Flashfake.ai   7%  4   Trojan-­‐Downloader.OSX.FavDonw.c   5%  5   Trojan-­‐Downloader.OSX.FavDonw.a   2%  6   Trojan-­‐Downloader.OSX.Flashfake.ab   2%  7   Trojan-­‐FakeAV.OSX.Defma.gen   2%  8   Trojan-­‐FakeAV.OSX.Defma.f   1%  9   Exploit.OSX.Smid.b   1%  10   Trojan-­‐Downloader.OSX.Flashfake.af   1%  

McAfee  an6virus  solu6on:  hVp:www.securelist.com  

Example  Malwares    

4  

Malware   Descrip#ons   PlaTorm  TrojanClicker.VB.395    

Trojan  socially  engineered  as  adobe  flash  update  

Windows    and  Mac  OS  X  

Faked  An6-­‐Virus    Trojan  or  Adware  socially  engineered  as  an6-­‐virus  so]ware    

Windows,  Mac  OS  X,  and  Linux  

Opfake  Browser  Malware  socially  engineered  as  Opera  Browser  

Android  

WireLuker  Legi6mate  applica6ons  socially  engineered  with  ad-­‐wares  and  Trojan    

Mac  OS  X  and  iOS  

Contribu#ons    

• Answer  the  ques6on:  “Is  this  program  doing  what  I  expected  it  to  do?”  • Bridge  the  seman6c  gap  between  func6onality  classes  and  low  level  behaviors  • Built  on  100  Linux  app  profiles    • High  detec6on  rate  on  50  Trojan  apps  

5  

TROGUARD  Architecture    TROGUARD

Offline

Online

Application Functionality

Tracing

Application Database

Functionality Class Profile Generation

Dynamic Functionality

Feature Extraction

Inference of Perceived

Functionality Class

Downloaded Application

Application Functionality

Tracing

Alert

Real-Time Classification

Application Functionality

Profile Database

Download Website

6  

sandbox  

Key  Premise  •  TROGUARD  detects  Trojans  based  on  the  premise  that  applica6ons  with  similar  func#onali#es  expose  similar  system-­‐level  behaviors  

•  Applica6ons  with  similar  func6onali6es  belong  to  a  func#onality  class,  they  should  exhibit  common  system  level  behaviors  •  Learn  web-­‐browser  behavior  of  well  know  instances(e.g.,  Firefox  and  Chrome)  •  Compare  the  web-­‐browser  profile  with  the  behaviors  of  unknown  downloaded  web-­‐browser  app  

7  

Func#onality  class  •  It  represents  both  user’s  understanding  of  so]ware  category  and  the  system’s  observa6on  of  a  so]ware  execu6on  behavior  

8  

Func#onality  class   soZpedia.com   download.cnet.com   tucows.com  

Graphic  Editor   Ar6s6c  so]ware   Graphic  Design  SW   Design  tools  

Game   Games     Games   Games  

Browser   Internet   Browsers  Internet  

Instant  Messenger   Communica6ons   Communica6ons  

Media  Player  

Mul6media  MP3/Audio  So]ware  

Audio/Video  Audio  Editor  

Video  Editor  

Text  Editor   office   Produc6vity  so]ware   Business  

IDE   Programming   Developer  Tools   Develop/Web  

Calculator   U6li6es   U6li6es   Home/Educa6on  

Applica#ons  Class   Studied  Applica#ons  

1.  Graphic  Editor   gimp,  pinta,  imagej,  inkscape,  kolourpaint,  rawtherapee,  mypaint,    gpaint,  gnome-­‐paint,  pencil    

2.  Games     sol,  wesnoth,  glchess,  neverball,  kmahjongg,  supertuxkart,  hedge-­‐  wars,  pingus,  frozen-­‐bubble,  eboard    

3.  Browser   chrome,  firefox  ,  opera,  epiphany,  midori,  chromium,  netsurf,  arora,  xxxterm,  rekonq    

4.  Instant  Messenger  

skype,  kmess,  emesene,  kopete,  pidgin,  psi,  gajim,  empathy,  amsn,  qu#m    

5.  Media  Player     smplayer,  vlc,  audacious,  quodli-­‐  bet,  gmusicbrowser,  qmmp,  abraca,  amarok,  guayadeque,  aqualung    

6.  Audio  Editor     audacity,  avidemux,  dvbcut,  og-­‐  gconvert,  kwave,  wavbreaker,  mp3splt-­‐gtk,  mhwaveedit,  fillmore,  soundconverter    

7.  Video  Editor     openshot,  lives,  iriverter,  kino,  pi#vi,  videocut,  winff,  arista-­‐gtk,  kdenlive,  curlew    

8.  Text  Editor   kile,  geany,  texmaker,  calligra-­‐  words,  soffice.bin,  lyx,  tea,  jed,  emacs,  vi    

9.  IDE      

anjuta,  codelite,  codeblocks,  net-­‐  beans,  monodevelop,  kdevelop,  spyder,  monkeystudio,  drracket,  idle    

10.  Calculator   grpn,  gcalctool,  EdenMath,  speed-­‐  crunch,  kcalc,  keurocalc,  extcalc,  gip,  galculator,  gnome-­‐genius    

9  

Func#onality  Tracing  

• Manual  tes6ng  • Run  60  seconds  for  each  applica6on  • System  call  trace  • User-­‐space  informa6on    User  interacBvity    Resource  consumpBon    IP  addresses  and  port  number  

10  

Feature  Extrac#on    

• Processing  tracing  data  • Four  groups  of  feature  file  system  Network  resource  usage  user  interacBvity  

• Intermediate  feature  11  

Intermediate  Features  

12  

Example:    if(  libssl3.so  &  fd  =  sys_socket(AF_INET,  ..)              &sys_write(fd,  ..)  &  sys_read(fd,  ..)  )  

           HTTP  =  true  

TROGUARD  Architecture    TROGUARD

Offline

Online

Application Functionality

Tracing

Application Database

Functionality Class Profile Generation

Dynamic Functionality

Feature Extraction

Inference of Perceived

Functionality Class

Downloaded Application

Application Functionality

Tracing

Alert

Real-Time Classification

Application Functionality

Profile Database

Download Website

13  

Web  Page  Analysis    

• Give  the  explicit  func6onality  class  • Web  page  contents  analysis    • OCR  to  extract  the  texts  in  the  images  • Analysis  based  on  keywords  

14  

User  Interface  •  Browser  extension  for  web  page  analysis    

15  

Sandboxing  

• SELinux  sandbox  • One  policy  for  each  app  class  • Automa6cally  generated  by  parsing  all  the  logs  from  an  app  class  

16  

Classifier  Evalua#on  

•   600  data  points  (10  second  each)  • 10  fold  cross  valida6on  • 5  classifiers  with  different  feature  group  • Precision  • Recall  • Confusion  Matrix  

17  

Precision    

18  

0

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

File Network CPU-Mem Interaction All

Prec

isio

n

Different Attributes Domains

BrowserOffice

IM

GameIDE

Media-Player

Graphic-EditorVideo-EditorAudio-Editor

CalculatorAverage

Recall  

19  

0

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

File Network CPU-Mem Interaction All

Rec

all

Different Attributes Domains

BrowserOffice

IM

GameIDE

Media-Player

Graphic-EditorVideo-EditorAudio-Editor

CalculatorAverage

Confusion  Matrices  

20  

File  features   Network  features  

Resource  usage  features   User  interac6vity  features  

Confusion  Matrices  

21  

Intermediate  Feature  Results  

22  

 54  90%  

   60  100%  

 57  95%  

 42  70%  

   60  100%  

 57  95%  

 58  97%  

 58  97%  

 56  93%  

 54  90%  

   6  

   3  

   8  

   5  

   3  

   5      5  

   1  

   1      1      1  

   2  

   1      1  

   0      0      0      0      0      0      0      0  

   0      0   0   0   0   0   0  

0   0   0   0  

0   0  

0  

0   0   0   0   0  

0  

0   0  

0   0  

0   0   0   0   0  

0   0   0  

0   0   0  

0   0   0  

0   0   0  

0   0   0  

0   0  

0  

0   0   0  

0  

0  

0  

0   0   0   0  

0  

0  

0  

0  

0  

0  

0   0  

0  0  

0  

Web  Page  Analysis  Accuracy  

23  

•  100  Web  page,  20  categories  

0

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Anti-virus

Ebook

Email

Media Player

Themes

Downloader

Driver

Calculator

GameOffice

Browser

Video Editor

Audio Editor

Database

IDEP2P App

IM Graphics

PDF Reader

Network

Education

Acc

urac

y

Text analysisOCR analysis

Case  Study  •  10  benign  apps  ×5  payload  =  50  Trojans    

 

24  

Func#onality  Class   Applica#on   Metasploit  Payload  1.  Graphic  Editor   gpaint  

linux/x86/shell_bind_tcp  linux/x86/shell/reverse_tcp  linux/x86/vncinject/bind_tcp  

linux/x86/meterpreter/bind_tcp    linux/x86/download_exec  

2.  Games     eboard  3.  Browser   xxxterm  4.  Instant  Messenger   psi  5.  Media  Player     qmmp  6.  Audio  Editor     winff  7.  Video  Editor     fillmore  8.  Text  Editor   tea  9.  IDE     spyder  10.  Calculator   gnome-­‐genius  

Case  Study    •  Predefined  acceptance  rate  0.8  

25  

0"

0.2"

0.4"

0.6"

0.8"

1"

True"Posi1ve"

False"Posi1ve"

Precision" Recall" F9Measure"

Rate"

Trojans" Benign"Apps"

Symbolic  Execu#on  •  Tested  Core  U6li6es  (four  func6onality  classes)    Dirlist  Filetype  Userinfo    Systeminfo  

•  Features  collected  from  symbolic  execu6on  give  us  52%  precision  •  Features  collected  from  user  execu6on  give  us  76%  precision  

26  

User  Execu#on  VS  Symbolic  Execu#on  

27  

Performance  Overhead  

28  

•  CPU  usage:    No-LTTng LTTng

Performance  Overhead  

29  

•  Memory  usage:    No-LTTng LTTng

Performance  Overhead  

30  

•  Disk  throughput:      No-LTTng LTTng

Performance  Overhead  

31  

•  Network  throughput:    No-LTTng LTTng

Conclusions    •  TROGUARD  detects  Trojans  based  on  the  premise  that  applica6ons  with  similar  func#onali#es  expose  similar  system-­‐level  behaviors  •  TROGUARD  can  detect  Trojan  applica6on  download  by  bridging  the  gap  between  the  user  perceived  func6ons  and  genuine  so]ware  func6ons  

32  

33  

Rui  Han  [email protected]  

Conclusions    •  TROGUARD  detects  Trojans  based  on  the  premise  that  applica6ons  with  similar  func#onali#es  expose  similar  system-­‐level  behaviors  •  TROGUARD  can  detect  Trojan  applica6on  download  by  bridging  the  gap  between  the  user  perceived  func6ons  and  genuine  so]ware  func6ons  

34  

Symbolic  Execu#on  Code  Coverage  

35  

SE  Code  Coverage  Evolu#on  

36