Malicious SoftwareMalicious SoftwareCIS 4361CIS 4361

Eng. Hector M Lugo-Cordero, MSEng. Hector M Lugo-Cordero, MS

Feb. 2012Feb. 2012

Computer Security: Computer Security: Principles and PracticePrinciples and Practice

First EditionFirst Edition

by William Stallings and Lawrie Brownby William Stallings and Lawrie Brown

Lecture slides by Lawrie BrownLecture slides by Lawrie Brown

Chapter 7 – Chapter 7 – Malicious SoftwareMalicious Software

Most Slides are FromMost Slides are From

Malicious SoftwareMalicious Software

programs exploiting system vulnerabilitiesprograms exploiting system vulnerabilities known as malicious software or malwareknown as malicious software or malware

program fragments that need a host programprogram fragments that need a host program• e.g. viruses, logic bombs, and backdoors e.g. viruses, logic bombs, and backdoors

independent self-contained programsindependent self-contained programs• e.g. worms, botse.g. worms, bots

replicating or notreplicating or not sophisticated threat to computer systemssophisticated threat to computer systems

Malware TerminologyMalware Terminology

VirusVirus WormWorm Logic bombLogic bomb Trojan horseTrojan horse Backdoor (trapdoor)Backdoor (trapdoor) Mobile codeMobile code Auto-rooter Kit (virus generator)Auto-rooter Kit (virus generator) Spammer and Flooder programsSpammer and Flooder programs KeyloggersKeyloggers RootkitRootkit Zombie, bot Zombie, bot

VirusesViruses

piece of software that infects programspiece of software that infects programs modifying them to include a copy of the virusmodifying them to include a copy of the virus so it executes secretly when host program is runso it executes secretly when host program is run

specific to operating system and hardwarespecific to operating system and hardware taking advantage of their details and weaknessestaking advantage of their details and weaknesses

a typical virus goes through phases of:a typical virus goes through phases of: dormantdormant propagationpropagation triggeringtriggering executionexecution

Virus StructureVirus Structure

components:components: infection mechanism - enables replicationinfection mechanism - enables replication trigger - event that makes payload activatetrigger - event that makes payload activate payload - what it does, malicious or benignpayload - what it does, malicious or benign

prepended / postpended / embedded prepended / postpended / embedded when infected program invoked, executes when infected program invoked, executes

virus code then original program codevirus code then original program code can block initial infection (difficult)can block initial infection (difficult) or propogation (with access controls)or propogation (with access controls)

Virus StructureVirus Structure

Compression VirusCompression Virus

Virus MutationVirus Mutation

From Szor and Ferrie, From Szor and Ferrie, “Hunting for Metamorphic”“Hunting for Metamorphic”

Virus ClassificationVirus Classification

boot sectorboot sector file infectorfile infector macro virusmacro virus encrypted virusencrypted virus stealth virusstealth virus polymorphic viruspolymorphic virus metamorphic virusmetamorphic virus

Macro VirusMacro Virus

became very common in mid-1990s sincebecame very common in mid-1990s since platform independentplatform independent infect documentsinfect documents easily spreadeasily spread

exploit macro capability of office appsexploit macro capability of office apps executable program embedded in office docexecutable program embedded in office doc often a form of Basicoften a form of Basic

more recent releases include protectionmore recent releases include protection recognized by many anti-virus programsrecognized by many anti-virus programs

E-Mail VirusesE-Mail Viruses

more recent developmentmore recent development e.g. Melissae.g. Melissa

exploits MS Word macro in attached docexploits MS Word macro in attached doc if attachment opened, macro activatesif attachment opened, macro activates sends email to all on users address listsends email to all on users address list and does local damageand does local damage

then saw versions triggered reading emailthen saw versions triggered reading email hence much faster propagationhence much faster propagation

Virus CountermeasuresVirus Countermeasures

prevention - ideal solution but difficultprevention - ideal solution but difficult realistically need:realistically need:

detectiondetection identificationidentification removalremoval

if detect but can’t identify or remove, must if detect but can’t identify or remove, must discard and replace infected programdiscard and replace infected program

Anti-Virus EvolutionAnti-Virus Evolution

virus & antivirus tech have both evolvedvirus & antivirus tech have both evolved early viruses simple code, easily removedearly viruses simple code, easily removed as become more complex, so must the as become more complex, so must the

countermeasurescountermeasures generationsgenerations

first - signature scannersfirst - signature scanners second - heuristicssecond - heuristics third - identify actionsthird - identify actions fourth - combination packagesfourth - combination packages

Generic DecryptionGeneric Decryption

runs executable files through GD scanner:runs executable files through GD scanner: CPU emulator to interpret instructionsCPU emulator to interpret instructions virus scanner to check known virus signaturesvirus scanner to check known virus signatures emulation control module to manage processemulation control module to manage process

lets virus decrypt itself in interpreterlets virus decrypt itself in interpreter periodically scan for virus signaturesperiodically scan for virus signatures issue is long to interpret and scanissue is long to interpret and scan

tradeoff chance of detection vs time delaytradeoff chance of detection vs time delay

Digital Immune SystemDigital Immune System

Behavior-Blocking SoftwareBehavior-Blocking Software

WormsWorms

replicating program that propagates over netreplicating program that propagates over net using email, remote exec, remote login using email, remote exec, remote login

has phases like a virus:has phases like a virus: dormant, propagation, triggering, executiondormant, propagation, triggering, execution propagation phase: searches for other systems, propagation phase: searches for other systems,

connects to it, copies self to it and runsconnects to it, copies self to it and runs

may disguise itself as a system processmay disguise itself as a system process concept seen in Brunner’s “Shockwave Rider”concept seen in Brunner’s “Shockwave Rider” implemented by Xerox Palo Alto labs in 1980’simplemented by Xerox Palo Alto labs in 1980’s

Morris WormMorris Worm

one of best know wormsone of best know worms released by Robert Morris in 1988released by Robert Morris in 1988 various attacks on UNIX systemsvarious attacks on UNIX systems

cracking password file to use login/password cracking password file to use login/password to logon to other systemsto logon to other systems

exploiting a bug in the finger protocolexploiting a bug in the finger protocol exploiting a bug in sendmailexploiting a bug in sendmail

if succeed have remote shell accessif succeed have remote shell access sent bootstrap program to copy worm oversent bootstrap program to copy worm over

Worm Propagation ModelWorm Propagation Model

Recent Worm AttacksRecent Worm Attacks Code RedCode Red

July 2001 exploiting MS IIS bugJuly 2001 exploiting MS IIS bug probes random IP address, does DDoS attackprobes random IP address, does DDoS attack consumes significant net capacity when activeconsumes significant net capacity when active

Code Red II variant includes backdoorCode Red II variant includes backdoor SQL SlammerSQL Slammer

early 2003, attacks MS SQL Serverearly 2003, attacks MS SQL Server compact and very rapid spreadcompact and very rapid spread

MydoomMydoom mass-mailing e-mail worm that appeared in 2004mass-mailing e-mail worm that appeared in 2004 installed remote access backdoor in infected systemsinstalled remote access backdoor in infected systems

Worm TechnologyWorm Technology

multiplatformmultiplatform multi-exploitmulti-exploit ultrafast spreadingultrafast spreading polymorphicpolymorphic metamorphicmetamorphic transport vehiclestransport vehicles zero-day exploit zero-day exploit

Worm propagation processWorm propagation process

Find new targetsFind new targets IP random scanningIP random scanning

Compromise targets Exploit vulnerability Trick users to run

malicious code -- Spam

Newly infected join infection army

Dr Zou’s CAP6135 class

05:29:00 UTC, January 25, 05:29:00 UTC, January 25, 20032003

[from Moore et al. “The Spread of the Sapphire/Slammer Worm”]

30 Minutes Later30 Minutes Later

[from Moore et al. “The Spread of the Sapphire/Slammer Worm”]

Size of circles is logarithmic inthe number of infected machines

Worm CountermeasuresWorm Countermeasures

overlaps with anti-virus techniquesoverlaps with anti-virus techniques once worm on system A/V can detectonce worm on system A/V can detect worms also cause significant net activityworms also cause significant net activity worm defense approaches include:worm defense approaches include:

signature-based worm scan filteringsignature-based worm scan filtering filter-based worm containmentfilter-based worm containment payload-classification-based worm containmentpayload-classification-based worm containment threshold random walk scan detectionthreshold random walk scan detection rate limiting and rate haltingrate limiting and rate halting

reCaptchasreCaptchas

Generate a question easy to be answered by a Generate a question easy to be answered by a human, hard by machineshuman, hard by machines Text spellingText spelling Image associationImage association Audio/visual mixtureAudio/visual mixture Semantic/Analogy questions (e.g. which does not Semantic/Analogy questions (e.g. which does not

belong)belong)

Google provides access to its reCaptcha Google provides access to its reCaptcha implementationimplementation http://www.google.com/recaptchahttp://www.google.com/recaptcha

reCaptchas by ExamplereCaptchas by Example

Proactive Worm ContainmentProactive Worm Containment

Viruses vs. WormsViruses vs. Worms

VIRUSVIRUS Propagates by infecting other Propagates by infecting other

programsprograms

Usually inserted into host code Usually inserted into host code (not a standalone program)(not a standalone program)

WORMWORM Propagates automatically by Propagates automatically by

copying itself to target systemscopying itself to target systems Is a standalone programIs a standalone program

Sometime it is hard to distinguish virus or worm

BotsBots

program taking over other computersprogram taking over other computers to launch hard to trace attacksto launch hard to trace attacks if coordinated form a botnetif coordinated form a botnet characteristics:characteristics:

remote control facilityremote control facility• via IRC/HTTP etcvia IRC/HTTP etc

spreading mechanismspreading mechanism• attack software, vulnerability, scanning strategyattack software, vulnerability, scanning strategy

various counter-measures applicablevarious counter-measures applicable

RootkitsRootkits

set of programs installed for admin accessset of programs installed for admin access malicious and stealthy changes to host O/Smalicious and stealthy changes to host O/S may hide its existencemay hide its existence

subverting report mechanisms on processes, files, registry subverting report mechanisms on processes, files, registry entries etcentries etc

may be:may be: persisitent or memory-basedpersisitent or memory-based user or kernel modeuser or kernel mode

installed by user via trojan or intruder on systeminstalled by user via trojan or intruder on system range of countermeasures neededrange of countermeasures needed

Example of Rootkit (TDL4)Example of Rootkit (TDL4)

From the Rootkit.Win32.TDSS familyFrom the Rootkit.Win32.TDSS family Installs in Master Boot RecordInstalls in Master Boot Record Runs before the Operating SystemRuns before the Operating System Blocks programs from runningBlocks programs from running Delivers advertisementsDelivers advertisements Google redirectsGoogle redirects Keeps a copy of payload in MBR so it can be reinstalledKeeps a copy of payload in MBR so it can be reinstalled Best way to get rid of it is by replacing the MBRBest way to get rid of it is by replacing the MBR Previous versions (infecting drivers) could be removed Previous versions (infecting drivers) could be removed

with TDSSKiller from Kasperry groupwith TDSSKiller from Kasperry group

Rootkit System Table ModsRootkit System Table Mods

Traditional Defense Traditional Defense ApproachesApproaches

Analyzing rootkits behaviorsAnalyzing rootkits behaviors Examples: Panorama, HookFinder, K-TracerExamples: Panorama, HookFinder, K-Tracer

Search common symptoms on infected Search common symptoms on infected computerscomputers Examples: Copilot, SBCFI, VMwatcherExamples: Copilot, SBCFI, VMwatcher

Preserve kernel code integrityPreserve kernel code integrity Examples: SecVisor, Patagonix, NICKLEExamples: SecVisor, Patagonix, NICKLE Can be bypassed by return-oriented rootkits Can be bypassed by return-oriented rootkits

• Hijack function pointers or return addressesHijack function pointers or return addresses• Utilize kernel code snippetsUtilize kernel code snippets

SummarySummary

introduced types of malicous softwareintroduced types of malicous software incl backdoor, logic bomb, trojan horse, mobileincl backdoor, logic bomb, trojan horse, mobile

virus types and countermeasuresvirus types and countermeasures worm types and countermeasuresworm types and countermeasures botsbots rootkitsrootkits