Understanding the World of your Enemy with I-CAT(Internet- Categorization of Attacks Toolkit)

Peter MellNIST, Computer Security Division

5-26-99

AbstractSecurity professionals need to understand the attacks and vulnerabilities utilized by

hackers to penetrate and shut down computer systems. However, security companies that collectsuch knowledge share very little of it with the general security community. The result is thatsecurity professionals must use the Internet as their source of computer attack and vulnerabilityinformation. While the Internet is a rich source of such information, the data is disorganized,distributed, and unverified. Because of the difficulty of collecting attack and vulnerabilityinformation on the Internet, security organizations spend an enormous amount of money siftingthrough the data. We are trying to mitigate this problem with the Internet- Categorization ofAttacks Toolkit (I-CAT). I-CAT is a tool that allows security professionals to quickly gleanattack and vulnerability information off the Internet. I-CAT has three main services: attackdescription lookup, statistics on the popularity of attacks, and measurements of current trends inattack publication. These three services enable security professionals to determine what kinds ofattacks are available, what specific attacks are popular, and how to find information on thepublished attacks.

1.0 IntroductionMany computer security professionals need to possess and use attack scripts. Attack scripts are

executable code or manual instructions that enable a user to compromise a computer system. Intrusiondetection companies obtain attacks in order to figure out how to detect them. Vulnerability scannercompanies obtain attacks in order to figure out what vulnerabilities they are exploiting. Penetrationexperts obtain attacks in order to perform penetration testing. Law enforcement and system administratorsobtain attacks in order to understand what attacks could have been used to penetrate a compromisedsystem.

While many security professionals feel the need to obtain attack scripts, few are willing to sharewhat they obtain. Security companies do not share in order to keep a competitive advantage. Others donot share because they fear the liability of giving away dangerous tools. Whatever the reason, very littlesharing of attacks occurs in the computer security industry.

The result is that computer security professionals must obtain attacks scripts from the Internet. Avariety of hacker and security web sites exists that publish attack scripts for the public to download.Rootshell1 has over 750 attack scripts. Fyodor’s Playhouse2 has over 350 attack scripts. The Legacy3 hasover 500 attack scripts.

While large collections of attack scripts exist on the Internet, they are either disorganized, poorlydocumented, or not searchable. The result is that security researchers spend an enormous amount of timeand money finding attack scripts. Because of the time and money involved, small companies and researchgroups can not effectively find the attack scripts that they need.

2.0 Existing Sources of Attack and Vulnerability InformationIt may seem strange that it is difficult for security professionals to find attacks on the Internet

because the Internet is such a rich source of attack and vulnerability information. Mailing lists likeBugtraq4 and NT Bugtraq5 provide daily information on the latest vulnerabilities. Attack script web siteslike Rootshell and Fyodor’s playhouse contain scripts that enable one to launch and understand the latest

computer attacks. Emergency response teams like CERT*6 provide advisories to warn systemadministrators when to patch their systems. However, each of these sources have drawbacks whichhamper their use by security professionals.

2.1 CERT (Computer Emergency Response Team)CERT maintains a public database of vulnerability information that is very useful to system

administrators seeking to protect their networks. CERT educates people about the existence of significantvulnerabilities and provides details on how to protect sites and apply patches. However, CERT does notrelease detailed information on vulnerabilities which makes it almost useless to security companies whoneed to collect attack scripts. Services similar to CERT are offered by:

• Federal Computer Incident Response Capability (FedCIRC)7

• the Forum of Incident Response and Security Teams8 (FIRST)• Computer Incident Advisory Capability (CIAC) run by the department of energy9

• the Australian Computer Emergency Response Team10 (AUSCERT)

2.2 Bugtraq and NT BugtraqBugtraq and NT Bugtraq are widely read mailing lists that discuss newly found vulnerabilities in

computer systems. They fully disclose all details in order to allow security professionals to understand thevulnerabilities. The problem with using these mailing lists is that they are high volume and the readermust wade through many documents discussing different facets of each vulnerability. While these lists arethe best source of vulnerability information on the Internet, the security community needs the informationin a more compact and organized format.

2.3 Attack Script Web SitesMany security and hacker web sites exist on the Internet that provide public access to attack

scripts. Rootshell and Infilsec11 are examples of white hat attack databases while the Legacy and Fyodor’sPlayhouse are examples of hacker related sites. While these sites offer useful information, it is often hardto find what one needs. Even though some sites may organize attacks by operating system or provide asimple text search capability on the attack descriptions, security professionals need more sophisticatedsearch techniques. Researchers need to be able to search for attacks by the goal of the attack, the targetplatform, the launching platform, the transmission method used, and many other features.

2.4 VulDa: Vulnerability DatabaseIBM, like many security companies, has developed a large database of attack and vulnerability

information. VulDa12 contains over 3.5 Gigabytes of compressed data that is available to the user usingkeyword searches and text mining techniques. While VulDa is very impressive, IBM is not ready toopenly share it to the security community. VulDa is just one example of many corporate attack andvulnerability databases that can not be released to the general security community forcing securityprofessionals to use existing resources on the Internet.

2.5 CERIAS† Vulnerability DatabaseCERIAS is making available to the security community a vulnerability database that describes

vulnerabilities at a great level of detail. It is a heavyweight solution to the problem of sharing attack andvulnerability information in that it requires a substantial amount of work to analyze and enter avulnerability into the database. If the security community would take ownership of this database then theworkload could be distributed and a very rich and detailed attack and vulnerability database would be

* CERT is the Computer Emergency Response Team† CERIAS is the Center for Education and Research in Information Assurance and Security run by PurdueUniversity

available to the security community. Until then, we must make the best of what is available on theInternet.

3.0 Our ApproachWe are working to make the attack scripts on the Internet more accessible to researchers until that

day when the security community begins to share their data. We feel that the security community needsan interim solution to be used until a database like the CERIAS database is adopted by the securitycommunity and fully populated with all known vulnerabilities.

Our solution is I-CAT, the Internet- Categorization of Attacks Toolkit. I-CAT enables a user tofind attacks published on the Internet. It can be used to find attacks that have a particular set ofcharacteristics or attacks that could penetrate a particular type of host. In addition, I-CAT monitors whichattacks scripts are most popular and takes statistics on attack publication on the Internet.

3.1 The I-CAT Attack DatabaseI-CAT contains descriptions of over 320 attack scripts published in the last 16 months. Each

attack script is categorized by over 70 characteristics. The broad categories of statistics include: scriptgoal, target type, transmission method, attacker platform, and requirements on the attacker to launch theattack.

We designed I-CAT such that each attack script could be categorized within five minutes. Thissmall amount of time needed per attack enables the maintainers to easily keep up with the pace of Internetattack publication that by our estimate is no more than 30 to 40 new attacks per month.

For the initial I-CAT development we used only attacks published on Rootshell. Rootshell is oneof the most important attack script publication sites and it publishes the majority of publicly availableattack scripts. While it is difficult to remotely measure the popularity of a web site, we know thatRootshell has close to 30,000 people on its mailing list and receives hundreds of thousands of queries toits search engine every month.

We categorize attack scripts using the I-CAT form shown in Figure 1. The example attack shownis smurf. The figure reveals that smurf is a remote denial of service attack that uses ICMP and effectsalmost all network devices. The source code for the attack was published on Rootshell on 10/30/97 underthe name “smurf.c”.

Figure 1: An Example I-CAT Attack Description Entry

Most of the fields in the attack characterization form are self explanatory, however we need tomake a few clarifications:1. The check boxes are completely dependent. Theoretically, a single attack could cause all check boxes

to be marked.2. We define an “attack toolkit” to be an attack script that exercises more than one vulnerability.3. The check box “target accesses attacker” is for attacks where the attacker can only hurt the target if

the target voluntarily visits the attacker’s host, application, or web site.4. The “get root” check box is for any attack that gains complete control of the target host. This check

box is thus applicable to both Windows and Unix machines.

3.2 Measuring the Popularity of AttacksI-CAT also measures the popularity of attack scripts published on the Internet. This is done by

measuring what people search for in the search engines of attack script web sites. Rootshell has a cgiscript which allows one to see the last 50 search requests made to the attack database with a date and timestamp included. I-CAT harvests this information periodically and determines the popularity of attacks.While we wish that an attack script web site would publish what attacks people download, thisinformation is not yet available and we are stuck using our indirect measurement method. Even thoughare measurements are indirect, we believe that they are significant. Our reasoning is that if one types thename of an attack into a search engine then they must be either planning to use the attack or elseconcerned that the attack will be used against them.

3.3 Measuring the Availability of AttacksI-CAT also measures statistics on Internet attack publication. In this case, we measure not the

attack scripts that people use but the population of attack scripts that is available for people to use. Forexample, we measure the percentage of attacks that are remote exploits, the percentage of attacks that useICMP, and the percentage of attacks that compromise routers. I-CAT provides measurements on what ispublicly available to the hacker community which is important for anyone trying to demonstrate theavailability of attack scripts on the Internet.

4.0 ResultsWe now show how to use I-CAT to find attacks that meet user defined criteria. We give two

examples in which we use I-CAT to answer the following two questions:1. What is the set of attacks that meet a particular user-defined set of criteria?2. What is the set of attacks that will penetrate a particular type of host?

Besides being useful for looking up attack scripts, I-CAT can be used to understand statistics onthe popularity of attacks and statistics on trends in attack publication. To demonstrate this, we list the top20 attack database search terms for December 1998 and give some details on the attacks found therein.Then, we give statistics on particular trends in attack publication for 1998.

4.1 Example Usage of I-CAT to Find Attacks that Meet Particular CriteriaAssume that we want to find attack scripts that enable a web site to attack its visitors.

Furthermore, suppose that we want the subset of these attacks that gives the web server complete controlover the victim. Finding these attacks on the Internet would be non-trivial but with I-CAT it is easy. InFigure 2, we show the I-CAT attack search screen. It allows the user to create arbitrary AND/ORexpressions between all fields shown. In addition, it allows the user to perform regular expressionmatching in any of the text fields.

In Figure 2 we form the search query by checking the “Remote” box to indicate that we wantremote penetration attacks. We check the “get root” box to indicate that we want only attacks that givecomplete control over the machine. We check the “target accesses attacker” box to indicate that we wantonly attacks that are launched when the target approaches the attacker in some way. We check the “website” box under attacker platform to indicate that we want only attacks that can be launched from a website. Last, we press the “apply filter” button to search the I-CAT database.

Figure 2: I-CAT Search Form, Example 1

I-CAT then returns a list of all attacks that meet the specified criteria. Figure 3 shows the firstattack in this list. The actual attack scripts can easily be found since the location of the attack ispinpointed by the fields titled “source”, “publish date”, and “attack name”. In this case, a user would loadthe Rootshell web site and type “nsover.txt” into Rootshell’s search engine to retrieve the attack script.

Figure 3: I-CAT Search Results, Example 1

4.2 Example Usage of I-CAT to Find Attacks that are Capable of Violating a Particular HostAssume that we want to find the set of attacks that could penetrate one’s multi-purpose mail

server and web server. The first step is to write down the operating system that is running on the host andany applications that offer network services. A typical list might be as follows*:

Operating System: LinuxApplications: IMAP server, Apache web server, Telnet, FTP

For each application, we query I-CAT separately to see if there exists attacks that can remotelypenetrate the host. However, it is possible to include all applications into the same query. In Figure 4, wequery I-CAT for remote penetration attacks that effect Linux operating systems and violate the IMAPapplication. We check the “Remote” box to specify a remote penetration attack. We check the “Hosts”and “Unix” boxes to indicate that we want only attacks that work against Unix hosts. We perform regularexpression matching on the “Specific OS/Version” to obtain attacks that work against Linux and attacksthat work against many operating systems. We check the “Application or Daemon” box to indicate thatthe attack should violate an application. Lastly, we do a regular expression search on the “Application orDaemon” text box to find attacks that violate IMAP.

* Unix hosts have a services file which lists all network services offered by the host

Figure 4: I-CAT Search Form, Example 2

After pressing the “Apply Filter” button, I-CAT returns the screen shown in Figure 5. The resultis two different attack scripts that will remotely penetrate a Linux host running an IMAP server. Asecurity expert would then look up the attack on the Internet using the fields “source”, “publish date”, and“attack name”. Afterwards, the I-CAT user formulates queries for the other applications running on thetarget host.

Figure 5: I-CAT Search Results, Example 2

4.3 Measurements on the Popularity of AttacksEvery month, I-CAT publishes the top 200 most requested search terms typed into the Rootshell

attack database search engine. The top 200 terms represent around 60% of the sample set. Note that weare measuring the popularity of an attack, not the usage of an attack. Attack usage statistics can beobtained from incident response teams like CERT.

In December of 1998, 33407 queries were sampled which represent 20 percent of the queriesmade that month. I-CAT estimated by looking at the date and time stamp of each query that Rootshellreceived 170,000 queries in December. Below are the top 20 search terms for December 1998. Operatingsystems are italicized. Names of attacks are bolded. Names of applications are underlined.

1. linux (2.29%)2. windows nt (2.25%)3. windows (1.45%)4. icq (1.40%)5. sendmail (1.40%)6. back orifice (1.36%)7. smurf (1.32%)8. teardrop (1.31%)9. imap (1.27%)10. “ “ (1.17%)*

11. solaris (1.10%)12. red hat (1.04%)

* 1.17 percent of the time the user of Rootshell pressed return in the search box without typing anything.

13. windows 98 (0.91%)14. netbus (0.81%)15. nuke (0.75%)16. scanner (0.75%)17. freebsd (0.72%)18. irix (0.69%)19. mscan (0.68%)20. nestea (0.66%)

Care must be taken in interpreting the search terms that are operating systems. People may besearching for attacks that are effective against a particular operating system. Alternatively, people may besearching for attacks that they can launch from a particular operating system. Since Linux is a verypopular platform for which to write attack scripts, its prominence at the top of the list is not unexpected.

The three applications that appeared on the top 20 are ICQ, Sendmail, and IMAP. ICQ is anadvanced chat program that lets Internet users find their friends when they are online. It is an importantapplication because of its popularity and because America Online uses it. Unfortunately, ICQ has hadmultiple vulnerabilities published about it in 1998. The ICQ protocol was insecure in that it let anyonepretend to be any other user. By taking on the identity of people’s friends, unethical people couldconvince others to receive and execute trojan horses. Sendmail is a very old but widely used applicationwhose enormous number of features have caused it to be consistently insecure. In 1998 it has againproved to have multiple vulnerabilities. Sendmail is proof that vulnerabilities may not disappear over timebecause as soon as the current vulnerabilities are patched, developers release new features with newvulnerabilities. IMAP is a protocol that people use to retrieve their mail. Unfortunately, a recent versionof an IMAP server contained buffer overflow problems which allow a remote user root access of themachine running the server.

Attacks that penetrate a particular application are typically named after the application. Attacksthat have unique names are typically denial of service attacks, trojan horses, and scanners. The attackslisted by name on the top 20 list can be categorized as follows.

Trojan Horse: Back Orifice, NetbusDenial of Service: Smurf, Teardrop, Nuke, NesteaScanners: Mscan

Back Orifice: Allows an attacker to remotely control a Windows 95 hostNet Bus: Allows an attacker to remotely control a Windows NT hostSmurf: Uses a network that accepts broadcast ping packets to flood the target with ping reply

packetsTeardrop: Freezes vulnerable Windows 95 and Linux hosts by exploiting a bug in the fragmented

packet re-assembly routines(Win)nuke: Freezes a Windows 95 host by sending it out of band TCP dataNestea: Variant on teardrop that freezes windows and Linux hosts

4.4 Statistics on Internet Attack PublicationVery few statistics have been taken on the nature of the attacks that are available on the Internet.

However, I-CAT offers users this type of hard data. I-CAT in a quantitative way can demonstrate theinsecurity that abounds in almost all major operating systems and applications.

Below are some sample statistics taken on the 237 attacks characterized by I-CAT in 1998.

Statistic: 29% of attacks can launch from Windows hostsLesson: One does not need to understand Unix to be dangerous anymore

Statistic: 20% of attacks are able to remotely penetrate network elementsLesson: Attacks that give remote users access to hosts are not rare

Statistic: 3% of the attacks were web sites attacking those who visited the siteLesson: Surfing the web is not a risk free activity

Statistic: 4% of attacks scan the Internet for vulnerable hostsLesson: Automated tools that find hosts which are easily compromised abound

Statistic: 5% of attacks are effective against routers and firewallsLesson: The Internet infrastructure components themselves are vulnerable to attack*

5.0 Future workI-CAT currently consists of attacks only from Rootshell. We wish to expand I-CAT to use data

from multiple attack script web sites. This will be especially useful from the standpoint of taking statisticson Internet attack publication. If other attack script web sites yield substantially different statistics thenwe will know that our statistics are the result of the publication bias of Rootshell and not representative ofInternet attack publication.

We plan on following the advisories published by organizations like CERT and to provide amapping between CERT advisories and the attack script database. Since CERT publishes advisories ononly the most important attacks and vulnerabilities, our mapping will enable security companies toanalyze the important attacks that exist in the hacker world.

We attempted to find trends in attack publication over time but since our database contains only asixteen month history we were unable to conclusively identify trends. However, we plan to publish thetrends that occur in attack publication over time as I-CAT encompasses more attacks over a larger timeperiod.

While it has been interesting to measure for what people search on Rootshell, we need to findmore measures of attack popularity. Ideal sources would be the attack download statistics from attackscript web sites and the incident statistics from computer emergency response teams.

6.0 ConclusionI-CAT has proven to be a useful tool that enables security researchers to understand the attacks

published on the Internet. By providing lists of the most popular attacks, I-CAT enables security expertsto focus on the greatest threats. By providing statistics on trends in Internet attack publication, I-CATenables security experts to understand what types of attack scripts are available and in what quantity.While I-CAT is not a silver bullet to security experts attempting to find attack and vulnerabilityinformation, it is a tool that gives security experts a new window for viewing the world of our enemy.

7.0 References 1 Rootshell, http://www.rootshell.com2 Fyodor’s Playhouse, http://www.insecure.org3 The Legacy, http://www.jabukie.com/hacking.html4 Bugtraq, http://geek-girl.com/bugtraq5 NT Bugtraq, http://listserv.ntbugtraq.com/archives/ntbugtraq.html6 Carnegie Mellon University Software Engineering Institute’s Computer Emergency Response Team (CERT),http://www.cert.org7 Federal Computer Incident Response Capability, http://www.fedcirc.gov8 Forum of Incident Response and Security Teams, http://www.first.org

* Most of these attacks are denial of service attacks and scanning attacks as opposed to penetration attacks.

9 Department of Energy’s Computer Incident Advisory Capability run out of Lawrence Livermore Laboratory,http://ciac.llnl.gov10 Australian Computer Emergency Response Team, http://www.auscert.org.au11 Infilsec, http://infilsec.com12 D. Alessandri, M. Dacier. VulDa: A Vulnerability Database. 2nd Workshop on Research with SecurityVulnerability Databases, January 21-22, 1998 at Purdue University.