the role of firewalls in network security

IN DEGREE PROJECT COMPUTER ENGINEERING,FIRST CYCLE, 15 CREDITS

, STOCKHOLM SWEDEN 2018

The Role of Firewalls in Network SecurityA Prestudy for Firewall Threat Modeling

JANI BONNEVIER

SEBASTIAN HEIMLÉN

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

Abstract

Firewalls help protect computer networks from intrusions and malware by en-forcing restrictions on what network traffic is allowed to pass through the fire-wall into the network. This thesis explores the role of firewalls in network se-curity, with the ultimate goal of advancing attempts to create a threat model forfirewalls. Five areas are explored, namely:

• Definitions of Concepts• Firewalls vs. Services as Targets for Direct Attack• The Past and Future of Firewalls• Approach to Estimating Firewall Security• Firewall Configuration and Security Policies

These areas are explored using a questionnaire survey. Each question in thequestionnaire is either tied to a particular area, or is used to evaluate the re-spondents’ credibility. The questionnaire has 15 questions, many of which askfor free text answers. The group of potential respondents consists of 209 indi-viduals, of whom about 75 % are authors of scientific articles that discuss fire-walls, penetration testing, and other relevant topics. The rest are informationsecurity professionals, journalists or bloggers of varying merit that were foundonline.

20 responses to the questionnaire were received. Responses to qualitative ques-tions were codified to produce some quantitative data.

The conclusions drawn based on the results include, among other things:

• Attackers tend to directly target network services rather than firewalls.• Respondents disagreed onwhether the role of firewalls is currently chang-ing.

• A possible approach to estimating firewall security takes into account thenetwork services that the firewall protects.

• Firewall configurations frequently do notmatch the security policies of theorganizations in which the firewalls are deployed.

Keywords

firewall; firewall configuration; threat modeling; network security; informationsecurity

Sammanfattning

Svensk titel: Brandväggars roll i nätverkssäkerhet: En förstudie för hotmodel-lering av brandväggar

Brandväggar hjälper att skydda datornätverk från intrång och skadeprogramgenomatt begränsa den trafik som tillåts passera genombrandväggen in i nätver-ket. Denna uppsats utforskar brandväggars roll i nätverkssäkerhet med måletatt göra framsteg i försök att skapa en hotmodell för brandväggar. Femområdenutforskas, nämligen:

• Definitioner av begrepp• Brandväggar kontra tjänster som mål för direkta angrepp• Brandväggens historia och framtid• Tillvägagångssätt för att estimera brandväggssäkerhet• Brandväggskonfiguration och säkerhetspolicyer

Dessa områden utforskas via en enkätstudie. Varje fråga i enkäten tillhör antin-gen ett specifikt område, eller används för att evaluera respondenternas tro-värdighet. Enkäten har 15 frågor, varav många efterfrågar fritextsvar. Gruppenpotentiella respondenter består av 209 individer, varav cirka 75 % är författareav vetenskapliga artiklar som behandlar brandväggar, penetrationstestning ochandra relevanta ämnen. Resten är professionella säkerhetskonsulter, journalis-ter eller bloggare med olika meriter inom informationssäkerhet eller nätverk.

20 svar på enkäten togs emot. Svar på kvalitativa frågor klassificerades för attproducera kvantitativ data.

Slutsatserna som drogs baserat på resultaten inkluderar bl.a.:

• Angripare tenderar att ha nätverkstjänster som sina direkta mål, snarareän brandväggar.

• Respondenterna var oense om huruvida brandväggars roll just nu förän-dras.

• Ettmöjligt tillvägagångssätt för att uppskatta brandväggssäkerhet tar hän-syn till de nätverkstjänster brandväggen skyddar.

• Brandväggskonfigurationer överrenstämmer ofta inte med säkerhetsrik-tlinjerna i de organisationer där brandväggarna är i bruk.

Nyckelord

brandvägg; brandväggskonfiguration; hotmodellering; nätverkssäkerhet; infor-mationssäkerhet

Glossary

attacker — A person who illegally gains access to and/or tampers with infor-mation in a computer system.

brute force attack — A repetitive method of trial and error used to obtain in-formation, typically a person’s username, password or cryptographic key. Soft-ware is used to generate a large number of guesses, which are then tested untilthe correct value is found.

DNS — Domain Name System. A system that maps domain names to IP ad-dresses. In the context of this thesis, DNS refers to the network services pro-vided by DNS servers.

firewall—A device that monitors traffic in and out of a local area network andeither allows or denies passage according to its configuration/ruleset.

FTP — File Transfer Protocol. In the context of this thesis, FTP refers to thenetwork services provided by FTP servers.

HTTP — Hypertext Transfer Protocol. The foundation of communication forthe World Wide Web. In the context of this thesis, HTTP refers to the networkservices provided by web servers.

ICMP — Internet Control Message Protocol. Used by network devices to com-municate outside of regular data transmission, as well as by tools such as pingand traceroute.

penetration tester; pentester—Aperson hired to conduct penetration test-ing.

penetration testing; pentesting—The authorized practice of attempting tofind vulnerabilities in a computer system or network through practical means.

SMTP — Simple Mail Transfer Protocol. A protocol for sending email. In thecontext of this thesis, SMTP refers to the network services provided by outgoingSMTP email servers.

Telnet—Aprotocol that enables remote terminal connections and interactionsbetween computers. Unencrypted and superseded by the encrypted SSH proto-col. In the context of this thesis, Telnet is referred to as a network service ratherthan a protocol.

threat model — A model containing the data required to analyze and assessthe security of an IT-system.

Table of Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Research Strategy . . . . . . . . . . . . . . . . . . . . . . . . . 31.5 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.6 Societal Benefits and Ethics . . . . . . . . . . . . . . . . . . . 41.7 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.8 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Theoretical Background . . . . . . . . . . . . . . . . . . . . . . . 52.1 Security Policies . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Firewall Fundamentals . . . . . . . . . . . . . . . . . . . . . 52.3 Research Methods and Methodologies . . . . . . . . . . . . 72.4 Practical Requirements for Scientificity . . . . . . . . . . . 102.5 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.1 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . 133.2 Practical Implementation of Research Method . . . . . . . 163.3 Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.4 Weighting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4 Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.1 Respondents’ Experience . . . . . . . . . . . . . . . . . . . . 214.2 Respondents’ Roles in Information Security . . . . . . . . . 224.3 Regularly Examines Firewalls . . . . . . . . . . . . . . . . . 234.4 Definition of a Firewall Configuration Error . . . . . . . . . 244.5 Definition of Firewall Breach . . . . . . . . . . . . . . . . . . 264.6 Firewall vs. Services as Targets for Attack . . . . . . . . . . 294.7 The Role of Firewalls Over the Last Five Years . . . . . . . 314.8 The Role of Firewalls in the Cloud . . . . . . . . . . . . . . . 324.9 Control Question 1: Firewall Ruleset . . . . . . . . . . . . . 334.10Control Question 2: Most Important Traffic to Block . . . . 354.11 Most Frequently Exposed Services . . . . . . . . . . . . . . . 364.12Most Frequently Vulnerable Services . . . . . . . . . . . . . 404.13MatchBetweenFirewallConfigurationsandSecurityPoli-

cies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424.14PercentageofFirewallswithConfiguration-PolicyMismatches 434.15CorrelationBetweenVulnerable andExposedNetworkSer-

vices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.16Correlation Between Q14 and Q15 . . . . . . . . . . . . . . . 46

i

5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.2 Research Methods . . . . . . . . . . . . . . . . . . . . . . . . 495.3 Validity and Reliability . . . . . . . . . . . . . . . . . . . . . . 505.4 Scientificity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525.5 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545.7 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Appendix A: The Questionnaire . . . . . . . . . . . . . . . . . . . . 61

ii

1 Introduction

This section introduces the thesis, the background to the problem, the problemstatement, the purpose, the research strategy, and goals of the project.

The history of computer networking in general and the Internet in particular,has consisted of one security disaster after the other, and thatwill likely continueto be the case for the foreseeable future. Over the years, a plethora of securityfeatures have been put in place to protect individuals and organizations frommalicious attacks.

One of those security features can be likened to a border control between theWildWest of the Internet and private local area networks, namely, the firewall1.A firewall is a device or computer program created to protect networks inside thefirewall from malicious traffic by filtering the traffic into and out of networks.Firewalls are, in some sense, not a core feature of the Internet; they have to bebought, installed and configured correctly to do their job properly. They alsohave to be continuously maintained. The same goes for applications that runinside the network. As a computer network grows larger and more applicationsare installed and used, the complexity of managing and updating these applica-tions, as well as the firewall, increases. Thus the risk of failing to maintain thesecurity of the network also increases as the network grows.

1.1 BackgroundThis study emerged from one company and their efforts to extend their product.

1.1.1 ForeseetiForeseeti is an IT-security company based and located in Stockholm, Sweden.Foreseetiwas founded in 2014 and strives to becomea “global leader in quantita-tive threat modeling and proactive risk management”. Foreseeti has developeda product called SecuriCAD®, which is a threat modeling and risk managementtool that can be used to analyze IT infrastructure and model threats and weak-nesses [1].

1.1.2 Problem BackgroundThe SecuriCAD® software developed by Foreseeti employs a probabilistic cal-culation engine to simulate attacks on IT infrastructure, for example corporatenetworks. This calculation engine requires a lot of data to be able to performgood attack simulations. Foreseeti reached out and wanted to investigate andobtain more data regarding firewall security; specifically the time it takes an

1The reader may think of a firewall as something that they have on their own computer. Thoseare indeed firewalls, but not of the kind this thesis is concerned with.

1

attacker to breach a firewall in the case it contains a misconfiguration. The cor-relation between firewall misconfigurations and the time to breach the firewallhas not been studiedmuch. Foreseeti wanted to conduct a quantitative study onfirewalls, the results of which could be used to directly improve SecuriCAD®.

1.2 Problem StatementA problem statement was proposed by Foreseeti and the study commenced.However, certain complications were encountered that ultimately resulted ina different problem statement.

1.2.1 Original ProblemCorporate firewalls supposedly play a part in restricting access to the organiza-tion’s local networks, protecting the company from malware and attackers [2].Configuring and managing these firewalls is, however, complex and prone tohuman error [3]. Studies [4], [5] have shown that firewalls protecting corporatenetworks are often poorly configured, which leads to security risks. The mainproblem statement requested by Foreseeti was the following.

How likely is it that a professional penetration tester can breach atypical enterprise firewall in a certain amount of time?

The statement essentially calls for the quantification of firewall security. This isa question Foreseeti have had trouble answering, and this project was initiallygoing to attempt the same.

1.2.2 Final Problem StatementAfter somework on the aforementioned problem, it became apparent during ourquestionnaire prestudy (Section 3.2.2) that it was rather complex. The state-ment seems simple enough at first glance, but only because it assumes a veryparticular way of looking at firewalls and network security. There are a numberof problems, or open questions, surrounding it that have to be answered prior toanswering the exact question posed by Foreseeti. This study is thus to be seenas a prestudy that seeks to enable answering the original problem statement, byexploring and answering the following questions.

• Definitions of Concepts: What exactly does it mean to “breach a fire-wall”? What constitutes a firewall configuration error?

• Firewalls vs. Services as Targets for Direct Attack: Do attackersusually even think of firewalls as targets for direct attack, or do they targetnetwork services?

• The Past and Future of Firewalls: Some studies of firewall securityhave been done (Section 2.5), but their relevance today depends on how

2

quickly the role of the firewall changes. How, if at all, has it changed re-cently, or will it change in the future?

• Approach to Estimating Firewall Security: Could one approach toestimating firewall security be to study the relationship between the fire-wall and the services it protects?

• Firewall Configuration and Security Policies: Howwell do firewallconfigurations match the security policies of the organizations in whichthey are deployed?

1.3 PurposeThe purpose of this thesis is to explore the role of firewalls as a security measurein corporate computer networks. This is done by researching a fewmore specificquestions concerning firewalls, such as definitions of concepts, to what degreethey are targeted by attackers, how their role is changing, how their security canbe estimated and how well configured they tend to be. This thesis aims to be astep along the way toward creating a threat model for firewalls.

1.4 Research StrategyGiven that the research is exploratory andmostly qualitative, the research strat-egy is to conduct a questionnaire survey. This was suggested by the researchgroup’s contact at Foreseeti.

1.5 GoalsSeveral goals with this thesis exist, namely:

1.5.1 AcademicThe academic goal of the thesis is to answer a problem statement by carrying outa project on a scientific basis using methods and methodologies that are provenand correct. By writing a good thesis that meets all course requirements, theauthors will finish their studies at KTH Royal Institute of Technology.

1.5.2 IndustrialThe industrial goal is to provide results that would be of use to Foreseeti in thefuture development of their threat model. Since the thesis tries to provide an-swers to problems regarding firewalls used in corporate settings, other actors inthe industry may also benefit from the findings.

1.5.3 ScientificScientifically, the goal is to carry out a valid and reliable study that can be of useto other researchers. As previously stated, this thesis can be seen as a prestudyfor future research.

3

1.6 Societal Benefits and EthicsThe results presented could possibly benefit corporations that wish to evaluatethe security of their systems. This, in turn, could be beneficial to society as awhole. Increasing amounts of people’s personal information are kept online to-day. When corporations become less vulnerable to attacks, the leakage of thispersonal information might be reduced. Higher security in corporations wouldlikely lead to less successful attacks and breaches, which would save corpora-tions and society a lot of money. Also, a large part of the Internet is made up ofcorporate networks. By securing these networks there would be fewer hosts formalware to spread through, which in turn could reduce the spreading of mal-ware in general [6].

Since firewalls are an important part of corporate network security, the datacollected as part of this project is considered sensitive. All respondents are keptcompletely anonymous because the information they might provide could po-tentially be used by bad actors to identify real, vulnerable systems.

1.7 DelimitationsThis study is concerned with only network firewalls and not personal firewalls,which are applications installed on individual workstations or laptops. Networkfirewalls, on the other hand, are software-, hardware- or cloud-based solutionsthat protect entire networks from the dangers that lurk outside. This study isnot concerned with information security matters that do not involve firewalls,unless brought up by respondents. One example of such amatterwould be socialengineering attacks.

1.8 OutlineSection 2 provides a theoretical background that some may need in order un-derstand the rest of the thesis. Moreover, it presents a theoretical overview ofresearchmethods andmethodologies that were considered for this study. It alsopresents practical requirements for scientificity in a research method. Lastly, itdiscusses previous work that is related to this thesis. Section 3 accounts for thechoice of theoretical research methods. Furthermore, each research question isdiscussed. Lastly, the practical implementation of the research method is pre-sented. In Section 4 the results of the study are presented. Section 5 discussesthe acquired results for each of the problem statements, the research methodsused in the study and the validity and reliability of the study. Furthermore, thescientificity of the study is evaluated, conclusions are presented and possiblefuture work related to the study is suggested.

4

2 Theoretical Background

This section aims to give a theoretical background needed to be able to under-stand the remainder of the thesis and also explains why our problem is a prob-lem in the first place. This section also gives a theoretical background on variousresearch methods and methodologies. The specific methodologies used in thisproject are discussed in Section 3.

Section 2.1 gives a short explanation of security policies. Section 2.2 explainswhy firewalls exist and describe a few ways of attacking networks. Section 2.3provides a theoretical overviewof researchmethods andmethodologies. Section2.4 discusses what is required for a project to be scientific. Section 2.5 is anaccount of related work and studies that have been made regarding firewallsand how this study differs from them.

2.1 Security PoliciesAn organization of sufficient size most likely has internal policies that state howcertain things should be done within the organization. One policy that alwaysshould be in place is the security policy, which is a document that states howan organization plans to protect its physical and information technology assets.The document should be monitored and updated as the organization and itssecurity requirements change [7]. The security policy should heavily affect theconfiguration of firewalls used by the organization, as they play an importantrole in the enforcement of the policy.

2.2 Firewall FundamentalsFirewalls are a fundamental part of network security and often function as thefirst line of defense, partially separating a local network from the Internet. Thetask of the firewall is to prevent unauthorizednetwork traffic frompassing throughitself into the protected network [2]. To make a firewall function properly it hasto be toldwhat traffic is to be authorized andwhat is not. This is done in a config-uration file typically called a ruleset. The ruleset is an ordered list of rules, whereeach rule is of the form predicate → action. The predicate typically contains arange of source IPs, a range of destination IPs, source port, destinationport, pro-tocol and the action typically is either accept, discard, log or a combination ofthese (see Table 1). When packets arrive at the firewall they arematched againstthis list of rules. Overlapping and conflicting rules are normal occurrences. Toresolve the conflicts, the action of the first rule that matches the packet will beenforced, therefore the order is crucial and changing the order could change thebehavior of the firewall drastically. The goal when configuring the firewall is torestrict as much traffic as possible to reduce the risk of letting malicious trafficin, while avoiding blocking legitimate traffic [8].

5

Table 1: An example firewall ruleset containing three rules.

rule Source IP Destination IP Source Port Destination Port Protocol Action#1 any 1.2.3.4 any 25 TCP accept#2 156.78.12.3 172.87.1.3 any 22 TCP accept#3 any any any any any discard

2.2.1 Ways of Attacking NetworksAn attacker must somehow either breach or circumvent the firewall to get fur-ther inside the network. A variety of ways of doing so exist.

Exposed network services constitute potential attack surfaces. A few commonexamples of network services are HTTP, FTP, SSH, SMTP and ICMP. They of-ten run inside a local area network, but they can be exposed on the Internet bythe firewall or be placed outside of the firewall entirely. In some cases, such asHTTP, this may be desirable. If a network service is exposed on the Internet,anyone (who possesses an elementary understanding of networks) can easilyfind it with a port scan and send traffic to the service. If the exposed service isvulnerable, it is an open door through which an attacker can steal information,carry out sabotage, or with some luck, reach further into the network and do thesame on an even larger portion of the network. In that case, the attacker haseffectively circumvented or breached the firewall.

Phishing is a social engineering attack inwhich an attacker typically sends emailsfrom a forged sender address to individuals working at the targeted organiza-tion, with the intent of tricking the individuals into either downloading mal-ware or sharing sensitive information with the attacker. These emails are madeto look very authentic and often claim to be very urgent. For example, the emailcould claim to be sent from a systems administrator asserting that the receiverurgently has to install a new security update that is attached to the email. Theattached file is of course not a security update, but a malicious program thatthe attacker can use to harvest credentials, monitor the receiver and potentiallyconnect the receiver’s computer to the attacker’s, thereby creating a connectionthrough the firewall [9].

Phishing is a widely used technique that takes advantage of the fact that a hu-man is much easier to manipulate and trick than a security system. Accordingto Dhamija [10], computer users, in general, lack the knowledge regarding op-erating systems and security needed to distinguish phishing from legit emailsand websites. Users are also easily deceived by the often very well made fakedimages and text in phishing attempts. By utilizing phishing, an attacker canessentially circumvent all security measures in place. It does not matter how

6

secure the perimeter of the infrastructure is, if the attacker can trick the correctperson into giving up the correct information or downloading malware.

Denial of firewalling is an attack that stems from the more general denial ofservice-attacks. During a DoF attack, carefully crafted traffic is used to overloada firewall. The overloading of the firewall has two possible outcomes. One isthat all traffic is denied, resulting in the network not being reachable; a form ofsabotage. The other, generallymore preferred outcome (for the attacker), is thatthe firewall becomes so busy that it cannot authenticate the traffic but insteadlets all traffic through. A well-crafted DoF attack can thus disable the firewallentirely, leaving the network completely vulnerable for further attacks [11].

Configuration errors are a matter of definition studied in this study. What fol-lows is therefore an introductory explanation of approximately what one mightmean by the word.

Configuration errors are either errors in the firewall ruleset or errors such asusing the default password for the administration interface, exposing the ad-ministration interface publicly on the Internet or allowing unencrypted remoteaccess to the administration interface via Telnet [4].

What specifically constitutes a firewall ruleset error depends on the wider se-curity policy of the organization in question. Any firewall ruleset that does notcomply with the organization’s stated security policy should be treated as a fire-wall ruleset error. Generally, if a firewall allows an unauthorized agent to accessinternal systems or information, it should most likely be considered a firewallruleset error.

Other configuration errors may compromise the security of the firewall inde-pendently of the actual firewall ruleset. It does not matter how well configuredthe firewalls rulesets are if the actual configuration of the firewall itself is leftinsecure. For example, exposing the administration interface publicly on theInternet is a major security issue even if the interface is password protected. Anattacker could perform a brute force attack to figure out the password and getaccess to the configuration of the firewall. Another example is using the defaultpassword for the administration interface. This renders the firewall useless ifan attacker gets access to the interface.

2.3 Research Methods and MethodologiesHåkansson has written a rather comprehensive overview of research method-ologies [12]. What follows is a summary of some of them.

7

2.3.1 Categorization of Research MethodsResearchmethods in [12] are divided into twomain categories; quantitative andqualitative. A quantitative study is concerned with collecting and/or analyzinglarge sets of concretely measurable data. A qualitative study, on the other hand,is more exploratory in nature and aims to reach tentative hypotheses and the-ories. Finally, triangulation is the practice of utilizing both quantitative andqualitative methods in order for them to complement each other.

2.3.2 Philosophical AssumptionsThe philosophical assumption is the starting point that the rest of the researchis based upon.

Positivism and Realism can be seen as quantitative assumptions, while Inter-pretivism and Criticalism are more distinctly qualitative.

Positivism assumes that “the reality is objectively given and independent of theobserver and instruments.” Therefore Positivism is useful in projects of experi-mental and testing character.

Realism assumes that there exists an objective reality independent of any ob-server or interpretation, but views scientific knowledge as a mere approxima-tion of the truth. The realist collects data by observing a phenomenon and thenworks with understanding the collected data and developing knowledge from it[12].

Interpretivism assumes that “access to reality (given or socially constructed)is only through social constructions such as language, consciousness, sharedmeanings, and instruments” [13]. They understand phenomena by interpret-ing the meaning that people assign to them.

Criticalism assumes that “the reality is socially, historically, and culturally con-stituted, produced and reproduced by people” Criticalism can be used to learnabout social, historical and cultural aspects of people and things produced bypeople [12].

2.3.3 Research MethodsResearch methods describe and provide the procedures used to accomplish theresearch task.

The Experimental research method studies reasons for, and effects of certainphenomena. It can be used to compare effects of different scenarios against eachother, given different causes. An example usage is system performance testing.

8

TheDescriptive researchmethod studies anddescribes characteristics of the dif-ferent phenomenon in different scenarios, but does not study causes and effectsof the phenomenon. The Descriptive research method often uses surveys, casestudies or observations to produce and describe representations of situations.

The Non-Experimental research method draws conclusions based on existingscenarios. The method is used to describe behavior or opinions and can for ex-ample be used to study users’ behavior or opinions.

The Conceptual research method is used to interpret existing concepts or cre-ating new ones. The Conceptual research method can for example be used toexamine content in a system.

The Fundamental research method is “curiosity-driven” and aims to generate“new ideas, principles and theories” [12]. It is used in all kinds of research anduseful for finding new questions to study more closely with other methods.

2.3.4 Research ApproachesThe research approach determines how conclusions are drawn and from whatdata. The main two approaches are the inductive and deductive ones. The in-ductive research approach aims to discover things to study and create hypothe-ses. The data are often collected using qualitative methods. The deductive re-search approach studies things that are already known to exist. Theories aretested by deducing and testing a hypothesis by using, almost exclusively, quan-titative methods with large data sets. Finally, the abductive approach is a com-bination of the two, that “derives likely conclusions from an incomplete set ofobservations” [12].

2.3.5 Research Strategy / DesignThe research strategy informs how the research is to be planned and carried outin practice. Examples of strategies listed in [12] include Experimental, Survey,Ex post facto, Case study and Exploratory.

In Experimental research, the aim is to conduct experiments where as manyvariables as possible are controlled, prove or falsify hypotheses and study rela-tionships between variables.

Surveys exist in two main varieties; cross-sectional and longitudinal. Cross-sectional surveys examine the relationships of a wide range of variables at a sin-gle point in time. Longitudinal surveys do the same over some period of time.The strategy works well for both quantitative and qualitative studies.

9

Ex post facto research is carried out after the data have already been collected.It attempts to find causal factors by studying the past.

Case study uses multiple sources of evidence in a real-life context. It is par-ticularly useful when the distinction between the studied phenomenon and itscontext is unclear.

Exploratory research aims to find relationships between as many variables aspossible, often using qualitative surveys. It identifies issues, rather than pro-vides definite answers to them.

2.3.6 Data CollectionThe data collectionmethod determines to a large degree what results can be pre-sented and what conclusions can be drawn. Common data collection methodsfor quantitative research are Observations, Experiments, Questionnaires andCase studies. For qualitative research; Questionnaires, Interviews, Observa-tions and Case studies. The quantitative methods are suited for collecting largedata sets, while the qualitative methods are more suited for collecting smallerbut more in-depth, data sets.

2.4 Practical Requirements for ScientificityAndersson and Ekholm have written a report in which they discuss scientificwork methods and the practical requirements for scientificity in projects [14].On page 17 in the report, they give an example of a scientific workmethod basedon their interpretation of a method originating from Bunge [15], in the form ofthe following series of steps.

1. Identify a problem within an area of research.2. Describe the problem carefully.3. Study existing knowledgewithin the problemarea. Find information,meth-

ods or tools that are relevant to the problem statement.4. Explain and solve the problem statement based on knowledge from step

3. If existing knowledge on the problem statement is not enough to solvethe problem statement, move to step 5. Otherwise, move to step 6.

5. Propose new ideas, theories or hypothesis and collect new empirical datato solve the problem.

6. Present either exact or approximate solutions to the problem.7. Derive the consequences of the proposed solution.8. Test the proposed solution.9. Correct the proposed solution according to the test results.10. Examine the proposed solution with respect to existing knowledge (step

3) and identify new problem statements.

10

Based on this practical sequence, Andersson and Ekholm also present a methodto evaluate the scientific quality of research projects that utilize the methodabove. The evaluation involves identifying the existence of certain aspects inthe written work that describes the project. These aspects are described by An-dersson and Ekholm as characteristic of a project with a scientific approach.

1. Background (Problem owner/stakeholders)2. Hypothesis/testable consequences3. Purpose/goals4. Delimitations5. Theory describing problem6. Research method7. Execution8. Expected result9. Achieved result

Based on whether these aspects are accounted for, one can get a hint of whethera given project used a scientific approach or not.

2.5 Related WorkWool performed two quantitative studies [4], [5] of firewall ruleset configura-tion errors in real, deployed firewalls. Both of these studies suggest that fire-walls, in general, are poorly configured and that the number of configuration er-rors is positively correlated with the number of rules in the ruleset. Wool’s stud-ies and our study are tangentially related, but while Wool studied the phenom-ena of configuration errors in great detail, our study discusses it with broaderstrokes as part of a bigger picture. Furthermore, Wool defines a firewall config-uration error as being any configuration of a set of configurations that he con-siders to be insecure in someway. Even though the research is probably the bestof its kind, the studies, as well as Wool’s set of configuration errors, are ratherold. This study aims to address both the definition of configuration errors andthe relevance of previous research.

Kamara et al. [16] propose a methodology for analyzing vulnerabilities in fire-walls. Twenty known firewall vulnerabilities are analyzed and categorized ac-cording to the suggested methodology. Our work does not analyze or categorizeany vulnerabilities, but again, discusses them in more general terms.

Alsaleh et al. [17] present a set of quantitativemetrics formeasuring the securitylevel of an enterprise firewall based on its ruleset configuration. The metric canbe used to compare the security of different firewalls against each other. Whilethe metric is useful for many purposes and certainly would be interesting to ap-ply to data such asWool’s, we cannot make use of it given our choice of research

11

methods. This study instead explores an alternative approach to estimating fire-wall security.

12

3 Method

What follows is a description of the study in terms of the methods and method-ologies discussed in Section 2.3.

The study uses triangulation, although it is somewhat more qualitative thanquantitative. It assumes the philosophical viewpoint of realism and uses thefundamental researchmethod. The study is primarily an inductive one. A cross-sectional survey, with aquestionnairedirected at information security researchersand professionals constitutes the main research strategy and data collection el-ements of the study. The quantitative data is analyzed statistically. The qual-itative data is, apart from being considered as it is, codified in order to extractquantitative data from them.

3.1 Research QuestionsIn this section, each of the research questions defined in the problem statementare discussed. Each questionnaire question is related to one of the researchquestions (Section 1.2.2).

3.1.1 Definitions of ConceptsWhat exactly does it mean to “breach a firewall”? What constitutes a firewallconfiguration error?

The project group hopes to collect definitions by experts for these concepts, notleast to see whether an established consensus already exists. The proposed def-initions are collected with the following questionnaire questions.

Q4: How do you define a firewall configuration error? (Free textanswer)

Q5: Howwould you define the act of “breaching a firewall”? (Freetext answer)

3.1.2 Firewalls vs. Services as Targets for Direct AttackDo attackers usually even think of firewalls as targets for direct attack, or dothey target network services?

The answers to this question can provide insights into how firewall securitycould be modeled. In the case that firewalls are a target for attackers, it is im-portant to consider the degree to which firewalls themselves are resistant to at-tacks. In the other case, firewall configuration is of greater importance. Par-ticularly relevant are answers from penetration testers who regularly breach or

13

circumvent firewalls. Answers to this question are collected using the followingquestionnaire question.

Q6: Do attackers tend to directly target firewalls or do they tendto target exposed network services? (Free text answer)

3.1.3 The Past and Future of FirewallsHow, if at all, has the role of firewalls changed recently, or will it change in thefuture?

The question tries to determine the degree to which previous studies on firewallsecurity are still relevant, or for how long they might remain relevant. Answersare collected using the following questionnaire questions.

Q8: Has the role of the firewalls changed during the last 5 years?How and why? (Free text answer)

Q9: Will the role of firewalls change as more companies move tothe cloud / SDN-networks? How and why? (Free text answer)

3.1.4 Approach to Estimating Firewall SecurityCould one approach to estimating firewall security be to study the relationshipbetween the firewall and the services it protects?

If it is the case that network services, rather than firewalls, are targets for directattack, one should explore ways tomodel firewall security with services inmind.To study this relationship, two sets of answers are wanted; one that determinesthe most frequently exposed network services and another that determines themost frequently vulnerable ones. The correlation between the two could then bestudied. Answers are collected using the following questionnaire questions.

Q12: In your experience, which network services in corporate net-works are most frequently exposed on the Internet? (Free text an-swer; list services in descending order of frequency)

Q13: In your experience, which network services in corporate net-works do most frequently contain vulnerabilities? (Free text an-swer; list services in descending order of frequency)

3.1.5 Firewall Configuration and Security PoliciesHow well do firewall configurations match the security policies of the organi-zations in which they are deployed?

14

One possible way of defining a firewall configuration error (whichwas one of thequestions in Section 3.1.1) is as a mismatch between the firewall configurationand the organization’s security policy. Given this definition, the project groupwants to study how well configured firewalls tend to be. Answers are collectedusing the following questionnaire questions. They are essentially the same ques-tion, formulated differently.

Q14: Howwell does the configuration of the typical perimeter fire-wall you have encountered match the organization’s security pol-icy? (Answer given as a number ranging from 1-5, where 1 represents“Several Mismatches” and 5 represents “Perfect Match”)

Q15: Approximately what percentage of perimeter firewalls thatyou have encountered have mismatches between their rulesets andthe organization’s security policy?

3.1.6 Control QuestionsThese are questions to which the answers are known, that are used to assess therespondents’ credibility.

Q10: Given this firewall ruleset, which of the following statementsare true? (Options; select all that apply. The ruleset and options areshown in Section 4.9)

Q11: Which of these types of inbound traffic is the most importantto block with a firewall? (Options; select one. The ruleset and op-tions are shown in Section 4.10)

3.1.7 Information About the RespondentA couple of questions are asked about the respondent’s role and experience inthe field of information security. They are mainly used to assess the respon-dents’ credibility.

Q1: For howmany years have you beenworkingwith, or research-ing information security?

Q2: What is your primary role within the field of information se-curity? (Options or free text answer)

Q3: Doyou regularly examine/analyze/test different firewalls? Thequestion refers to individual firewalls and their configurations; notbrands, models or types. (Yes / No)

15

3.2 Practical Implementation of Research MethodThepracticalwork of this project aimed to utilize the theoretical researchmethod(described in the beginning of Section 3) while ensuring some level of scien-tificity (discussed in Section 2.4) and keeping in concordance with the projecttriangle, as proposed by Ekholm [18].

This research project was therefore divided into three phases. The phases arelisted in chronological order and depend on each other. Each phase is an essen-tial part of the project and contributes to the end result. One of the phases wasconducted as an iterative process. What follows is a description of each phaseof the project.

3.2.1 Literature Study PhaseThe first phase of the project was the literature study, which commenced as soonas the problem statement was proposed by Foreseeti. The aim of the literaturestudy was for the authors to get a better understanding of the problem at hand.

Three main areas were researched, namely:

• Firewalls• Network security• Network architecture

The literature study was mainly conducted by reading published literature thatwas found through research libraries such as IEEE Xplore2 and ACM DigitalLibrary3. The research yielded relevant information in the form of previousstudies and references that could be used in this study, as well as perspectivesthat were used to formulate the research questions.

After researching these problem-specific areas, literature regarding scientific re-search methods and project methods was read, such as [14], [15] and [12]. Thisresearch was used to decide what methods and methodologies were appropri-ate to be used in the project and ultimately resulted in the project method nowdescribed.

3.2.2 Questionnaire Design and Publication PhaseThe literature study reinforced the choice of using a questionnaire as themethodfor data collection. The design and creation of said questionnaire became thesecond phase, which itself can be divided into three parts, carried out in theorder given below.

2https://ieeexplore.ieee.org3https://dl.acm.org/

16

https://ieeexplore.ieee.org

https://dl.acm.org/

• Questionnaire design. The initial design of the questionnaire and thequestions at hand were formed.

• Questionnaire prestudy. A prestudy was conducted, where the ques-tionnaire was tested and evaluated.

• Redesign of the questionnaire. After the prestudy, the questionnairewas redesigned according to the feedback from the prestudy.

The design of the questionnaire was done according to some tips given by Har-rison [19]. The questionnaire was kept as short as possible, with as few andas straightforward questions as possible. 15 questions in total made for a goodlength. This should be enough questions to collect the desired data, while still al-lowing the respondents to properly fill in the questionnaire in a short amount oftime. The questionnaire had to be general enough so that it could be filled out bypeople working with information security, but not necessarily strictly firewalls.

As recommended by Harrison [19], a prestudy was conducted where the ques-tionnaire was answered and discussed (separately) by two information securityprofessionals; one penetration tester from the industry and one academic pro-fessor. This was done to verify that the questionnaire was well designed, under-standable and possible to answer properly. These two discussions gave somegreat insight that was used to further improve the questionnaire prior to it be-ing sent out.

During the entirety of this project phase, another process was also carried out,namely that of finding potential respondents. In order to conduct as good ofa study as possible, it was important to find competent respondents who pos-sessed knowledge of information security in general and firewalls in particular.To help ensure that a sufficient number of responseswere received, fromvariousperspectives, the questionnaire was not solely distributed to firewall specialists.People from various professions in the field of information security were sought,although the bulk (about 75%) of potential respondents ended up being authorsof scientific articles that discuss firewalls, penetration testing and other relevanttopics. The rest were information security or network professionals, journalistsor bloggers of varying merit that were found online.

Subsequently, the questionnaire was deemed ready for publishing. The ques-tionnaire was created and distributed via Google Forms, which allowed for aneasy andquick distribution of the survey to a large number of respondents aroundthe world. It also allowed for anonymity (with some caveats, see Section 5.5.1)which was important to offer to all respondents. The questionnaire remainedopen for responses for one week. This was determined to be enough time to al-lowmost people who wanted to respond to do so. About half of the respondentssubmitted their response within the first 24 hours of the questionnaire being

17

opened.

Questionnaire design

Questionnaire prestudy

Publish questionnaire

Find potentialrespondents

Questionnaire redesign

Figure 1: Questionnaire design and publication process.

3.2.3 Result Analysis PhaseThe third phase commenced when the questionnaire was closed. Qualitative re-sponses were codified in order to extract some quantitative data from them (ex-plained further in Section 3.3). Interesting responses were quoted in the thesis.For some questionnaire questions, the results were weighted (explained furtherin Section 3.4). Additionally, correlations were studied and charts were created.

To utilize the advantages of iterative processes, the analysis of the data was donein iterations. One iteration was carried out for each of the research questionslisted in Section 3.1. In each iteration, the data from the questionnaire regard-ing that research question were analyzed and discussed. Thanks to this iterativeprocess, the project ran no risk of running out of time prior to deriving any con-clusions at all.

3.3 CodingQualitative free text answers are codified in an inductive manner, meaning thatthe set of possible coding categories for each question is not known in advance,

18

but derived from the responses. The percentage PC of responses codified undercategory C is calculated as

PC =RC

R

whereRC is the number of responses codified underC, andR is the total numberof responses (very elementary math).

For some questionnaire questions, a single response can be codified undermorethan one category. The sum of the percentages of responses codified under eachof the categories respectively may in such cases exceed 100 %. For example, ifthe categories are “Yes” and “No”, a nuanced response (e.g. “Yes, because ___,but on the other hand, ___”)may be codified as both. Amore decisively positiveresponse would be codified as simply “Yes”. In this example, PYes = RYes

R= 2

2=

100% and PNo =RNoR

= 12= 50%, and the sum would be 100% + 50% = 150% >

100%.

3.4 WeightingThe results of two of the questionnaire questions (Q12 andQ13) areweighted us-ing a custom method. The exact formula is presented in Section 4.11.1 (becauseit is quite closely tied to the specific questions), but its essence is that the valueof each response is multiplied by the sum of the respondent’s stated experience(Q1) and score on control questions (Q10 and Q11).

Experience ranges from 0 to the maximum length of a person’s career (on theorder of tens of years) and the control question score ranges from 0 to 6. Thebalance between these two aspects affects the weighting significantly and wasconsidered carefully. Ultimately, however, the current balance, where the max-imum control question score is worth as much as 6 years of experience, is ad-mittedly somewhat arbitrary. The group concluded that experience in the field,even if not directly related to firewalls, is more important for credibility than thecontrol question score. On the other hand, the control question score is arguablymore reliable, because it is not subject to fabrication in the same way that thestated amount of experience is.

The weighting is applied to a score that is based on the ranking of options inresponses (also explained in Section 4.11.1). Because the weighting method isexperimental andwithout basis in literature, the unweighted scores are also pre-sented. Furthermore, just in case the scoring system itself turns out to be flawed,results based on the number of mentions of options, regardless of ranking, arealso presented.

19

4 Result

The section begins with a presentation of the results for each individual ques-tionnaire question (Sections 4.1–4.14). Then, correlations between the resultsof some of the questions are investigated (Sections 4.15–4.16). This all buildsup to the discussion and conclusions regarding the research questions in theremainder of the thesis.

The questionnaire was sent to 246 potential respondents. At least 37 of ourrequests could not be delivered, bringing the number down to 209. 20 responseswere received, of which 15 were from academic researchers. All questions wereoptional, so not all respondents answered all questions.

4.1 Respondents’ ExperienceQ1: For howmany years have you beenworkingwith, or research-ing information security?

The average respondent claimed to have about 11 years of experience in the fieldof information security.

Table 2: Statistics for Q1.Responses 16Response rate 80 %Average 10.88Median 10Standard deviation 6.73

21

0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34

Num

berofresponses

Years of experience

Q1 - Distribution of responses

Figure 2: Chart for Q1.

4.2 Respondents’ Roles in Information SecurityQ2: What is your primary role within the field of information se-curity? (Options or free text answer)

75 % of respondents described themselves as academic researchers. The pro-portion is the same as that of authors of scientific articles among potential re-spondents (Section 3.2.2).

Table 3: Statistics for Q2.Responses 20Response rate 100 %

22

Table 4: Results for Q2.Number Percentage

Academic researcher 15 75.00 %Penetration tester 2 10.00 %Administrator 1 5.00 %Architect 1 5.00 %Journalist 1 5.00 %

0

2

4

6

8

10

12

14

16

Academic researcher Penetration tester Administrator Architect Journalist

Num

berofresponses

Q2 - Role in the field of information security


4.3 Regularly Examines FirewallsQ3: Doyou regularly examine/analyze/test different firewalls? Thequestion refers to individual firewalls and their configurations; notbrands, models or types. (Yes / No)

35 % of respondents answered “Yes” and the rest answered “No”.

23

Table 5: Statistics for Q3.Responses 20Responses by academic researchers 15Responses by others 5Response rate 100 %

Table 6: Results for Q3. Percentages of respondents who answered a certainway. The percentages are with regard to the column’s specified group.

All who responded Academic researchers OthersYes 35.00 % 33.33 % 40.00 %No 65.00 % 66.67 % 60.00 %

0

10

20

30

40

50

60

70

80

90

100

Yes No

Percentageofresponses

Q3 - Respondent regularly examines firewalls

All who respondedAcademic researchers

Others


4.4 Definition of a Firewall Configuration ErrorQ4: How do you define a firewall configuration error? (Free textanswer)

69 % of respondents mentioned mismatches between firewall rulesets and se-curity policies as either their definition or a part of their definition of a firewall

24

configuration error. Other concepts were mentioned in 44 % of responses.

A single response could be codified intomultiple categories. Thus the presentedpercentages do not necessarily add up to 100 % (Section 3.3).


Table 8: Results for Q4. Percentages of respondents who mentioned a certainconcept in their definition. The percentages are with regard to the col-umn’s specified group.

Coding category All who responded Academic researchers OthersRuleset-policy mismatch 68.75 % 75.00 % 50.00 %Ruleset too permissive 25.00 % 16.67 % 50.00 %Contradictory rules 18.75 % 16.67 % 25.00 %

0

10

20

30

40

50

60

70

80

90

100

Ruleset-policy mismatch Ruleset too permissive Contradictory rules


Q4 - Definition of firewall configuration error


Others


25

Examples of responses that were codified as “Ruleset-policy mismatch” (possi-bly among other things):

“At the highest level it is when the firewall does not implement adefined business’ security policy. Unfortunately, somebusinesses donot create such a definition leading to a problem in understandingerrors.”

“It’s either a configuration that don’t respect rules and guidelines, orconfiguration that represents contradictions with other rulsets.”

“Any configuration that does not align with the corporation’s secu-rity policy.”

Some responses were somewhat ambiguous, but if they discussed intent or ex-pectations, they were also codified as “Ruleset-policy mismatch”.

“A bug that doesn’t separate allowed from not-allowed in the in-tended way.”

“When a firewall configuration fails to perform as expected.”

Others did not mention policies at all.

“Letting unwantedpackets that threaten security enter organizations’network.”

“A man-made mistake resulting in over-permissive ruleset.”

“Any configuration that enables an unauthorized party to gain con-trol.”

4.5 Definition of Firewall BreachQ5: Howwould you define the act of “breaching a firewall”? (Freetext answer)

54 % of respondents mentioned illegitimate traffic passing through the firewallas their definition or part of their definition of what itmeans to breach a firewall.Other concepts were mentioned in 60 % of responses.

A single response could be codified intomultiple categories. Thus the presentedpercentages do not necessarily add up to 100 % (Section 3.3).

26


Table 10: Results for Q5. Percentages of respondents who mentioned a certainconcept in their definition. The percentages are with regard to thecolumn’s specified group.

Coding category All who responded Academic researchers OthersIllegitimate traffic through firewall 53.85 % 60.00 % 33.33 %Exploitation of firewall vulnerability 30.77 % 30.00 % 33.33 %Unauthorized access to systems 15.38 % 10.00 % 33.33 %Circumvention 15.38 % 20.00 % 0.00 %

27

0

10

20

30

40

50

60

70

80

90

100

Illegitimatetraffic throughfirewall

Exploitation of firewall vulnerability

Unauthorized access to systems

Circumvention


Q5 - Definition of breaching a firewall


Others


Examples of responses that were codified as “Illegitimate traffic through fire-wall” (possibly among other things):

“Illegitimate network traffic passing through the firewall ”

“A hackermanages to initiate a connection through the FW for ama-licious purpose”

“It could mean one of two things (i) using misconfigurations to getthrough it, or (ii) using a hack or exploit to corrupt or otherwise cir-cumvent awell defined set of policies, Generally the outcome is pack-ets getting to where they should not.”

Exploitations of firewall vulnerabilities were the second most mentioned con-cept (as also seen in the last of above responses).

28

“Exploiting a vulnerability in a firewall to gain access to the deviceor changing the behavior of the firewall.”

Some considered circumvention to be a “breach”. Granted, there is some am-biguity here regarding what these respondents really meant, because their an-swers were so concise.

“Devising a round-about way to bypass the firewall”.

“Bypassing firewall logic that filters unwanted packets”

4.6 Firewall vs. Services as Targets for AttackQ6: Do attackers tend to directly target firewalls or do they tendto target exposed network services? (Free text answer)

The free text answers were codified to produce the presented quantitative data.57 % of respondents said attackers tend to target services, while 36 % said at-tackers target both services and firewalls. Thus, about 93 % of respondentsmentioned network services and 36 % mentioned firewalls. No one said thatattackers only target firewalls.


Table 12: Results for Q6. Percentages of respondents who answered a certainway. The percentages are with regard to the column’s specified group.

Coding category All who responded Academic researchers OthersServices 57.14 % 70.00 % 25.00 %Both 35.71 % 30.00 % 50.00 %Firewall 0.00 % 0.00 % 0.00 %Neither 7.14 % 0.00 % 25.00 %

29

0

10

20

30

40

50

60

70

80

90

100

Services Both Firewall Neither


Q6 - Firewalls vs. services as targets for direct attack


Others


The single respondent whose answer was codified as “Neither” suggested thatphishing attacks are a far more likely and easier way of penetrating a firewallthan using malware.

One respondent whose answer was codified as “Services”, noted however thatattacks on firewalls certainly do exist, such as denial of service and other hacks.

Some simply stated “Both” as their sole answer, while others expanded uponwhat roles the alternatives play.

“Both, if the firewall can be compromised intrusion and attacks be-come relatively straightforward. Otherwise targeting insecure net-work services is a good alternative.”

“it depends on the attacker’s intention, if his target is behind a fire-wall surely hewill attack the firewall to reach it, but exposed networkservices are always a piece of cake.”

Others were more decidedly on one side.

“Hackers are not interested in attacking firewall. They are interested

30

in the services it protects.”

4.7 The Role of Firewalls Over the Last Five YearsQ8: Has the role of the firewalls changed during the last 5 years?How and why? (Free text answer)

The free text answerswere codified and resulted in the quantified data presentedbelow. 47 % claimed that the role of firewalls have changed during the last fiveyears, while 52 % claimed that it has not. The definition of “role” was purposelynot specified in this question, because what professionals in the field regard asthe role of the firewall is in itself a relevant question. In other words, omittingthe definition allowed for more diverse qualitative responses.


Table 14: Codified results for Q8. Percentage of respondents who answered acertain way. The percentages are with regard to the column’s specifiedgroup.

Coding category All who responded Academic researchers OthersYes 47.06 % 50.00 % 40.00 %No 52.94 % 50.00 % 60.00 %

Nearly half of the respondents thought that the role had changed and the otherhalf thought that it had not. Many elaborative responses arguing for both sideswere received.

One respondent claimed that the time span was too short.

“No. To short of a timespan. Increase to 10 years and firewalls hasbeen going from traditional port-based to application aware firewalls.”

Another respondent claimed that the fundamentals of the firewall hadnot changed,but it had become better at what it does.

“Fundamentally there were limited changes - it’s still a checkpointsystem. However there are quite someadvances in termsof the speed

31

of firewall checking, limited deep packet analysis, and firewall man-agement.”

One respondent claimed that firewalls now have to inspect more protocols asapplications become more advanced and hide their traffic behind generic pro-tocols.

“The technical scope in terms of the protocols inspected has becomewider. There is a tug of war between App designers who use genericprotocols such as http and the FWs that try to control meaningfulentities that try to hide behind the generic protocol.”

Others claimed that changes to network architectures have lead to consequencesregarding firewalls.

“Many services migrating to the cloud makes old firewalls less use-full.”

“Of course, because of SDN, NAT and programmable switches“

4.8 The Role of Firewalls in the CloudQ9: Will the role of firewalls change as more companies move tothe cloud / SDN-networks? How and why? (Free text answer)

The free text answerswere codified and resulted in the quantified data presentedbelow. 67 % claimed that the role of firewalls will change as we move fromtraditional network architectures to cloud-based network architectures. 25 %claimed that the role would not change and 8 % of the responses could not becodified.


32

Table 16: Codified results for Q9. Percentage of respondents who answered acertain way. The percentages are with regard to the column’s specifiedgroup.

Coding category All who responded Academic researchers OthersYes 66.67 % 75.00 % 50.00 %No 25.00 % 12.50 % 50.00 %Not codified 8.33 % 12.50 % 0.00 %

Fewer responses were received than for Q8, but therewere still some elaborativeanswers given.

One motivation for the role of firewalls not changing with cloud/SDN networkarchitectures was the following.

“I think firewalls will maintain their legacy role, as a ground-levellayer of protection, and one of many systems that can help informthe emerging generation of platform security solutions.”

Another respondent reasoned differently.

“Yes, with crucial services beingmoved to the cloud, hackers will tar-get the cloud more than the enterprise network.”

One respondent argued that the role of firewalls will not change, as they are notenough.

“i don’t think so, firewalls aren’t enough, otherwise they are indis-pensable in any good security policy. and new security solutionsused in SDN nets and Cloud are based on firewalls.”

One respondent answered yes on the question, but did not specify in what waythe cloud protection will differ from “traditional” firewalls.

“Yes, Companies will have to rely on the cloud provider protection.”

4.9 Control Question 1: Firewall RulesetQ10: Given this firewall ruleset, which of the following statementsare true? (Options; select all that apply)

33

Figure 8: The ruleset referred to in Q10.

The options were the following.

• The local network most likely uses NAT. Correct.• The ruleset most likely allows hosts on the local network to browse theWWW. Correct.

• The ruleset contains masked/overlapping rules. Correct.• The ruleset is a realistic and complete example of how a typical corporatefirewall might be configured.

• The ruleset contains a serious security flaw.

Onepointwas awarded for each statement thatwas correctly checkedor unchecked,so the highest possible score was five. There was no way of knowing whether arespondent abstained from answering or thought that all statements were incor-rect. Thus it was assumed that all respondents answered and leaving all check-boxes empty resulted in two points.

Table 17: Results for Q10. Number of points scored by respondents.Average 3.3Median 3Standard deviation 1.13

34

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6

Num

berofresponses

Score

Q10 - Distribution of scores


4.10 Control Question 2: Most Important Traffic to BlockQ11: Which of these types of inbound traffic is the most importantto block with a firewall? (Options; select one)

The options were the following.

• Telnet. Correct.• ICMP.• Whois.• FTP.


35

Table 19: Results for Q11. Percentages of respondents who answered with eachof the possible answers. The percentages are with regard to the col-umn’s specified group.

All who responded Academic researchers OthersTelnet 75.00 % 84.62 % 33.33 %FTP 0.00 % 0.00 % 0.00 %ICMP 18.75 % 15.38 % 33.33 %Whois 6.25 % 0.00 % 33.33 %

0

10

20

30

40

50

60

70

80

90

100

Telnet FTP ICMP Whois


Q11 - Most important service to block


Others


4.11 Most Frequently Exposed ServicesQ12: In your experience, which network services in corporate net-works are most frequently exposed on the Internet? (Free text an-swer; list services in descending order of frequency)

4.11.1 AnalysisResults are presented for all services that were mentioned in either Q12 or Q13.Three ways of measuring the results were used. The rationale behind thesemethods is discussed in Section 3.4.

36

• Mentions. The percentage of respondents that mentioned a given ser-vice.

• Score. For each response, the listed services were given points based ontheir position in the list. The first service was given five points, the nextwas given four points, the next three and so on. The score S for a givenservice was simply the sum total of points across all responses, accordingto the following formula.

S =5∑

p=1

pnp

where p is the number of points “awarded” based on position and np isthe number of times across all responses that the service occurred in theposition worth p points.

• Weighted score. The weighted score takes into account the respon-dents’ stated experience and number of correct answers to control ques-tions. It is scaled such that one weighted point is worth the same as oneunweighted point. The weighted score SW for a given service was calcu-lated according to the following formula.

SU =5∑

p=1

(p

np∑i=1

(Eip + Cip)

)

SW =SUSsum

SUsum

where– SU is the unscaled, weighted score.– p is the number of points “awarded” based on position.– np is the number of responses in which the service occurred in theposition worth p points.

– Eip is the experience in years of respondent number i that mentionedthe service in the position worth p points (Q1, Section 4.1).

– Cip is the number of correct answers (at most 6) to control questions(Q10, Q11) by respondent number i that mentioned the service in theposition worth p points.

– Ssum is the sum of the unweighted scores of all services.– SUsum is the sum of the unscaled, weighted scores of all services.

4.11.2 ResultsBothHTTP and SMTPwerementioned by at least half of the respondents. HTTPreceived a score of 47, SMTP about half of that and then FTP less than half ofSMTP’s score.

37


Table 21: Results for Q12.Service Mentioned by Score Weighted scoreHTTP 71.43 % 47 49.70STMP 50.00 % 25 27.50FTP 21.43 % 10 8.03Telnet 14.29 % 8 6.41WLAN 7.14 % 5 1.28VPN 7.14 % 5 8.54DNS 7.14 % 4 4.44Proxy 7.14 % 4 2.39SMB 7.14 % 2 1.71NTP 0.00 % 0 0.00NBT 0.00 % 0 0.00IRC 0.00 % 0 0.00SNMP 0.00 % 0 0.00Sum 110 110.00

38

0

10

20

30

40

50

60

70

80

90

100

HTTP SMTP FTP Telnet WLAN VPN DNS Proxy SMB NTP NBT IRC SNMP


Q12 - Most exposed network services - Mentioned by percentage of responses

Figure 11: Frequencies of services in Q12.

0

5

10

15

20

25

30

35

40

45

50

55


Score

Q12 - Most exposed network services - Score

ScoreWeighted score

Figure 12: Scores of services in Q12.

39

4.12 Most Frequently Vulnerable ServicesQ13: In your experience, which network services in corporate net-works do most frequently contain vulnerabilities? (Free text an-swer; list services in descending order of frequency)

4.12.1 AnalysisSame as Q12, see Section 4.11.1.

4.12.2 ResultsAlmost half of the respondents mentioned HTTP. The other services were men-tioned in at most 22 % of responses. In terms of score, HTTP, DNS, SMTP andFTP were found to be the most frequently vulnerable, in that order.


Table 23: Results for Q13.Service Mentioned by Score Weighted scoreHTTP 44.44 % 20 24.33STMP 22.22 % 9 8.56FTP 22.22 % 9 7.46Telnet 11.11 % 3 2.59WLAN 0.00 % 0 0.00VPN 0.00 % 0 0.00DNS 22.22 % 10 9.42Proxy 0.00 % 0 0.00SMB 11.11 % 5 3.92NTP 11.11 % 3 2.35NBT 11.11 % 2 1.57IRC 11.11 % 4 5.65SNMP 11.11 % 4 3.14Sum 69 69.00

40

0

10

20

30

40

50

60

70

80

90

100



Q13 - Most vulnerable network services - Mentioned by percentage of responses

Figure 13: Frequencies of services in Q13.

0

2

4

6

8

10

12

14

16

18

20

22

24

26


Score

Q13 - Most vulnerable network services - Score

ScoreWeighted score

Figure 14: Scores of services in Q13.

41

4.13 Match Between Firewall Configurations and Security PoliciesQ14: Howwell does the configuration of the typical perimeter fire-wall you have encountered match the organization’s security pol-icy? (Answer given as a number ranging from 1-5, where 1 represents“Several Mismatches” and 5 represents “Perfect Match”)

As can be seen below in table 24, the average andmedian rating was 2.50. Not asingle respondent answered 5, which suggests that respondents usually find atleast one mismatch.

Table 24: Statistics for Q14.Responses 14Response rate 70.00 %Average 2.50Median 2.50Standard deviation 1.09

0

1

2

3

4

1 2 3 4 5 6Several Mismatches Perfect Match

Num

berofresponses

Match between typical firewall and organization security policy


Figure 15: Results for Q14.

42

4.14 Percentage of Firewalls with Configuration-Policy MismatchesQ15: Approximately what percentage of perimeter firewalls thatyou have encountered have mismatches between their rulesets andthe organization’s security policy?

On average, respondents stated that about 59 % of firewall rulesets containmis-matches with applicable security policies.

Table 25: Statistics for Q15.Responses 11Response rate 55.00 %Average 58.64Median 50.00Standard deviation 30.67

0

1

2

0 10 20 30 40 50 60 70 80 90 100 110

Num

berofresponses

Percentage of firewalls with mismatches


Figure 16: Results for Q15.

4.15 CorrelationBetweenVulnerable andExposedNetwork ServicesHere, the results from Q12 (Section 4.11) and Q13 (Section 4.12) are compared.Somewhat of a positive correlation appears to exist between levels of exposureand vulnerability. Four services clearly stood out as the most insecure; HTTP,

43

SMTP, FTP and DNS. As showcased in Figures 17, 18 and 19, these four serviceshad the highest scores for both exposure and vulnerability. The red lines in theplots are a linear function fitted to the data by Gnuplot “using the nonlinearleast-squares Marquardt-Levenberg algorithm” [20].

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Percentageofresponsesthatmentioned

serviceasfrequentlyvulnerable

Percentage of responses that mentioned service as frequently exposed

Q12 vs Q13 - Correlation between most exposed and most vulnerable services - Mentions

HTTP

SMTPFTP

Telnet

WLANVPN

DNS

Proxy

SMBNTPNBTIRCSNMP

Figure 17: Correlation between frequency of mentions of services in Q12 andQ13.

44

0

2

4

6

8

10

12

14

16

18

20

22

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54

Vulnerabilityscore

Exposure score

Q12 vs Q13 - Correlation between most exposed and most vulnerable services - Unweighted score

HTTP

SMTPFTP

Telnet

WLANVPN

DNS

Proxy

SMB

NTPNBT

IRCSNMP

Figure 18: Correlation between unweighted scores of services in Q12 and Q13.

0

2

4

6

8

10

12

14

16

18

20

22

24

26

28

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56

Vulnerabilityscore

Exposure score

Q12 vs Q13 - Correlation between most exposed and most vulnerable services - Weighted score

HTTP

SMTPFTP

Telnet

WLAN VPN

DNS

Proxy

SMBNTPNBT

IRC

SNMP

Figure 19: Correlation between weighted scores of services in Q12 and Q13.

45

4.16 Correlation Between Q14 and Q15Here, there results for Q14 (Section 4.13) and Q15 (Section 4.14) are compared.The questions provide two ways of comparing firewall configurations and se-curity policies. In a perfect world, the typical firewall configuration would per-fectly match the security policy, and thus the rating on Q14 would be 5. In Q15the respondents were to estimate the percentage of firewalls they had encoun-tered that containedmismatches with security policies. Not a single respondentanswered 0 %, while two respondents answered 100 %.

The responses to Q14 and Q15 were expected to be negatively correlated (typicalfirewall matches policy well ⇐⇒ few firewalls have mismatches). Figure 20displays the correlation between the responses to these two questions, which itwas not as clear as expected. All in all this result suggests that firewall config-urations generally match their applicable security policies poorly. The red linein the plot is a linear function fitted to the data by Gnuplot “using the nonlinearleast-squares Marquardt-Levenberg algorithm” [20].

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5Several Mismatches Perfect Match

PercentageofFW

swithruleset-policymismatches

Ruleset-policy match for typical firewall

Q14 vs Q15 - Correlation between responses

Figure 20: Correlation between responses to Q14 and Q15.

46

5 Discussion

This section discusses the research methods applied (Section 5.2), the acquiredresults (Section 5.1) and the validity and reliability of the study (Section 5.3).The conclusion of the study is presented in Section 5.6 and possible future workthat is related to this study is presented in Section 5.7.

5.1 ResultsIn this section, the results tied to each research question are discussed.

5.1.1 Definitions of ConceptsThe definitions of two concepts were researched using the questionnaire ques-tions Q4 and Q5 (results presented in Sections 4.4 and 4.5)

Based on the results of Q4, there appears to be a consensus that a firewall con-figuration error could, at the very least, be defined as a mismatch between thefirewall ruleset and the organization’s security policy. The validity of this resultis reinforced by the fact that no such idea had been mentioned at that point inthe questionnaire; many respondents independently arrived at something ap-proximating this definition.

It just so happens that toward the end of the questionnaire, there were two ques-tions regarding how well firewall configurations match security policies (Q14,Q15). Given the definition we have now arrived at, these two questions can bethought of as “how well configured do firewalls tend to be?”.

Based on the results of Q5, a firewall breach could be defined as illegitimatetraffic passing through the firewall. Unfortunately, this definition does not statehow a breach is executed. It does, however, appear to exclude things such ascircumvention, social engineering, etc.

5.1.2 Firewalls vs. Services as Targets for Direct AttackThe sole questionnaire question for this problem was Q6 (results presented inSection 4.6). 93 % of respondents mentioned network services as targets fordirect attack, whereas only 36 % mentioned firewalls.

From this onemight conclude that services are a farmore critical part of networksecurity, but that firewalls themselves are not insignificant either.

It is possible that the phrasing of the question suggested to some that it wasan either-or question, even though it was possible to answer “Both”. Had thequestion explicitly included “Both” as an alternative, perhapsmore respondents

47

would have answered with that. On the other hand, as it is now we can be morecertain that those who did answer “Both” really meant it, rather than simplyopting for it as a safe bet.

5.1.3 The Past and Future of FirewallsQ8 and Q9 of the questionnaire were used to study this problem (results pre-sented in Sections 4.7 and 4.8) The samenumber of respondents answered “Yes”on Q8 as Q9, and a majority of those who answered “Yes” on Q8 also answered“Yes” on Q9. Fewer responses were received for Q9 than for Q8. It may be thatrespondents, in general, found it more difficult to reason about the future thanto do so about the past, which would make sense.

While the result was quite even on Q8, in the case of Q9 67 % of respondentsthought that the role of the firewall would in fact change as the network archi-tectures shift toward the cloud. This suggests that much of the research thatexists in the field may become less relevant as this shift progresses.

These results also give rise to some other questions. Will the ways of attackingnetworks change as networks become cloud-based and if so, how? Will existingsecurity measures become obsolete given the new network architecture? Giventhat corporations that move to the cloud will lose some control over securitymeasures, how can they assure their customers of the security of their data?

5.1.4 Approach to Estimating Firewall SecurityTwo questionnaire questions were used to research this problem, namely Q12and Q13 (results presented in Sections 4.11, 4.12 and 4.15).

The weighting of scores did not significantly affect the outcome, as the sameservices stood out as the most insecure. In fact, the weighted and unweightedresults may be identical within the margins of error that this study has anyway.The score-based and mention-based results were also very similar.

Somewhat of a positive correlation appeared to exist between levels of exposureand vulnerability. A possible causality could be that some services have morevulnerabilities discovered in them because they are more often exposed (we donot think that the opposite, exposure as a consequence of vulnerability, seemslikely). Arguably the services with high scores play the biggest roles for networksecurity (and by extension firewall configuration), but one can not necessarilyconclude that these services in and of themselves are the most insecure ones.

If one wishes to evaluate the security of a firewall configuration, we propose thatthe threat model would take into account a measure of the risks associated with

48

particular network services, similar to the one used here.

5.1.5 Firewall Configuration and Security PoliciesThe questionnaire questions Q14 and Q15 were used to study this topic (resultspresented in Sections 4.13, 4.14 and 4.16).

The results suggested that firewall configurations generally match their appli-cable security policies poorly. Studying the cause of this was outside the scopeof this study, but can be speculated upon. It may be difficult to translate thesecurity policy into rules in a firewall ruleset, or perhaps it is as one respondentsaid in their optional comment.

“Organizational policies for firewalls are often non-existent.”

Note that just because a firewall complieswith the security policy, does notmeanthat it is secure. A perfectly compliant firewall configuration can still be inse-cure and allow attackers to breach the network. It is then not only the firewallconfiguration, but also the organization’s security policy that needs to be re-considered. Regardless, it is still a problem if firewalls do not match securitypolicies, because then corporations may not have the security they believe theyhave.

5.2 Research MethodsThis section discusses the research methods applied in the study.

5.2.1 Research Category - Quantitative vs. QualitativeThis research was categorized as a triangulation study, or in other words, it wasboth qualitative and quantitative. That said, it did lean more in the qualitativedirection. Strictly quantitative data would simply not have been helpful in an-swering many of the problem statements. The research questions were ratherinvestigatory and qualitative research allowed for the respondents to providemore thorough andmotivated answers, with less influence from the researchers’preconceptions.

5.2.2 Research Approach - InductiveAs already stated (in Section 1.2.1), the project started with a concrete problemstatement.


However, difficulties arose when attempting to study the problem using an ab-

49

ductive approach. Thus a new set of problems statements were formulated andthe project was forced to employ a more inductive approach instead, as manyof the new problem statements were not really about measuring anything, butrather exploring what there is to measure.

5.2.3 Data Collection - QuestionnaireBecause the research was inductive and the research questions required a morequalitative approach, a questionnaire seemed more appropriate than for exam-ple an experimental approach. The issue with experiments is that they oftenutilize test benches or other experimental tools to produce results that very of-ten are quantitative [12]. Creating an experimental environment to answer theproblem statements of this particular study would be difficult and subject to theresearchers’ preconceptions. A questionnaire, on the other hand, can relativelyeasily and inexpensively be designed to collect relevant answers to the problemstatements.

5.3 Validity and ReliabilityThis section discusses the validity and reliability of the results acquired fromthis study. Shenton refers to four specific criteria that should be discussed inorder to ensure a trustworthy study; credibility, transferability, dependabilityand confirmability [21].

5.3.1 CredibilityCredibility is concernedwith the question “How congruent are the findings withreality?” [21] and is one of the most important criteria for ensuring a trustwor-thy study. Shenton provides a few examples of provisions that can be used topromote confidence in that the results of the study are trustworthy, such as thefollowing.

• Adoption of established research methods.• Developing familiarity early with participating organizations/individualsbefore the first data collection takes place.

• Establishing tactics to help ensure honesty amongst participants, for ex-ample by allowing anonymity, or by accepting rejection and not harass theparticipant with participation requests [21].

The project method used in this study adopted several of these provisions. Wellestablished researchmethods have been adopted andmotivated. Data collectionwas done via a questionnaire with a controlled group of potential respondents.The potential respondents were chosen based on their presumed skill set andability to answer the questions. The questionnaire was completely anonymous,with the ambition of allowing the participants to be honest (although anonymity

50

may not be entirely good in our case, see Section 5.5). The main reason forthe control questions was that the competence of respondents could to somedegree bedetermined. The questionnairewas distributedby email, with a single,personalized email sent to each recipient (except in one case, where two emailswere sent by mistake). There was no nagging or begging for participation.

5.3.2 TransferabilityTransferability “is concerned with the extent to which the findings of one studycan be applied to other situations” [21]. Shenton suggests that the question re-garding transferability is highly impacted by the boundaries of the study, andgives examples of a few things that need to be conveyed to the reader prior todiscussing transferability, such as the following.

• The number of organizations/individuals that participate in the study.• Any restrictions on the type of people who contributed data.• The data collection methods employed.• The number and length of data collecting sessions.• The time period over which the data were collected.

These things have been clearly addressed in this thesis.

5.3.3 DependabilityDependability is concernedwithwhether a repetition of the study, with the samemethodunder the same circumstances, gives the same result. Shenton [21] spec-ifies three sections that should exist in the scientific report to allow readers toreason about the dependability of the study, namely:

• Descriptions of methods and plans.• Detailed descriptions regarding the data gathering.• Evaluation of the process and it’s results.

This thesis includes all of these, and therefore it should be possible for anyone torepeat this study quite closely. Even if our methods themselves are dependable,our resultsmay not be, due to the small number of respondents. Thus the resultsof a repeated study may not be the same.

5.3.4 ConfirmabilityConfirmability is concerned with the objectivity of the study. Real objectivityis very hard to ensure in studies utilizing questionnaires or tests since they aredesigned by humans, and therefore the intrusion of the researchers’ biases isinevitable. A few examples of things that increase the objectivity of a study arethe following.

51

• Decisions made and methods adopted should be acknowledged and moti-vated within the report.

• Preliminary theories that were not born out of data should be discussed.• Descriptions on how research questions gave rise to work should be in-cluded in the report [21].

The chosen methods, questionnaire questions and strategies for answering theresearch questions have been motivated and described, mostly in Sections 1and 3. Many of the questions in the questionnaire allowed for free text an-swers, which allowed for objectivity, since the respondent is not forced to chooseamong answers or definitions predetermined by the researchers. Some of thequestions, for example Q9 (Section 4.8), contained certain assumptions, whichgoes against objectivity as the question itself may be biased.

5.4 ScientificityOur work quite closely matches the scientific work method described in Section2.4, particularly steps 1–6. Steps 8–10 (testing, correction and examination ofthe solution) have not been done at all; we simply present our results and con-clusions and that is all.

In table 26 the scientificity of the study has been evaluated according to the eval-uation method described in Section 2.4. The study seems to fall short with re-gard to the existence of a hypothesis; there simply is none (although the neces-sity of a hypothesis in an inductive study is not entirely evident to us).

Based on the criteria laid out by Andersson and Ekholm [14], we may concludethat our study is somewhat scientific, but not perfectly so.

Table 26: Evaluation of the scientificity of the study.Aspect Missing Mentioned Examined1 Background X2 Problem statement X3 Hypothesis X4 Purpose/goals X5 Delimitations X6 Problem theory X7 Research method X8 Execution X9 Expected result X10 Achieved result X

52

5.5 LimitationsGiven the size of the field of information security, the number of respondentsin the study was very low with only 20; a fact that significantly diminishes thereliability of the study. Possible reasons for the low response rate are discussedin 5.5.1.

Anonymity and the use of an open online questionnaire impact the credibility ofthe result negatively because it cannot be guaranteed that all actual respondentswere among the potential respondents that had been chosen by us. That said,we have no reason to suspect that the questionnaire at any point was subject touncontrolled distribution.

A prestudy was conducted to verify the first version of the questionnaire. Thesecond version of the questionnaire was not properly verified due to time con-straints, but another verification would have been valuable because the problemstatement itself was reformulated as a result of the prestudy and the changes tothe questionnaire were quite radical.

5.5.1 Possible Reasons for Few ResponsesThe participation rate of the questionnaire was 10 %, and there are many possi-ble reasons for why this might be the case.

Some people may refrain from providing information regarding their compe-tence and profession in a service (Google Forms) where they know that all of theentered data can be read by Google. The respondents are anonymous to us, butthey may not be to Google.

Email addresses to researchers were extracted from published articles they hadauthored, but it is possible that many of the researchers have since moved onand no longer use the addresses found in the articles.

Some of the potential respondents may reside in China. Considering that thecountry employs the so-called “Great Firewall of China” (a phrase used to de-scribe Internet censorship in the country [22]), there is noway of knowingwhetherour emails were delivered to these individuals.

Apart from these possibilities, the low participation rate probably just stemsfrom completely “normal” causes, such asmissing the email in the inbox, lackinginterest, not having time, or forgetting to respond.

53

5.6 ConclusionThe purpose of this study was to explore the role of the firewall in network se-curity. This was done by researching five more specific problems. Two of themwere concerned with the relationship between firewalls and network services,and it is in this area we believe this study makes its foremost contribution. Withregard to the question about firewall configurations, our results are in line withfindings from other studies, not least those byWool [4], [5]. Realistically, we donot consider our results to be that revolutionary nor reliable. What follows is ashort summary of our findings.

• Definitions of Concepts: A firewall configuration error is generally amismatch between the firewall ruleset and the organization’s security pol-icy or otherwise intended behavior.

• Firewalls vs. Services as Targets for Direct Attack: Attackers tendto directly target network services rather than firewalls, although firewallsthemselves are not an insignificant target either.

• ThePast andFutureof Firewalls: Respondents disagreed onwhetherthe role of firewalls is currently changing, but seemed to argue from dif-ferent perspectives.

• Approach to Estimating Firewall Security: A possible approach toestimating firewall security takes into account the network services thatthe firewall protects.

• Firewall Configuration and Security Policies: Firewall configura-tions frequently do not match the security policies of the organizations inwhich the firewalls are deployed.

5.7 Future WorkThis study is thought of as a prestudy for one that can answer the original prob-lem statement (Section 1.2.1), which was:


Naturally, the findings of this study should be of use in a future study that re-sumes the work on this problem. Our proposition for such a future study is thatit be an experimental one, for example:

1. Determine an appropriate definition of a firewall breach (based on Section5.1.1).

2. Set up an experimental environment with a mock network consisting ofa firewall, the most crucial services (based on Section 5.1.4) with someknown vulnerabilities and perhaps some workstations. Several network

54

and firewall configurations could be created, between which the exposureof services and/or vulnerabilities in the services vary.

3. Let penetration testers attack the network and measure the time it takesthem to breach the firewall according to the determined definition. Theservices present in the network and/or exposed by the firewall would con-stitute variables to be accounted for.

Additional questions exist that we believe could be studied more closely.

• This thesis discusses the role of traditional firewalls in traditional networkinfrastructures and also how the role of these firewalls will change as wemove toward cloud- and SDN-networks. Some respondents argued thatthe firewall will keep its main purpose to filter and authorize traffic, butthat the actual implementation and configuration of firewalls will haveto change in response to the requirements of new cloud architectures. Astudy that examines in what way these implementation and configurationchanges would best be incorporated could therefore be in place.

• Another potential study could be to compare firewall vulnerabilities in tra-ditional networks with those in the cloud, to see if the new architectureleads to different vulnerabilities or if the same security issues are presentin that environment.

55

References

[1] Foreseeti. (2018) About foreseeti. Resource accessed: 2018-03-27.[Online]. Available: https://www.foreseeti.com/about

[2] J. E. Canavan, Fundamentals of Network Security. Norwood, MA, USA:Artech House, Inc., 2001, p. 153-155.

[3] T. Wong, “On the usability of firewall configuration,” 05 2012. [Online].Available: https://cups.cs.cmu.edu/soups/2008/USM/wong.pdf

[4] A.Wool, “A quantitative study of firewall configuration errors,” Computer,vol. 37, pp. 62–67, 06 2004. [Online]. Available: http://www.eng.tau.ac.il/~yash/computer2004.pdf

[5] ——, “Trends in firewall configuration errors: Measuring the holes in swisscheese,” Internet Computing, IEEE, vol. 14, pp. 58 – 65, 09 2010.

[6] A. Zaharia, “Ransomware distribution: How one in-fection can go network-wide,” 2016, visited 2018-04-15. [Online]. Available: https://heimdalsecurity.com/blog/ransomware-distribution-one-infection-network-wide/

[7] M. Rouse. (2007) security policy. Resource accessed: 2018-06-08.[Online]. Available: https://searchsecurity.techtarget.com/definition/security-policy

[8] A. X. Liu, “Firewall policy change-impact analysis,” ACM Trans. InternetTechnol., vol. 11, no. 4, Mar. 2008. [Online]. Available: http://doi.acm.org./10.1145/2109211.2109212

[9] J. Hong, “The state of phishing attacks,” Commun. ACM, vol. 55, no. 1,pp. 74–81, Jan. 2012. [Online]. Available: http://doi.acm.org.focus.lib.kth.se/10.1145/2063176.2063197

[10] R. Dhamija, J. D. Tygar, and M. Hearst, “Why phishing works,” pp. 581–590, 2006. [Online]. Available: http://doi.acm.org/10.1145/1124772.1124861

[11] A. X. Liu, A. R. Khakpour, J. W. Hulst, Z. Ge, D. Pei, and J. Wang,“Firewall fingerprinting and denial of firewalling attacks,” Trans. Info.For. Sec., vol. 12, no. 7, pp. 1699–1712, Jul. 2017. [Online]. Available:https://doi.org/10.1109/TIFS.2017.2668602

57

https://www.foreseeti.com/about

https://cups.cs.cmu.edu/soups/2008/USM/wong.pdf

http://www.eng.tau.ac.il/~yash/computer2004.pdf

http://www.eng.tau.ac.il/~yash/computer2004.pdf

https://heimdalsecurity.com/blog/ransomware-distribution-one-infection-network-wide/

https://heimdalsecurity.com/blog/ransomware-distribution-one-infection-network-wide/

https://searchsecurity.techtarget.com/definition/security-policy

https://searchsecurity.techtarget.com/definition/security-policy

http://doi.acm.org./10.1145/2109211.2109212

http://doi.acm.org./10.1145/2109211.2109212

http://doi.acm.org.focus.lib.kth.se/10.1145/2063176.2063197

http://doi.acm.org.focus.lib.kth.se/10.1145/2063176.2063197

http://doi.acm.org/10.1145/1124772.1124861

http://doi.acm.org/10.1145/1124772.1124861

https://doi.org/10.1109/TIFS.2017.2668602

[12] A. Håkansson, Portal of Research Methods and Methodologies forResearch Projects and Degree Projects. CSREA Press U.S.A, 2013,p. 67–73, [ed] Hamid R. Arabnia, Azita Bahrami, Victor A. Clincy,Leonidas Deligiannidis, George Jandieri. [Online]. Available: http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-136960

[13] M. D. Myers, Qualitative Research in Business & Management, il-lustrated edition ed. Sage Publications Ltd, Nov. 2008, p. 38.[Online]. Available: http://www.amazon.com/exec/obidos/redirect?tag=citeulike07-20&path=ASIN/141292166X

[14] N. Andersson and A. Ekholm, “Vetenskaplighet - utvärdering av tre imple-menteringsprojekt inom it bygg & fastighet 2002,” Institutionen för Byg-gande och Arkitektur, Lunds Universitet, Tech. Rep., 2002, p 16-17.

[15] M. Bunge, Epistemology and Methodology I: Exploring the World. Rei-del, 1983, vol. 5.

[16] S. Kamara, S. Fahmy, E. Schultz, F. Kerschbaum, and M. e. Frantzen,“Analysis of vulnerabilities in internet firewalls,” vol. 22, 04 2003.[Online]. Available: https://www.cs.purdue.edu/homes/fahmy/papers/firewall-analysis.pdf

[17] M. N. Alsaleh, S. Al-Haj, and E. Al-Shaer, “Objective metrics for firewallsecurity: A holistic view,” pp. 470–477, Oct 2013. [Online]. Available:https://ieeexplore.ieee.org/document/6682762/

[18] S. Eklund, Arbeta i projekt. Studentlitteratur, 2011, vol. 4, pp. 128–129.

[19] C. Harrison, “Tip sheet on question wording,” 2007, visited 2018-04-19. [Online]. Available: https://psr.iq.harvard.edu/files/psr/files/PSRQuestionnaireTipSheet_0.pdf

[20] T. Williams and C. Kelley, “gnuplot 5.2: An interactive plottingprogram,” 05 2018, resource accessed: 2018-05-29. [Online]. Available:http://gnuplot.info/docs_5.2/Gnuplot_5.2.pdf

[21] A. K. Shenton, “Strategies for ensuring trustworthiness in qualitativeresearch projects.” Education for Information, vol. 22, no. 2, pp. 63–75,2004. [Online]. Available: http://dblp.uni-trier.de/db/journals/efi/efi22.html#Shenton04

[22] F. Shen, “Great firewall of china,” pp. 599–602, 01 2014. [On-

58

http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-136960

http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-136960

http://www.amazon.com/exec/obidos/redirect?tag=citeulike07-20&path=ASIN/141292166X

http://www.amazon.com/exec/obidos/redirect?tag=citeulike07-20&path=ASIN/141292166X

https://www.cs.purdue.edu/homes/fahmy/papers/firewall-analysis.pdf

https://www.cs.purdue.edu/homes/fahmy/papers/firewall-analysis.pdf

https://ieeexplore.ieee.org/document/6682762/

https://psr.iq.harvard.edu/files/psr/files/PSRQuestionnaireTipSheet_0.pdf

https://psr.iq.harvard.edu/files/psr/files/PSRQuestionnaireTipSheet_0.pdf

http://gnuplot.info/docs_5.2/Gnuplot_5.2.pdf

http://dblp.uni-trier.de/db/journals/efi/efi22.html#Shenton04

http://dblp.uni-trier.de/db/journals/efi/efi22.html#Shenton04

line]. Available: https://www.researchgate.net/publication/281030754_Great_Firewall_of_China

59

https://www.researchgate.net/publication/281030754_Great_Firewall_of_China

https://www.researchgate.net/publication/281030754_Great_Firewall_of_China

4/26/2018 Corporate Firewall Security Survey

https://docs.google.com/forms/d/16z1FpCL0perwfQzQZ0IjotQXaafmt7LniLitU4Y5Jug/edit 1/4

Corporate Firewall Security SurveyWe are interested in the role of firewalls in network security. Feel free to skip any question if you feel that you do not have a good answer. All answers are completely anonymous.

Definitions: "firewall" = Firewall protecting an corporate local area network. NOT a personal firewall.

This survey is conducted as part of a Bachelor's thesis by Jani Bonnevier <[email protected]> and Sebastian Heimlén <[email protected]> at KTH Royal Institute of Technology.

1. For how many years have you been workingwith, or researching information security?

2. What is your primary role within the field of information security?Mark only one oval.

Academic researcher

Penetration tester

Network/firewall administrator

Journalist / blogger

None (I do not work within the field)

Other:

3. Do you regularly examine/analyze/test different firewalls?The question refers to individual firewalls and their configurations; not brands, models or types.Mark only one oval.

Yes

No

Your OpinionsDefinitions: "firewall" = Firewall protecting an corporate local area network. NOT a personal firewall.

4. How do you define a firewall configuration error?



5. How would you define the act of "breaching a firewall"?

6. Do attackers tend to directly target firewalls or do they tend to target exposed networkservices?

7. Do firewalls instill a false sense of security? How?

8. Has the role of the firewalls changed during the last 5 years? How and why?

9. Will the role of firewalls change as more companies move to the cloud / SDN-networks?How and why?



10. Given this firewall ruleset, which of the following statements are true?

Check all that apply.

The local network most likely uses NAT.

The ruleset most likely allows hosts on the local network to browse the WWW.

The ruleset contains masked/overlapping rules.

The ruleset is a realistic and complete example of how a typical corporate firewall might beconfigured.

The ruleset contains a serious security flaw.

11. Which of these types of inbound traffic is the most important to block with a firewall?Mark only one oval.

ICMP

Telnet

Whois

FTP

Your EstimationsBase your answers to the following questions on your own experience and expert knowledge. Definitions: "firewall" = Firewall protecting an corporate local area network. NOT a personal firewall.

12. In your experience, which network services in corporate networks are most frequentlyexposed on the Internet?Specify in order, with most frequent first.

13. In your experience, which network services in corporate networks do most frequentlycontain vulnerabilities?Specify in order, with most frequent first.



Powered by

14. How well does the configuration of the typical perimeter firewall you have encounteredmatch the organization's security policy?A "perimeter firewall" is a firewall deployed between a local area network and the Internet.Mark only one oval.

1 2 3 4 5

Several mismatches Perfect match

15. Approximately what percentage of perimeterfirewalls that you have encountered havemismatches between their rulesets and theorganization's security policy?Percent, 0-100. A "perimeter firewall" is afirewall deployed between a local area networkand the Internet.

16. Optional commentThank you for answering the survey! If you have any comments, feel free to post them below.

TRITA TRITA-EECS-EX-2018:134

www.kth.se

the role of firewalls in network security

Documents