Assessing the Effectiveness of Attack Detection at aHackfest on Industrial Control Systems
Sridhar Adepu and Aditya MathuriTrust Center for Research in Cyber Security
Singapore University of Technology and DesignSingapore
Email adepu sridharmymailsutdedusg aditya mathursutdedusg
AbstractmdashA hackfest named SWaT Security Showdown (S3)has been organized consecutively for two years S3 has enabledresearchers and practitioners to assess the effectiveness of meth-ods and products aimed at detecting cyber attacks launchedin real-time on an operational water treatment plant namelySecure Water Treatment (SWaT) In S3 independent attack teamsdesign and launch attacks on SWaT while defence teams protectthe plant passively and raise alarms upon attack detectionAttack teams are scored according to how successful they arein performing attacks based on specific intents while the defenseteams are scored based on the effectiveness of their methods todetect the attacks This paper focuses on the first two instances ofS3 and summarizes the benefits of hackfest and the performanceof an attack detection mechanism named Water Defense thatwas exposed to attackers during S3
Index TermsmdashAttack Detection Capture-The-Flag (CTF)Cyber-physical Attacks Cyber-Physical Systems Cyber SecurityIndustrial Control Systems Hackfest Water Defense WaterTreatment Plant
I INTRODUCTION
Industrial Control Systems (ICS) [39] considered in thiswork are complex interconnected systems deployed to controland monitor among others critical infrastructures such aswater treatment and electric power systems [17] The increasein successful cyber attacks on public infrastructure [14] [29][48] and other mostly unsuccessful attempts [25] has raisedthe importance of deploying cyber defense mechanisms inICS Attackers are often bypassing the defense mechanisms(prevention and detection) by exploiting software and hard-ware vulnerabilities or through social engineering Thereforeit becomes important to look for ways of detecting processanomalies in an ICS caused by an attacker who has gainedunauthorized entry Water Defense [5] referred to in this paperas WD is one such anomaly detection mechanism
The technology underlying WD has been described andexperimentally evaluated by the authors [1] [4] This pa-per describes an independent supplementary assessment ofWD based on attacks launched by teams of attackers Theassessment was carried out over two consecutive years inan event labeled ldquohackfest (S3)rdquo [9] [35] [36] The eventwas organized by a team of faculty students and staff atthe Singapore University of Technology and Design Several
This work was supported by research grant 9013102373 from the Ministryof Defense and NRF2014-NCR-NCR001-040 from the National ResearchFoundation Singapore
attack and defense teams [9] [35] [36] participated in S3The attack teams designed and launched attacks on SWaTndashthe system used for assessment of WD by the authors Thispaper is focused on analysis of data used for assessing theperformance of WD in detecting process anomalies resultingfrom the attacks Details of S3 are in Section IV
The following two research questions were the focus of S3
with respect to WD
RQ1 How do attackers compromise the security of an ICS
RQ2 How effective is WD in detecting attacks launched byindependent attack teams
Contributions This paper (a) summarizes the performanceof the WD mechanism during two consecutive S3 eventsand (b) reports on observations and lessons learned fromthe experience all attacks launched during S3 are reportedInformation presented in this work will likely be valuable toresearchers attempting to assess the performance of processanomaly detection methods other than WD A complete listof the invariants (see Section III) used in WD and the S3
dataset are also available for download [41]
Organization The remainder of this paper is organized asfollows Section II contains material to aid in understanding theremainder of this paper An overview of the WD is presented inSection III Section IV contains a description of S3 Preparationfor the event is described in Section V Attacks performed byattacker teams during both S3 events are in Section VI Resultsfrom the events are presented in Section VII Section VIIIreturns to the research questions mentioned above and answersthem with respect to the data generated during S3 Section IXis a summary of the work Conclusions based on S3 aresummarized in Section X
II PRELIMINARIES AND BACKGROUND
This section is a brief introduction to Industrial ControlSystems (ICS) [39] in the context of SWaT An exampleillustrates the nature of a cyber attack on SWaT and itspotential impact on system response
A Industrial Control Systems
An ICS [39] consists of physical control and networkdevices (Figure 1) Physical devices include sensors and actua-
arX
iv1
809
0478
6v1
[cs
CR
] 1
3 Se
p 20
18
tors Control devices include Programmable Logic Controllers(PLCs) Supervisory Control and Data Acquisition (SCADA)workstations and Human-Machine-Interface (HMI) devicesNetwork devices include network switches and access pointsThe PLCs SCADA and HMI monitor and control the physicalprocess Communication channels in the network act as abridge between the physical process and the control devicesCommunication channels communicate the state of physicalprocess to controllers and control signals to the actuatorsPLCs receive data from sensors compute control actions andapply these actions to specific devices The PLCs in an ICScan be viewed collectively as a distributed control system thattransforms the state of the process through the use of sensorsand actuators
Fig 1 High level view of an ICS
B SWaT Architecture and components
SWaT [30] [40] is a testbed for water treatment It is usedto investigate the response to cyber attacks and experimentwith novel designs of defense mechanisms such as the onesdescribed in [3] The architecture and components in SWaTare described next
Stages in SWaT As shown in Figure 2 SWaT consists ofsix stages labeled Stage 1 through Stage 6 Each stage iscontrolled by its own set of PLCs Stage 1 controls the inflowof water to be treated by opening or closing a valve thatconnects the inlet pipe to the raw water tank (T101) Waterfrom the raw water tank is pumped via a chemical dosing(Stage 2) station to another Ultrafiltration (UF) feed watertank (T301) in Stage 3 In Stage 3 a UF feed pump sendswater via the UF unit to a Reverse Osmosis (RO) feed watertank (T401) in Stage 4 In Stage 4 an RO feed pump sendswater through an ultraviolet dechlorination unit controlled bya PLC Differential pressure sensors in Stage 3 measure thepressure drop across the UF unit A backwash cycle is initiatedwhen the pressure drop exceeds 04 bar indicating that themembranes need immediate cleaning Stage 5 contains a PLCto control the Reverse Osmosis (RO) unit that further filtersthe water using a 2-stage RO process The output of the ROunit enters storage tanks T601 and T602 in Stage 6 Tank T601contains the reject from RO and is used to clean the UF unitusing a backwash process Water in tank T602 contains thepermeate which is recycled into tank T101 in Stage 1
Sensors and actuators SWaT contains 42 sensors and actu-ators across the six stages These include sensors that relateto the dynamics of the process such as water level in tanksflow indicators and pressure indicators as well as thosefor measuring chemical properties of water including pHconductivity and hardness Each PLC has its own set of sensorsand actuators connected through a ring network Thus whena PLC needs to obtain state information from another PLCit must request such information via a suitable commandthe requested data is sent over level 1 network as shown inFigure 3
Communications Figure 3 shows the architecture of thecommunications infrastructure in SWaT Each PLC obtainsdata from sensors associated with the corresponding stageand controls pumps and valves in its domain Flow of wateracross various stages is controlled through opening and closingof valves and turning pumps ON or OFF Level sensors ineach tank enable the PLCs to decide when to turn a pumpON or OFF Several other sensors are available to check onthe physical and chemical properties of water flowing throughthe six stages PLCs communicate with each other through aseparate network Communications among sensors actuatorsand PLCs can be via either wired or wireless links controlledmanually Both wired and wireless networks connect PLCsto the physical process and to the HMI and engineeringworkstation ie the SCADA workstation
The control network is connected to the SCADA worksta-tion [40] through a wired network using a 16-port switch ThePLCs and SCADA workstation are also connected througha wireless network configured using star topology Switcheslocated at each stage enable the use of either wired or wirelesscommunications The communications network is layered intotwo levels For each PLC level 0 refers to the communicationlayer between sensors actuators and the PLC Level 0 net-work is implemented as a ldquodevice level ringrdquo [9] [43] whichincludes a Remote IO (RIO) device The RIO is connectedto the physical sensors and actuators Monitoring and controlinformation is exchanged between the PLC sensors andactuators across a Distributed Logical Router (DLR) Level 1refers to the communication layer among PLCs This layeris implemented in a star topology and includes a SCADAworkstation an HMI and a Historian
Controllers (PLCs) SWaT is equipped with Allen BradleyControlLogix PLCs Therefore some of the attacks describedin the remainder of this paper require consideration of theprotocols used by EtherNetIP [32] for Allen Bradley PLCsSCADA software is developed with tools from Rockwellautomation [8]
Attack detection mechanisms As shown in Figure 3 SWaTcontains four attack detection mechanisms D1 D2 D3 andOrthogonal Defense Mechanism (ODM) [6] D1 is based ona modified version of the open source Bro intrusion detectiontool [10] D2 is a set of three commercially available intrusiondetection systems that use a mix of process dynamics and
Raw Waterinput UltraltrationChemical
dosing Dechlorination
PLC1
Reverse Osmosis
PLC5PLC4PLC3PLC2
Backwash
PLC6
S=LIT101 FIT101
A=MV101 P101
S=AIT202
A=MV201
S=LIT301 FIT301 DPIT301
A=MV201 MV302 P301
S=LIT401 FIT401
A= P401
S=FIT501 FIT502 AIT501
A=MV501 P501
T101 T301Storage tanks T401[No storage]
FromT501From
T502
MV201P101MV101 FIT101
P6
Water fromexternalsource
DPIT301LIT301FIT301P301
MV302P302
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
Stage 6T601 T602
Fig 2 Six stages in SWaT with corresponding PLCs sensors and actuators Five water storage tanks as shown are labeled Txxx Water level in each tankis measured by the corresponding level indicator labeled as LITxxx AITxxx FITxxx and DPITxxx measure respectively chemical properties of water flowrate in a pipe and differential pressure across the ultrafiltration unit Pxxx denote pumps at various stages
Historian SCADASwitch
PLC 2
AS
PLC 6
AS
PLC 1
AS
HMI
Externaldevices
D1 D1 D1
D2
D3 D3D3
ODM ODMODM
Level 0
Level 1
Level 2
S SensorA ActuatorODM Orthogonal Defense Mechanism
Fig 3 Communications structure of SWaT D1 D2 and D3 denote setsof defense mechanisms in SWaT ODM (Orthogonal Defense Mechanism) isindependent of these mechanisms
other techniques for anomalous process behavior [12] [24][27] D3 (WD) sits inside PLCs and implements a distributedattack detection mechanism that relies exclusively on processdynamics [1] The ODM has direct access to sensors and anal-yses the data received for the existence of process anomaly
C An illustrative attack on SWaT
Consider Stage 1 of SWaT in Figure 2 This stage has amotorized valve labeled MV101 which when open causeswater to flow into tank T101 The inflow into T101 is measuredby flow meter FIT101 and the water level by a level sensorlabeled LIT101 Pump P101 sends water to the next stageFlow meter FIT201 measures the outflow of water fromStage 1 to Stage 3 PLC1 receives the LIT101 reading and
controls the motorized valve MV101 Similarly PLC1 receivesLIT301 readings from PLC3 and controls pump P101
Tanks T101 and T301 have four markers each labeled Low(L) Low Low (LL) High (H) and High High (HH) Eachmarker corresponds to a specific value of water level in thetank These markers are used by the corresponding PLCs tocontrol the states of motorized valves and pumps Thus forexample when the water level in T101 reaches L PLC1 opensMV101 and closes it when the level reaches H When waterlevel in T301 reaches L PLC1 turns P101 ON and turns it OFFwhen the level reaches H The following example illustrates theimpact of compromising level sensor LIT101 with the intentof damaging pump P101
Example Consider an attack where the attackerrsquos intentionis to underflow T101 and damage P101 by making it runwithout any incoming water The attack is launched on LIT101with Stage 1 in the following state LIT301 955mm MV101Closed P101 OFF UF is operational and therefore waterlevel in tank T301 is decreasing Assume now that the attackersets LIT101 reading to a constant value of 790mm In thisattack even though the water level in T101 is changing(decreasing) PLC1 receives a constant value After a whilewhen LIT301 reaches L pump P101 is turned ON by PLC1However the actual water level in tank T101 is lower thanL say at LL This leads to the outflow from the pump beingreduced to less than the intended flow rate Pump P101 runsdry when there is no water in T101 and will eventually getdamaged unless a corrective action is taken
Figure 4 shows the water level in tank T101 during theattack It can be observed that the outflow increases graduallywhen the attack is removed Note that the sudden drop in thevalue of LIT101 soon after attack removal corresponds to thefact that the PLC begins to receive the correct measurement
Fig 4 Water level in tank T101 when LIT101 is attacked LIT101readings are observed by PLC1
Fig 5 Level sensor LIT101 is under attack The attackerrsquos intention isto underflow T101 tank and damage P101 The first arrow indicatesthe outflow reducing time second arrow indicates the pump noisestarting time
of water level in T101 When the water level goes down to150 mm tank T101 does not have enough water to send totank T301 Figure 5 shows the change in flow rate duringthe attack as measured by flow meter FIT201 The two arrowsindicate the start of reduction of outflow from T101 At around10 seconds there is no water flowing from P101 even thoughthe pump is ON At this point the pump becomes noisy and theflow rate reduces to zero If not removed this attack may leadto pump damage due to overheating Of course a mechanicalcut off at the pump would avoid such damage
The above example shows how an attacker could potentiallydamage a pump by changing the sensor values and actuatorstates More complex attacks mentioned in Section VI can bedesigned and launched to reduce the chances of being detected
III OVERVIEW OF WDWD is a mechanism to detect process anomalies A process
is considered anomalous when it deviates from its expectedbehavior WD detects such anomalies through the use of in-variants An invariant [4] is a condition among physical andorchemical properties of the process that must hold wheneveran ICS is in a given state At a given time instant sensormeasurements of a suitable set of such properties constitutethe observable state of the physical process as known to theICS
The invariants serve as checkers of the system state Theseare coded and the code placed inside each PLC used for attack
detection The checker code is added to the control code thatalready exists in each PLC The PLC executes the code in acyclic manner In each cycle data from the sensors is obtainedcontrol actions computed and applied when necessary and theinvariants checked against the state variables or otherwise Dis-tributing the attack detection code among various controllersadds to the scalability of the proposed method During S3 theimplementation was located inside the Programmable LogicControllers (PLCs) as well as embedded in the communicationnetwork
Two types of invariants were considered state dependent(SD) and state agnostic (SA) While both types use statesto define relationships that must hold the SA invariants areindependent of any state based guard while SD invariants areAn SD invariant is true when the plant is in a given state anSA invariant is always true
A State-Dependent (SD) invariants
Consider for example the case when the motorized valveMV101 is Open In this case the flow rate indicator FIT101must provide a non-zero reading to the PLC This phys-ical fact leads to the following state-dependent invariantMV101=Open =rArr FIT101lt δ where δ denotes a thresholdindicating flow Note that an SD invariant may include con-ditions from across the various stages of SWaT thus enablingdistributed detection of attacks Derivation of SD invariants isbased on the design of the ICS and is described in [4]
B State-Agnostic (SA) invariants
Under normal system operation an SA invariant mustalways be true regardless of the system state One SA invariantwas derived for each tank in SWaT to detect attacks that affectthe flow of water into and out of a tank These invariants arebased on the flow of water and water level in a tank andhence are identical in terms of the mathematical relationshipthat they capture
As an example of an SA invariant consider the water levelin a tank At time instant k+1 the water level in T101 dependson the level at time k and the inflow and outflow at instant kThis relationship is captured in the following idealized discretetime model of the tank
x(k + 1) = x(k) + α(ui(k)minus uo(k)) (1)
where ui(k) and uo(k) denote the inflow and outflow ratesat time k and α is a proportionality constant that convertsflow rate to change in level using the tank dimensions x(k)is the true state of the water level Let y(k) denote the sensormeasurement of the water level x(k) an estimate of the levelsensor reading and ε a threshold based on experimentationBased on Eqn 1 the statistics obtained experimentally andconverting the true states to their estimates the followinginvariant is derived to test whether or not the tank fillingprocess is anomalous
sumn
i=1|(x(i)minus y(i))|n
gt ε under attack (2)
le ε normal (3)
Fig 6 Invariant to detect anomalous behavior of LIT 101
IV SWAT SECURITY SHOWDOWN (S3)
This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time
Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team
Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could
Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I
A S3-2016
Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks
B S3-2017
Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack
TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES
Profile Constraints
Cyber-criminal Limited number of attempts to realize a goal
Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed
Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed
Allowed to alter the network topology
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed
TABLE IITARGETS OF ATTACKS IN S3
Target Description
Physical Process Attacks
Valves Control the motorized valves
Pumps Disrupt pump control operations
Pressure Alter the pressure in pipes
Tank fill level Alter water level in a tank
Chemical dosing Alter chemical dosing
Sensor Data Attacks
Historian Alter data in the Historian
HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI
PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends
RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin
teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system
C Attack targets
The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian
V PREPARATION FOR S3
To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions
Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH
The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC
A Scope of WD
WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack
B Scope of WDH
WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks
VI S3 ATTACKS
The attacks launched by teams participating in the two S3
events are described next
A S3-2016 Attacks
All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks
DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA
Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD
DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1
As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant
B S3-2017 Attacks
All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V
TABLE IIIATTACKS LAUNCHED DURING S3-2016
SNO Target Method Attack Tool
1 Tank fill levelLIT101
Use HMI access Close MV101 and Stop P101 andP102
HMI
2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap
3 PLC Manual access Removed the cable at the ring atlevel 0
Manual
4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI
DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state
Ettercap
5 Tank fill levelLIT101
Use HMI access Attack on LIT101 ManualHMI
6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation
ManualSCADA
7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation
ManualSCADA
8 Historian DoS attack using CPPPO andloop
Attack between HMI and PLC CPPPO
9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed
ManualSCADA
10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA
11 Tank fill levelLIT301
Using SCADA access LIT301 set point altered ManualSCADA
12 Chemical dosingP201
Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing
Dosing pump attack on P201 ManualSCADA
13 HMISCADALIT101
Functional block introduce newconstant tag tie that to output tagcould only do zero
LIT101 set to zero from PLC Studio5000
14 Chemical dosingpump P205
Use SCADA access Manipulation of the chemical dosingpump (P205)
ManualSCADA
15 HMISCADA DoS on HMI using Level 1 net-work
Attack on HMI EttercapPycomm
16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)
Ettercap
17 Chemical dosingpump P203
HMI-based direct manipulation Attack on P203 while the four dosingpumps are running
ManualHMI
18 HMISCADALIT101
Re-program PLC to fix LIT101value to an arbitrary value
Attack on LIT101 Studio5000
416 are cyber criminal attacks in S3-2016
Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework
Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script
TABLE IVATTACKS LAUNCHED DURING S3-2017
SNo Target Method Attack Tool
1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI
2 Historian ARP and drop Change the value stored at the Historian Ettercap
3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000
4 Tank fill level LIT301 420to 320
Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm
Manual HMI
5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI
6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2
Manual SCADA
7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC
Manual
8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)
Manual
9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script
10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000
11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI
12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap
13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000
14 RIODisplay Disconnect specific IO PIN basedon manual
Disrupt the sensor reading send to PLC throughremote IO (RIO)
Manual
15 Chemical dosing pumpP403 AIT501
Based on captured traffic betweenHMI and PLC4
Change chemical dosing function VNC Python script PycommWireshark
16 PLC LT101 from 742mm to500mm
Level 0 MITM Change the commands and values that the PLCreceives and sends
Aircrack Airodump AireplayNetfilterqueue Scapy
17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap
Compromise historian data Ettercap Aircrack
18 Pressure sensor DPIT30130MV301-4
SMB to EW get project files runFT
Disrupt valves operation of Ultrafiltration andBackwash (P3)
SMB
19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc
20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite
Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy
21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000
22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy
23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy
24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig
25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin
Manual
26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued
Manual SCADA
27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI
28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap
29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA
Pycomm
30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing
Studio 5000
31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm
TABLE VCYBER CRIMINAL ATTACKS IN S3
Cyber Criminal Attacks Insider Attacks
S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18
S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30
1 3 4 5 6 7 8 1114 23 25 26 27 31
To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination
Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian
Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601
in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304
Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which
1httpswwwcvedetailscomcveCVE-2008-2160
controlled the ICS the attackers changed the simulation tagassociated with water level in T101
Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on
VII RESULTS
Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017
A S3-2016 results
We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly
Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected
It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime
Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
tors Control devices include Programmable Logic Controllers(PLCs) Supervisory Control and Data Acquisition (SCADA)workstations and Human-Machine-Interface (HMI) devicesNetwork devices include network switches and access pointsThe PLCs SCADA and HMI monitor and control the physicalprocess Communication channels in the network act as abridge between the physical process and the control devicesCommunication channels communicate the state of physicalprocess to controllers and control signals to the actuatorsPLCs receive data from sensors compute control actions andapply these actions to specific devices The PLCs in an ICScan be viewed collectively as a distributed control system thattransforms the state of the process through the use of sensorsand actuators
Fig 1 High level view of an ICS
B SWaT Architecture and components
SWaT [30] [40] is a testbed for water treatment It is usedto investigate the response to cyber attacks and experimentwith novel designs of defense mechanisms such as the onesdescribed in [3] The architecture and components in SWaTare described next
Stages in SWaT As shown in Figure 2 SWaT consists ofsix stages labeled Stage 1 through Stage 6 Each stage iscontrolled by its own set of PLCs Stage 1 controls the inflowof water to be treated by opening or closing a valve thatconnects the inlet pipe to the raw water tank (T101) Waterfrom the raw water tank is pumped via a chemical dosing(Stage 2) station to another Ultrafiltration (UF) feed watertank (T301) in Stage 3 In Stage 3 a UF feed pump sendswater via the UF unit to a Reverse Osmosis (RO) feed watertank (T401) in Stage 4 In Stage 4 an RO feed pump sendswater through an ultraviolet dechlorination unit controlled bya PLC Differential pressure sensors in Stage 3 measure thepressure drop across the UF unit A backwash cycle is initiatedwhen the pressure drop exceeds 04 bar indicating that themembranes need immediate cleaning Stage 5 contains a PLCto control the Reverse Osmosis (RO) unit that further filtersthe water using a 2-stage RO process The output of the ROunit enters storage tanks T601 and T602 in Stage 6 Tank T601contains the reject from RO and is used to clean the UF unitusing a backwash process Water in tank T602 contains thepermeate which is recycled into tank T101 in Stage 1
Sensors and actuators SWaT contains 42 sensors and actu-ators across the six stages These include sensors that relateto the dynamics of the process such as water level in tanksflow indicators and pressure indicators as well as thosefor measuring chemical properties of water including pHconductivity and hardness Each PLC has its own set of sensorsand actuators connected through a ring network Thus whena PLC needs to obtain state information from another PLCit must request such information via a suitable commandthe requested data is sent over level 1 network as shown inFigure 3
Communications Figure 3 shows the architecture of thecommunications infrastructure in SWaT Each PLC obtainsdata from sensors associated with the corresponding stageand controls pumps and valves in its domain Flow of wateracross various stages is controlled through opening and closingof valves and turning pumps ON or OFF Level sensors ineach tank enable the PLCs to decide when to turn a pumpON or OFF Several other sensors are available to check onthe physical and chemical properties of water flowing throughthe six stages PLCs communicate with each other through aseparate network Communications among sensors actuatorsand PLCs can be via either wired or wireless links controlledmanually Both wired and wireless networks connect PLCsto the physical process and to the HMI and engineeringworkstation ie the SCADA workstation
The control network is connected to the SCADA worksta-tion [40] through a wired network using a 16-port switch ThePLCs and SCADA workstation are also connected througha wireless network configured using star topology Switcheslocated at each stage enable the use of either wired or wirelesscommunications The communications network is layered intotwo levels For each PLC level 0 refers to the communicationlayer between sensors actuators and the PLC Level 0 net-work is implemented as a ldquodevice level ringrdquo [9] [43] whichincludes a Remote IO (RIO) device The RIO is connectedto the physical sensors and actuators Monitoring and controlinformation is exchanged between the PLC sensors andactuators across a Distributed Logical Router (DLR) Level 1refers to the communication layer among PLCs This layeris implemented in a star topology and includes a SCADAworkstation an HMI and a Historian
Controllers (PLCs) SWaT is equipped with Allen BradleyControlLogix PLCs Therefore some of the attacks describedin the remainder of this paper require consideration of theprotocols used by EtherNetIP [32] for Allen Bradley PLCsSCADA software is developed with tools from Rockwellautomation [8]
Attack detection mechanisms As shown in Figure 3 SWaTcontains four attack detection mechanisms D1 D2 D3 andOrthogonal Defense Mechanism (ODM) [6] D1 is based ona modified version of the open source Bro intrusion detectiontool [10] D2 is a set of three commercially available intrusiondetection systems that use a mix of process dynamics and
Raw Waterinput UltraltrationChemical
dosing Dechlorination
PLC1
Reverse Osmosis
PLC5PLC4PLC3PLC2
Backwash
PLC6
S=LIT101 FIT101
A=MV101 P101
S=AIT202
A=MV201
S=LIT301 FIT301 DPIT301
A=MV201 MV302 P301
S=LIT401 FIT401
A= P401
S=FIT501 FIT502 AIT501
A=MV501 P501
T101 T301Storage tanks T401[No storage]
FromT501From
T502
MV201P101MV101 FIT101
P6
Water fromexternalsource
DPIT301LIT301FIT301P301
MV302P302
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
Stage 6T601 T602
Fig 2 Six stages in SWaT with corresponding PLCs sensors and actuators Five water storage tanks as shown are labeled Txxx Water level in each tankis measured by the corresponding level indicator labeled as LITxxx AITxxx FITxxx and DPITxxx measure respectively chemical properties of water flowrate in a pipe and differential pressure across the ultrafiltration unit Pxxx denote pumps at various stages
Historian SCADASwitch
PLC 2
AS
PLC 6
AS
PLC 1
AS
HMI
Externaldevices
D1 D1 D1
D2
D3 D3D3
ODM ODMODM
Level 0
Level 1
Level 2
S SensorA ActuatorODM Orthogonal Defense Mechanism
Fig 3 Communications structure of SWaT D1 D2 and D3 denote setsof defense mechanisms in SWaT ODM (Orthogonal Defense Mechanism) isindependent of these mechanisms
other techniques for anomalous process behavior [12] [24][27] D3 (WD) sits inside PLCs and implements a distributedattack detection mechanism that relies exclusively on processdynamics [1] The ODM has direct access to sensors and anal-yses the data received for the existence of process anomaly
C An illustrative attack on SWaT
Consider Stage 1 of SWaT in Figure 2 This stage has amotorized valve labeled MV101 which when open causeswater to flow into tank T101 The inflow into T101 is measuredby flow meter FIT101 and the water level by a level sensorlabeled LIT101 Pump P101 sends water to the next stageFlow meter FIT201 measures the outflow of water fromStage 1 to Stage 3 PLC1 receives the LIT101 reading and
controls the motorized valve MV101 Similarly PLC1 receivesLIT301 readings from PLC3 and controls pump P101
Tanks T101 and T301 have four markers each labeled Low(L) Low Low (LL) High (H) and High High (HH) Eachmarker corresponds to a specific value of water level in thetank These markers are used by the corresponding PLCs tocontrol the states of motorized valves and pumps Thus forexample when the water level in T101 reaches L PLC1 opensMV101 and closes it when the level reaches H When waterlevel in T301 reaches L PLC1 turns P101 ON and turns it OFFwhen the level reaches H The following example illustrates theimpact of compromising level sensor LIT101 with the intentof damaging pump P101
Example Consider an attack where the attackerrsquos intentionis to underflow T101 and damage P101 by making it runwithout any incoming water The attack is launched on LIT101with Stage 1 in the following state LIT301 955mm MV101Closed P101 OFF UF is operational and therefore waterlevel in tank T301 is decreasing Assume now that the attackersets LIT101 reading to a constant value of 790mm In thisattack even though the water level in T101 is changing(decreasing) PLC1 receives a constant value After a whilewhen LIT301 reaches L pump P101 is turned ON by PLC1However the actual water level in tank T101 is lower thanL say at LL This leads to the outflow from the pump beingreduced to less than the intended flow rate Pump P101 runsdry when there is no water in T101 and will eventually getdamaged unless a corrective action is taken
Figure 4 shows the water level in tank T101 during theattack It can be observed that the outflow increases graduallywhen the attack is removed Note that the sudden drop in thevalue of LIT101 soon after attack removal corresponds to thefact that the PLC begins to receive the correct measurement
Fig 4 Water level in tank T101 when LIT101 is attacked LIT101readings are observed by PLC1
Fig 5 Level sensor LIT101 is under attack The attackerrsquos intention isto underflow T101 tank and damage P101 The first arrow indicatesthe outflow reducing time second arrow indicates the pump noisestarting time
of water level in T101 When the water level goes down to150 mm tank T101 does not have enough water to send totank T301 Figure 5 shows the change in flow rate duringthe attack as measured by flow meter FIT201 The two arrowsindicate the start of reduction of outflow from T101 At around10 seconds there is no water flowing from P101 even thoughthe pump is ON At this point the pump becomes noisy and theflow rate reduces to zero If not removed this attack may leadto pump damage due to overheating Of course a mechanicalcut off at the pump would avoid such damage
The above example shows how an attacker could potentiallydamage a pump by changing the sensor values and actuatorstates More complex attacks mentioned in Section VI can bedesigned and launched to reduce the chances of being detected
III OVERVIEW OF WDWD is a mechanism to detect process anomalies A process
is considered anomalous when it deviates from its expectedbehavior WD detects such anomalies through the use of in-variants An invariant [4] is a condition among physical andorchemical properties of the process that must hold wheneveran ICS is in a given state At a given time instant sensormeasurements of a suitable set of such properties constitutethe observable state of the physical process as known to theICS
The invariants serve as checkers of the system state Theseare coded and the code placed inside each PLC used for attack
detection The checker code is added to the control code thatalready exists in each PLC The PLC executes the code in acyclic manner In each cycle data from the sensors is obtainedcontrol actions computed and applied when necessary and theinvariants checked against the state variables or otherwise Dis-tributing the attack detection code among various controllersadds to the scalability of the proposed method During S3 theimplementation was located inside the Programmable LogicControllers (PLCs) as well as embedded in the communicationnetwork
Two types of invariants were considered state dependent(SD) and state agnostic (SA) While both types use statesto define relationships that must hold the SA invariants areindependent of any state based guard while SD invariants areAn SD invariant is true when the plant is in a given state anSA invariant is always true
A State-Dependent (SD) invariants
Consider for example the case when the motorized valveMV101 is Open In this case the flow rate indicator FIT101must provide a non-zero reading to the PLC This phys-ical fact leads to the following state-dependent invariantMV101=Open =rArr FIT101lt δ where δ denotes a thresholdindicating flow Note that an SD invariant may include con-ditions from across the various stages of SWaT thus enablingdistributed detection of attacks Derivation of SD invariants isbased on the design of the ICS and is described in [4]
B State-Agnostic (SA) invariants
Under normal system operation an SA invariant mustalways be true regardless of the system state One SA invariantwas derived for each tank in SWaT to detect attacks that affectthe flow of water into and out of a tank These invariants arebased on the flow of water and water level in a tank andhence are identical in terms of the mathematical relationshipthat they capture
As an example of an SA invariant consider the water levelin a tank At time instant k+1 the water level in T101 dependson the level at time k and the inflow and outflow at instant kThis relationship is captured in the following idealized discretetime model of the tank
x(k + 1) = x(k) + α(ui(k)minus uo(k)) (1)
where ui(k) and uo(k) denote the inflow and outflow ratesat time k and α is a proportionality constant that convertsflow rate to change in level using the tank dimensions x(k)is the true state of the water level Let y(k) denote the sensormeasurement of the water level x(k) an estimate of the levelsensor reading and ε a threshold based on experimentationBased on Eqn 1 the statistics obtained experimentally andconverting the true states to their estimates the followinginvariant is derived to test whether or not the tank fillingprocess is anomalous
sumn
i=1|(x(i)minus y(i))|n
gt ε under attack (2)
le ε normal (3)
Fig 6 Invariant to detect anomalous behavior of LIT 101
IV SWAT SECURITY SHOWDOWN (S3)
This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time
Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team
Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could
Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I
A S3-2016
Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks
B S3-2017
Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack
TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES
Profile Constraints
Cyber-criminal Limited number of attempts to realize a goal
Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed
Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed
Allowed to alter the network topology
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed
TABLE IITARGETS OF ATTACKS IN S3
Target Description
Physical Process Attacks
Valves Control the motorized valves
Pumps Disrupt pump control operations
Pressure Alter the pressure in pipes
Tank fill level Alter water level in a tank
Chemical dosing Alter chemical dosing
Sensor Data Attacks
Historian Alter data in the Historian
HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI
PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends
RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin
teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system
C Attack targets
The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian
V PREPARATION FOR S3
To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions
Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH
The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC
A Scope of WD
WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack
B Scope of WDH
WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks
VI S3 ATTACKS
The attacks launched by teams participating in the two S3
events are described next
A S3-2016 Attacks
All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks
DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA
Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD
DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1
As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant
B S3-2017 Attacks
All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V
TABLE IIIATTACKS LAUNCHED DURING S3-2016
SNO Target Method Attack Tool
1 Tank fill levelLIT101
Use HMI access Close MV101 and Stop P101 andP102
HMI
2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap
3 PLC Manual access Removed the cable at the ring atlevel 0
Manual
4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI
DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state
Ettercap
5 Tank fill levelLIT101
Use HMI access Attack on LIT101 ManualHMI
6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation
ManualSCADA
7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation
ManualSCADA
8 Historian DoS attack using CPPPO andloop
Attack between HMI and PLC CPPPO
9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed
ManualSCADA
10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA
11 Tank fill levelLIT301
Using SCADA access LIT301 set point altered ManualSCADA
12 Chemical dosingP201
Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing
Dosing pump attack on P201 ManualSCADA
13 HMISCADALIT101
Functional block introduce newconstant tag tie that to output tagcould only do zero
LIT101 set to zero from PLC Studio5000
14 Chemical dosingpump P205
Use SCADA access Manipulation of the chemical dosingpump (P205)
ManualSCADA
15 HMISCADA DoS on HMI using Level 1 net-work
Attack on HMI EttercapPycomm
16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)
Ettercap
17 Chemical dosingpump P203
HMI-based direct manipulation Attack on P203 while the four dosingpumps are running
ManualHMI
18 HMISCADALIT101
Re-program PLC to fix LIT101value to an arbitrary value
Attack on LIT101 Studio5000
416 are cyber criminal attacks in S3-2016
Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework
Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script
TABLE IVATTACKS LAUNCHED DURING S3-2017
SNo Target Method Attack Tool
1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI
2 Historian ARP and drop Change the value stored at the Historian Ettercap
3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000
4 Tank fill level LIT301 420to 320
Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm
Manual HMI
5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI
6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2
Manual SCADA
7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC
Manual
8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)
Manual
9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script
10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000
11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI
12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap
13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000
14 RIODisplay Disconnect specific IO PIN basedon manual
Disrupt the sensor reading send to PLC throughremote IO (RIO)
Manual
15 Chemical dosing pumpP403 AIT501
Based on captured traffic betweenHMI and PLC4
Change chemical dosing function VNC Python script PycommWireshark
16 PLC LT101 from 742mm to500mm
Level 0 MITM Change the commands and values that the PLCreceives and sends
Aircrack Airodump AireplayNetfilterqueue Scapy
17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap
Compromise historian data Ettercap Aircrack
18 Pressure sensor DPIT30130MV301-4
SMB to EW get project files runFT
Disrupt valves operation of Ultrafiltration andBackwash (P3)
SMB
19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc
20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite
Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy
21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000
22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy
23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy
24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig
25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin
Manual
26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued
Manual SCADA
27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI
28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap
29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA
Pycomm
30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing
Studio 5000
31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm
TABLE VCYBER CRIMINAL ATTACKS IN S3
Cyber Criminal Attacks Insider Attacks
S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18
S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30
1 3 4 5 6 7 8 1114 23 25 26 27 31
To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination
Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian
Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601
in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304
Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which
1httpswwwcvedetailscomcveCVE-2008-2160
controlled the ICS the attackers changed the simulation tagassociated with water level in T101
Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on
VII RESULTS
Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017
A S3-2016 results
We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly
Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected
It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime
Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
Raw Waterinput UltraltrationChemical
dosing Dechlorination
PLC1
Reverse Osmosis
PLC5PLC4PLC3PLC2
Backwash
PLC6
S=LIT101 FIT101
A=MV101 P101
S=AIT202
A=MV201
S=LIT301 FIT301 DPIT301
A=MV201 MV302 P301
S=LIT401 FIT401
A= P401
S=FIT501 FIT502 AIT501
A=MV501 P501
T101 T301Storage tanks T401[No storage]
FromT501From
T502
MV201P101MV101 FIT101
P6
Water fromexternalsource
DPIT301LIT301FIT301P301
MV302P302
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
Stage 6T601 T602
Fig 2 Six stages in SWaT with corresponding PLCs sensors and actuators Five water storage tanks as shown are labeled Txxx Water level in each tankis measured by the corresponding level indicator labeled as LITxxx AITxxx FITxxx and DPITxxx measure respectively chemical properties of water flowrate in a pipe and differential pressure across the ultrafiltration unit Pxxx denote pumps at various stages
Historian SCADASwitch
PLC 2
AS
PLC 6
AS
PLC 1
AS
HMI
Externaldevices
D1 D1 D1
D2
D3 D3D3
ODM ODMODM
Level 0
Level 1
Level 2
S SensorA ActuatorODM Orthogonal Defense Mechanism
Fig 3 Communications structure of SWaT D1 D2 and D3 denote setsof defense mechanisms in SWaT ODM (Orthogonal Defense Mechanism) isindependent of these mechanisms
other techniques for anomalous process behavior [12] [24][27] D3 (WD) sits inside PLCs and implements a distributedattack detection mechanism that relies exclusively on processdynamics [1] The ODM has direct access to sensors and anal-yses the data received for the existence of process anomaly
C An illustrative attack on SWaT
Consider Stage 1 of SWaT in Figure 2 This stage has amotorized valve labeled MV101 which when open causeswater to flow into tank T101 The inflow into T101 is measuredby flow meter FIT101 and the water level by a level sensorlabeled LIT101 Pump P101 sends water to the next stageFlow meter FIT201 measures the outflow of water fromStage 1 to Stage 3 PLC1 receives the LIT101 reading and
controls the motorized valve MV101 Similarly PLC1 receivesLIT301 readings from PLC3 and controls pump P101
Tanks T101 and T301 have four markers each labeled Low(L) Low Low (LL) High (H) and High High (HH) Eachmarker corresponds to a specific value of water level in thetank These markers are used by the corresponding PLCs tocontrol the states of motorized valves and pumps Thus forexample when the water level in T101 reaches L PLC1 opensMV101 and closes it when the level reaches H When waterlevel in T301 reaches L PLC1 turns P101 ON and turns it OFFwhen the level reaches H The following example illustrates theimpact of compromising level sensor LIT101 with the intentof damaging pump P101
Example Consider an attack where the attackerrsquos intentionis to underflow T101 and damage P101 by making it runwithout any incoming water The attack is launched on LIT101with Stage 1 in the following state LIT301 955mm MV101Closed P101 OFF UF is operational and therefore waterlevel in tank T301 is decreasing Assume now that the attackersets LIT101 reading to a constant value of 790mm In thisattack even though the water level in T101 is changing(decreasing) PLC1 receives a constant value After a whilewhen LIT301 reaches L pump P101 is turned ON by PLC1However the actual water level in tank T101 is lower thanL say at LL This leads to the outflow from the pump beingreduced to less than the intended flow rate Pump P101 runsdry when there is no water in T101 and will eventually getdamaged unless a corrective action is taken
Figure 4 shows the water level in tank T101 during theattack It can be observed that the outflow increases graduallywhen the attack is removed Note that the sudden drop in thevalue of LIT101 soon after attack removal corresponds to thefact that the PLC begins to receive the correct measurement
Fig 4 Water level in tank T101 when LIT101 is attacked LIT101readings are observed by PLC1
Fig 5 Level sensor LIT101 is under attack The attackerrsquos intention isto underflow T101 tank and damage P101 The first arrow indicatesthe outflow reducing time second arrow indicates the pump noisestarting time
of water level in T101 When the water level goes down to150 mm tank T101 does not have enough water to send totank T301 Figure 5 shows the change in flow rate duringthe attack as measured by flow meter FIT201 The two arrowsindicate the start of reduction of outflow from T101 At around10 seconds there is no water flowing from P101 even thoughthe pump is ON At this point the pump becomes noisy and theflow rate reduces to zero If not removed this attack may leadto pump damage due to overheating Of course a mechanicalcut off at the pump would avoid such damage
The above example shows how an attacker could potentiallydamage a pump by changing the sensor values and actuatorstates More complex attacks mentioned in Section VI can bedesigned and launched to reduce the chances of being detected
III OVERVIEW OF WDWD is a mechanism to detect process anomalies A process
is considered anomalous when it deviates from its expectedbehavior WD detects such anomalies through the use of in-variants An invariant [4] is a condition among physical andorchemical properties of the process that must hold wheneveran ICS is in a given state At a given time instant sensormeasurements of a suitable set of such properties constitutethe observable state of the physical process as known to theICS
The invariants serve as checkers of the system state Theseare coded and the code placed inside each PLC used for attack
detection The checker code is added to the control code thatalready exists in each PLC The PLC executes the code in acyclic manner In each cycle data from the sensors is obtainedcontrol actions computed and applied when necessary and theinvariants checked against the state variables or otherwise Dis-tributing the attack detection code among various controllersadds to the scalability of the proposed method During S3 theimplementation was located inside the Programmable LogicControllers (PLCs) as well as embedded in the communicationnetwork
Two types of invariants were considered state dependent(SD) and state agnostic (SA) While both types use statesto define relationships that must hold the SA invariants areindependent of any state based guard while SD invariants areAn SD invariant is true when the plant is in a given state anSA invariant is always true
A State-Dependent (SD) invariants
Consider for example the case when the motorized valveMV101 is Open In this case the flow rate indicator FIT101must provide a non-zero reading to the PLC This phys-ical fact leads to the following state-dependent invariantMV101=Open =rArr FIT101lt δ where δ denotes a thresholdindicating flow Note that an SD invariant may include con-ditions from across the various stages of SWaT thus enablingdistributed detection of attacks Derivation of SD invariants isbased on the design of the ICS and is described in [4]
B State-Agnostic (SA) invariants
Under normal system operation an SA invariant mustalways be true regardless of the system state One SA invariantwas derived for each tank in SWaT to detect attacks that affectthe flow of water into and out of a tank These invariants arebased on the flow of water and water level in a tank andhence are identical in terms of the mathematical relationshipthat they capture
As an example of an SA invariant consider the water levelin a tank At time instant k+1 the water level in T101 dependson the level at time k and the inflow and outflow at instant kThis relationship is captured in the following idealized discretetime model of the tank
x(k + 1) = x(k) + α(ui(k)minus uo(k)) (1)
where ui(k) and uo(k) denote the inflow and outflow ratesat time k and α is a proportionality constant that convertsflow rate to change in level using the tank dimensions x(k)is the true state of the water level Let y(k) denote the sensormeasurement of the water level x(k) an estimate of the levelsensor reading and ε a threshold based on experimentationBased on Eqn 1 the statistics obtained experimentally andconverting the true states to their estimates the followinginvariant is derived to test whether or not the tank fillingprocess is anomalous
sumn
i=1|(x(i)minus y(i))|n
gt ε under attack (2)
le ε normal (3)
Fig 6 Invariant to detect anomalous behavior of LIT 101
IV SWAT SECURITY SHOWDOWN (S3)
This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time
Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team
Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could
Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I
A S3-2016
Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks
B S3-2017
Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack
TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES
Profile Constraints
Cyber-criminal Limited number of attempts to realize a goal
Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed
Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed
Allowed to alter the network topology
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed
TABLE IITARGETS OF ATTACKS IN S3
Target Description
Physical Process Attacks
Valves Control the motorized valves
Pumps Disrupt pump control operations
Pressure Alter the pressure in pipes
Tank fill level Alter water level in a tank
Chemical dosing Alter chemical dosing
Sensor Data Attacks
Historian Alter data in the Historian
HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI
PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends
RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin
teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system
C Attack targets
The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian
V PREPARATION FOR S3
To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions
Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH
The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC
A Scope of WD
WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack
B Scope of WDH
WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks
VI S3 ATTACKS
The attacks launched by teams participating in the two S3
events are described next
A S3-2016 Attacks
All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks
DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA
Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD
DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1
As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant
B S3-2017 Attacks
All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V
TABLE IIIATTACKS LAUNCHED DURING S3-2016
SNO Target Method Attack Tool
1 Tank fill levelLIT101
Use HMI access Close MV101 and Stop P101 andP102
HMI
2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap
3 PLC Manual access Removed the cable at the ring atlevel 0
Manual
4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI
DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state
Ettercap
5 Tank fill levelLIT101
Use HMI access Attack on LIT101 ManualHMI
6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation
ManualSCADA
7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation
ManualSCADA
8 Historian DoS attack using CPPPO andloop
Attack between HMI and PLC CPPPO
9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed
ManualSCADA
10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA
11 Tank fill levelLIT301
Using SCADA access LIT301 set point altered ManualSCADA
12 Chemical dosingP201
Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing
Dosing pump attack on P201 ManualSCADA
13 HMISCADALIT101
Functional block introduce newconstant tag tie that to output tagcould only do zero
LIT101 set to zero from PLC Studio5000
14 Chemical dosingpump P205
Use SCADA access Manipulation of the chemical dosingpump (P205)
ManualSCADA
15 HMISCADA DoS on HMI using Level 1 net-work
Attack on HMI EttercapPycomm
16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)
Ettercap
17 Chemical dosingpump P203
HMI-based direct manipulation Attack on P203 while the four dosingpumps are running
ManualHMI
18 HMISCADALIT101
Re-program PLC to fix LIT101value to an arbitrary value
Attack on LIT101 Studio5000
416 are cyber criminal attacks in S3-2016
Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework
Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script
TABLE IVATTACKS LAUNCHED DURING S3-2017
SNo Target Method Attack Tool
1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI
2 Historian ARP and drop Change the value stored at the Historian Ettercap
3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000
4 Tank fill level LIT301 420to 320
Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm
Manual HMI
5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI
6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2
Manual SCADA
7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC
Manual
8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)
Manual
9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script
10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000
11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI
12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap
13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000
14 RIODisplay Disconnect specific IO PIN basedon manual
Disrupt the sensor reading send to PLC throughremote IO (RIO)
Manual
15 Chemical dosing pumpP403 AIT501
Based on captured traffic betweenHMI and PLC4
Change chemical dosing function VNC Python script PycommWireshark
16 PLC LT101 from 742mm to500mm
Level 0 MITM Change the commands and values that the PLCreceives and sends
Aircrack Airodump AireplayNetfilterqueue Scapy
17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap
Compromise historian data Ettercap Aircrack
18 Pressure sensor DPIT30130MV301-4
SMB to EW get project files runFT
Disrupt valves operation of Ultrafiltration andBackwash (P3)
SMB
19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc
20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite
Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy
21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000
22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy
23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy
24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig
25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin
Manual
26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued
Manual SCADA
27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI
28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap
29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA
Pycomm
30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing
Studio 5000
31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm
TABLE VCYBER CRIMINAL ATTACKS IN S3
Cyber Criminal Attacks Insider Attacks
S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18
S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30
1 3 4 5 6 7 8 1114 23 25 26 27 31
To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination
Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian
Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601
in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304
Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which
1httpswwwcvedetailscomcveCVE-2008-2160
controlled the ICS the attackers changed the simulation tagassociated with water level in T101
Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on
VII RESULTS
Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017
A S3-2016 results
We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly
Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected
It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime
Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
Fig 4 Water level in tank T101 when LIT101 is attacked LIT101readings are observed by PLC1
Fig 5 Level sensor LIT101 is under attack The attackerrsquos intention isto underflow T101 tank and damage P101 The first arrow indicatesthe outflow reducing time second arrow indicates the pump noisestarting time
of water level in T101 When the water level goes down to150 mm tank T101 does not have enough water to send totank T301 Figure 5 shows the change in flow rate duringthe attack as measured by flow meter FIT201 The two arrowsindicate the start of reduction of outflow from T101 At around10 seconds there is no water flowing from P101 even thoughthe pump is ON At this point the pump becomes noisy and theflow rate reduces to zero If not removed this attack may leadto pump damage due to overheating Of course a mechanicalcut off at the pump would avoid such damage
The above example shows how an attacker could potentiallydamage a pump by changing the sensor values and actuatorstates More complex attacks mentioned in Section VI can bedesigned and launched to reduce the chances of being detected
III OVERVIEW OF WDWD is a mechanism to detect process anomalies A process
is considered anomalous when it deviates from its expectedbehavior WD detects such anomalies through the use of in-variants An invariant [4] is a condition among physical andorchemical properties of the process that must hold wheneveran ICS is in a given state At a given time instant sensormeasurements of a suitable set of such properties constitutethe observable state of the physical process as known to theICS
The invariants serve as checkers of the system state Theseare coded and the code placed inside each PLC used for attack
detection The checker code is added to the control code thatalready exists in each PLC The PLC executes the code in acyclic manner In each cycle data from the sensors is obtainedcontrol actions computed and applied when necessary and theinvariants checked against the state variables or otherwise Dis-tributing the attack detection code among various controllersadds to the scalability of the proposed method During S3 theimplementation was located inside the Programmable LogicControllers (PLCs) as well as embedded in the communicationnetwork
Two types of invariants were considered state dependent(SD) and state agnostic (SA) While both types use statesto define relationships that must hold the SA invariants areindependent of any state based guard while SD invariants areAn SD invariant is true when the plant is in a given state anSA invariant is always true
A State-Dependent (SD) invariants
Consider for example the case when the motorized valveMV101 is Open In this case the flow rate indicator FIT101must provide a non-zero reading to the PLC This phys-ical fact leads to the following state-dependent invariantMV101=Open =rArr FIT101lt δ where δ denotes a thresholdindicating flow Note that an SD invariant may include con-ditions from across the various stages of SWaT thus enablingdistributed detection of attacks Derivation of SD invariants isbased on the design of the ICS and is described in [4]
B State-Agnostic (SA) invariants
Under normal system operation an SA invariant mustalways be true regardless of the system state One SA invariantwas derived for each tank in SWaT to detect attacks that affectthe flow of water into and out of a tank These invariants arebased on the flow of water and water level in a tank andhence are identical in terms of the mathematical relationshipthat they capture
As an example of an SA invariant consider the water levelin a tank At time instant k+1 the water level in T101 dependson the level at time k and the inflow and outflow at instant kThis relationship is captured in the following idealized discretetime model of the tank
x(k + 1) = x(k) + α(ui(k)minus uo(k)) (1)
where ui(k) and uo(k) denote the inflow and outflow ratesat time k and α is a proportionality constant that convertsflow rate to change in level using the tank dimensions x(k)is the true state of the water level Let y(k) denote the sensormeasurement of the water level x(k) an estimate of the levelsensor reading and ε a threshold based on experimentationBased on Eqn 1 the statistics obtained experimentally andconverting the true states to their estimates the followinginvariant is derived to test whether or not the tank fillingprocess is anomalous
sumn
i=1|(x(i)minus y(i))|n
gt ε under attack (2)
le ε normal (3)
Fig 6 Invariant to detect anomalous behavior of LIT 101
IV SWAT SECURITY SHOWDOWN (S3)
This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time
Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team
Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could
Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I
A S3-2016
Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks
B S3-2017
Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack
TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES
Profile Constraints
Cyber-criminal Limited number of attempts to realize a goal
Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed
Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed
Allowed to alter the network topology
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed
TABLE IITARGETS OF ATTACKS IN S3
Target Description
Physical Process Attacks
Valves Control the motorized valves
Pumps Disrupt pump control operations
Pressure Alter the pressure in pipes
Tank fill level Alter water level in a tank
Chemical dosing Alter chemical dosing
Sensor Data Attacks
Historian Alter data in the Historian
HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI
PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends
RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin
teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system
C Attack targets
The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian
V PREPARATION FOR S3
To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions
Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH
The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC
A Scope of WD
WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack
B Scope of WDH
WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks
VI S3 ATTACKS
The attacks launched by teams participating in the two S3
events are described next
A S3-2016 Attacks
All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks
DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA
Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD
DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1
As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant
B S3-2017 Attacks
All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V
TABLE IIIATTACKS LAUNCHED DURING S3-2016
SNO Target Method Attack Tool
1 Tank fill levelLIT101
Use HMI access Close MV101 and Stop P101 andP102
HMI
2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap
3 PLC Manual access Removed the cable at the ring atlevel 0
Manual
4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI
DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state
Ettercap
5 Tank fill levelLIT101
Use HMI access Attack on LIT101 ManualHMI
6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation
ManualSCADA
7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation
ManualSCADA
8 Historian DoS attack using CPPPO andloop
Attack between HMI and PLC CPPPO
9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed
ManualSCADA
10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA
11 Tank fill levelLIT301
Using SCADA access LIT301 set point altered ManualSCADA
12 Chemical dosingP201
Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing
Dosing pump attack on P201 ManualSCADA
13 HMISCADALIT101
Functional block introduce newconstant tag tie that to output tagcould only do zero
LIT101 set to zero from PLC Studio5000
14 Chemical dosingpump P205
Use SCADA access Manipulation of the chemical dosingpump (P205)
ManualSCADA
15 HMISCADA DoS on HMI using Level 1 net-work
Attack on HMI EttercapPycomm
16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)
Ettercap
17 Chemical dosingpump P203
HMI-based direct manipulation Attack on P203 while the four dosingpumps are running
ManualHMI
18 HMISCADALIT101
Re-program PLC to fix LIT101value to an arbitrary value
Attack on LIT101 Studio5000
416 are cyber criminal attacks in S3-2016
Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework
Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script
TABLE IVATTACKS LAUNCHED DURING S3-2017
SNo Target Method Attack Tool
1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI
2 Historian ARP and drop Change the value stored at the Historian Ettercap
3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000
4 Tank fill level LIT301 420to 320
Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm
Manual HMI
5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI
6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2
Manual SCADA
7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC
Manual
8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)
Manual
9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script
10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000
11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI
12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap
13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000
14 RIODisplay Disconnect specific IO PIN basedon manual
Disrupt the sensor reading send to PLC throughremote IO (RIO)
Manual
15 Chemical dosing pumpP403 AIT501
Based on captured traffic betweenHMI and PLC4
Change chemical dosing function VNC Python script PycommWireshark
16 PLC LT101 from 742mm to500mm
Level 0 MITM Change the commands and values that the PLCreceives and sends
Aircrack Airodump AireplayNetfilterqueue Scapy
17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap
Compromise historian data Ettercap Aircrack
18 Pressure sensor DPIT30130MV301-4
SMB to EW get project files runFT
Disrupt valves operation of Ultrafiltration andBackwash (P3)
SMB
19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc
20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite
Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy
21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000
22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy
23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy
24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig
25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin
Manual
26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued
Manual SCADA
27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI
28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap
29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA
Pycomm
30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing
Studio 5000
31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm
TABLE VCYBER CRIMINAL ATTACKS IN S3
Cyber Criminal Attacks Insider Attacks
S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18
S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30
1 3 4 5 6 7 8 1114 23 25 26 27 31
To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination
Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian
Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601
in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304
Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which
1httpswwwcvedetailscomcveCVE-2008-2160
controlled the ICS the attackers changed the simulation tagassociated with water level in T101
Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on
VII RESULTS
Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017
A S3-2016 results
We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly
Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected
It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime
Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
sumn
i=1|(x(i)minus y(i))|n
gt ε under attack (2)
le ε normal (3)
Fig 6 Invariant to detect anomalous behavior of LIT 101
IV SWAT SECURITY SHOWDOWN (S3)
This section presents details of the two S3 [36] eventsincluding guidelines and selected information on participantsIn S3 the attackers are challenged to realise concrete goals inSWaT Points earned by an attack team are weighted based onthe capabilities needed to launch the attack and the number ofdefence mechanisms successfully bypassed during the attackThe goal was to meet as many pre-defined challenges aspossible within the pre-allocated time
Information disclosed to the attack teams Technical detailson SWaT such as network architecture protocols and devicesused are released to the attackers one month prior to theirarrival for participation in the event Publicly available whitepapers on mechanisms deployed by the defence teams areshared with each attack team
Information disclosed to the defenders S3 organizers workedclosely with the defense teams to integrate their defence mech-anisms into SWaT Information about the normal operation ofSWaT was disclosed to the defenders to enable them to fine-tune their detection systems and reduce false alarms as muchas they could
Attacker profiles Attack teams were asked to select from aset of attacker profiles [34] The following attacker profileswere available cyber-criminal insider or a combination ofboth An attacker profile is intended to restrict availability ofresources and limit the access rights of the attackers as shownin Table I
A S3-2016
Attack teams included three from industry and three fromacademia Similarly there were three defense teams from theindustry and three from academia During the live phase heldat the SWaT testbed all six [35] defence mechanisms weresimultaneously in place Each team was given 12 hours forpassive reconnaissance and team was assigned a 3-hour slotduring which they were able to launch attacks
B S3-2017
Attack teams included one from industry and four fromacademia There were two defense teams from the industryand two from academia Each attack team was given two ses-sions [36] of four hours each to conduct reconnaissance on thetestbeds During these sessions various attacks were preparedand tested with the assistance of the SWaT laboratory engineerDuring the actual event each team was given two hours todemonstrate their attacks that were prepared previously Attack
TABLE IRESOURCES AND ACCESS RIGHTS FOR ATTACKER PROFILES
Profile Constraints
Cyber-criminal Limited number of attempts to realize a goal
Physical access not allowed manual manipulation ofthe sensors and actuators are not allowed
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 not allowed
Insider Physical access to SWaT allowed manual manipula-tion of the sensors and actuators are allowed
Allowed to alter the network topology
Direct connection to PLCs using any software suchas Allen Bradleyrsquos Studio5000 allowed
TABLE IITARGETS OF ATTACKS IN S3
Target Description
Physical Process Attacks
Valves Control the motorized valves
Pumps Disrupt pump control operations
Pressure Alter the pressure in pipes
Tank fill level Alter water level in a tank
Chemical dosing Alter chemical dosing
Sensor Data Attacks
Historian Alter data in the Historian
HMISCADA Alter the sensor actuator values at HMI orSCADA DoS Attacks on SCADA HMI
PLC Reprogram PLC DoS attacks on PLCsChange the commands and values in which thePLC receives and sends
RIODisplay Control of the RIO through disconnected ana-logue InputOutput pin
teams were also given a separate network for Internet up-linkand up to three Virtual Machines (VMs) running either Linuxor Windows operating system
C Attack targets
The attack teams were given a list of components andsubsystems in SWaT that could serve as the target of theirattacks Table II lists the targets available to the attack teamsTable II has two kinds of attacks physical process attacks andsensor data attacks In physical process attacks an attackerrsquosobjective is to alter the physical process In the case of sensordata attacks an attackerrsquos objective is to alter the sensor oractuator tags during communication or in the Historian
V PREPARATION FOR S3
To prepare for S3-2016 an earlier version of WD wasextended to all six stages of SWaT This extension requiredthe generation of invariants across all stages coding of theinvariants and placement of the code inside [1] the six PLCsThe modified WD was tested on SWaT by running the plantunder various operating conditions
Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH
The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC
A Scope of WD
WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack
B Scope of WDH
WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks
VI S3 ATTACKS
The attacks launched by teams participating in the two S3
events are described next
A S3-2016 Attacks
All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks
DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA
Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD
DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1
As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant
B S3-2017 Attacks
All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V
TABLE IIIATTACKS LAUNCHED DURING S3-2016
SNO Target Method Attack Tool
1 Tank fill levelLIT101
Use HMI access Close MV101 and Stop P101 andP102
HMI
2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap
3 PLC Manual access Removed the cable at the ring atlevel 0
Manual
4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI
DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state
Ettercap
5 Tank fill levelLIT101
Use HMI access Attack on LIT101 ManualHMI
6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation
ManualSCADA
7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation
ManualSCADA
8 Historian DoS attack using CPPPO andloop
Attack between HMI and PLC CPPPO
9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed
ManualSCADA
10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA
11 Tank fill levelLIT301
Using SCADA access LIT301 set point altered ManualSCADA
12 Chemical dosingP201
Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing
Dosing pump attack on P201 ManualSCADA
13 HMISCADALIT101
Functional block introduce newconstant tag tie that to output tagcould only do zero
LIT101 set to zero from PLC Studio5000
14 Chemical dosingpump P205
Use SCADA access Manipulation of the chemical dosingpump (P205)
ManualSCADA
15 HMISCADA DoS on HMI using Level 1 net-work
Attack on HMI EttercapPycomm
16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)
Ettercap
17 Chemical dosingpump P203
HMI-based direct manipulation Attack on P203 while the four dosingpumps are running
ManualHMI
18 HMISCADALIT101
Re-program PLC to fix LIT101value to an arbitrary value
Attack on LIT101 Studio5000
416 are cyber criminal attacks in S3-2016
Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework
Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script
TABLE IVATTACKS LAUNCHED DURING S3-2017
SNo Target Method Attack Tool
1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI
2 Historian ARP and drop Change the value stored at the Historian Ettercap
3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000
4 Tank fill level LIT301 420to 320
Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm
Manual HMI
5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI
6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2
Manual SCADA
7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC
Manual
8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)
Manual
9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script
10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000
11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI
12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap
13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000
14 RIODisplay Disconnect specific IO PIN basedon manual
Disrupt the sensor reading send to PLC throughremote IO (RIO)
Manual
15 Chemical dosing pumpP403 AIT501
Based on captured traffic betweenHMI and PLC4
Change chemical dosing function VNC Python script PycommWireshark
16 PLC LT101 from 742mm to500mm
Level 0 MITM Change the commands and values that the PLCreceives and sends
Aircrack Airodump AireplayNetfilterqueue Scapy
17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap
Compromise historian data Ettercap Aircrack
18 Pressure sensor DPIT30130MV301-4
SMB to EW get project files runFT
Disrupt valves operation of Ultrafiltration andBackwash (P3)
SMB
19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc
20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite
Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy
21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000
22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy
23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy
24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig
25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin
Manual
26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued
Manual SCADA
27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI
28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap
29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA
Pycomm
30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing
Studio 5000
31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm
TABLE VCYBER CRIMINAL ATTACKS IN S3
Cyber Criminal Attacks Insider Attacks
S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18
S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30
1 3 4 5 6 7 8 1114 23 25 26 27 31
To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination
Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian
Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601
in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304
Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which
1httpswwwcvedetailscomcveCVE-2008-2160
controlled the ICS the attackers changed the simulation tagassociated with water level in T101
Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on
VII RESULTS
Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017
A S3-2016 results
We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly
Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected
It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime
Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
Based on lessons learned during S3-2016 several newinvariants were generated coded and added to the PLCs ForS3-2017 we decided to use an additional monitoring systemplaced outside the PLCs This system collects data from theHistorian and evaluates the invariants All invariants wereimplemented in a Linux environment using a Piwebclient APIto talk to the Historian This new implementation is referredto as WDH
The invariants in WD are coded using ladder logic andstructured text while those in WDH in Python Both imple-mentations use the same set of invariants the difference is intheir placement The Historian may not get all the data andcommands that flow across the PLCs sensors and actuatorsHowever as WDH gets its data directly from the Historian ithas access to information flowing across SCADA workstationand the Historian This information may be compromised byan attacker and is not available to the PLC
A Scope of WD
WD is designed to detect process anomalies Thus anyabnormal behavior in the water treatment process in SWaTought to be detected by WD However there could be attacksthat do not cause the process to deviate from its normalbehavior but lead to undesirable consequences An exampleof such an attack is one intended to deface the screen on theSCADA workstation or the HMI Such an attack will not bedetected by WD Attacks that may cause process anomaly butonly after an attack has been removed from the system mayalso not be detected by WD Denial of Service is one suchattack
B Scope of WDH
WDH and WD use the same set of invariants Howeverthe placement of WDH could lead to a difference in detectioncapabilities of the two defense mechanisms WDH gets its datafrom Historian while WD directly from the PLC Data that isnot programmed to be logged in the Historian will not beaccessible to WDH Thus any anomaly that requires such datawill likely not be detected by WDH Similarly attacks thatmanipulate data entering the Historian or SCADA may not bevisible to WD Thus while the two invariant-based processanomaly detection mechanisms are identical in the invariantsthey use their placement in SWaT is expected to result indifferent performance in detecting attacks
VI S3 ATTACKS
The attacks launched by teams participating in the two S3
events are described next
A S3-2016 Attacks
All attacks designed and launched during S3-2016 areenumerated in Table III Three attacks selected from Table IIIare described next Details of all attacks are available in [9]Of the 18 attacks in Table III 4 and 16 are cyber criminalattacks and the remaining are insider attacks
DoS attack on SCADA In this attack (attack 4 in Table III)the attackerrsquos intention was to deface the SCADA workstationscreen and hence prevent the operator from observing plantstate The cyber-criminal attacker model was used to designthis attack To realize the intention the attacker launched anARP poisoning Man-in-the-Middle attack in two steps In thefirst step all traffic intended for HMI was redirected to theSCADA workstation In the second step this redirected trafficwas dropped and thus no packets were received at the SCADAworkstation This led to the screen on the workstation becom-ing completely gray and no state information was displayedThis attack was not detected by WD as it did not lead toany process anomaly It is an ARP spoofing attack and not atraditional DoS attack As part of the DoS attack the attackertargeted the PLC and sent millions of packets at a time Thisled to the same effect as would be the case when an ARPspoofing attack is performed on SCADA
Manipulation of the chemical dosing pump Intention of theattacker in this case (attack 14 in Table III) was to manipulatethe pH of water entering Stage 3 of SWaT The insider-attackermodel was used in the design of this attack This attack wasexecuted in two steps In the first step PLC 2 was set to manualmode Note that in manual mode the plant operator can directlycontrol the actuators eg the dosing pumps in this case In thesecond step the attacker altered the chemical dosing processin the Pre-treatment Stage 2 of SWaT by interacting directlywith the HMI interface and overriding the commands sent bythe PLC WD was able to detect this attack because the set-points changed by the attacker were different from those setin WD
DoS to PLC by SYN flooding The intention of the attackerin this case (attack 16 in Table III) was to disable the HMIso that an operator is unable to view or control the plantoperation The insider-attacker model was used in the designof this attack In this way the attacker had an access to theadministrator account and the associated tools The attackerperformed a SYN flooding attack on EthernetIP server ofPLC1
As a result of this DoS attack the HMI was unable toobtain the current state values to display and would insteaddisplay 0 or characters WD was unable to detect this attackphysical process as not affected During the attack period PLCwas controlling the process as expected Such attacks whilenot altering process behavior may impede supervision of theprocess in an operational plant
B S3-2017 Attacks
All attacks designed and launched during S3-2017 areenumerated in Table IV Selected attacks from Table III aredescribed next Details of all attacks are available in [20] Ofthe 31 attacks in Table IV 17 can be classified as cybercriminal attacks and the remaining as insider attacks (Figure I)All attacks launched during S3-2016 and S3-2017 are listedand categorized in Table V
TABLE IIIATTACKS LAUNCHED DURING S3-2016
SNO Target Method Attack Tool
1 Tank fill levelLIT101
Use HMI access Close MV101 and Stop P101 andP102
HMI
2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap
3 PLC Manual access Removed the cable at the ring atlevel 0
Manual
4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI
DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state
Ettercap
5 Tank fill levelLIT101
Use HMI access Attack on LIT101 ManualHMI
6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation
ManualSCADA
7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation
ManualSCADA
8 Historian DoS attack using CPPPO andloop
Attack between HMI and PLC CPPPO
9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed
ManualSCADA
10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA
11 Tank fill levelLIT301
Using SCADA access LIT301 set point altered ManualSCADA
12 Chemical dosingP201
Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing
Dosing pump attack on P201 ManualSCADA
13 HMISCADALIT101
Functional block introduce newconstant tag tie that to output tagcould only do zero
LIT101 set to zero from PLC Studio5000
14 Chemical dosingpump P205
Use SCADA access Manipulation of the chemical dosingpump (P205)
ManualSCADA
15 HMISCADA DoS on HMI using Level 1 net-work
Attack on HMI EttercapPycomm
16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)
Ettercap
17 Chemical dosingpump P203
HMI-based direct manipulation Attack on P203 while the four dosingpumps are running
ManualHMI
18 HMISCADALIT101
Re-program PLC to fix LIT101value to an arbitrary value
Attack on LIT101 Studio5000
416 are cyber criminal attacks in S3-2016
Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework
Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script
TABLE IVATTACKS LAUNCHED DURING S3-2017
SNo Target Method Attack Tool
1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI
2 Historian ARP and drop Change the value stored at the Historian Ettercap
3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000
4 Tank fill level LIT301 420to 320
Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm
Manual HMI
5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI
6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2
Manual SCADA
7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC
Manual
8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)
Manual
9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script
10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000
11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI
12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap
13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000
14 RIODisplay Disconnect specific IO PIN basedon manual
Disrupt the sensor reading send to PLC throughremote IO (RIO)
Manual
15 Chemical dosing pumpP403 AIT501
Based on captured traffic betweenHMI and PLC4
Change chemical dosing function VNC Python script PycommWireshark
16 PLC LT101 from 742mm to500mm
Level 0 MITM Change the commands and values that the PLCreceives and sends
Aircrack Airodump AireplayNetfilterqueue Scapy
17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap
Compromise historian data Ettercap Aircrack
18 Pressure sensor DPIT30130MV301-4
SMB to EW get project files runFT
Disrupt valves operation of Ultrafiltration andBackwash (P3)
SMB
19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc
20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite
Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy
21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000
22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy
23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy
24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig
25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin
Manual
26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued
Manual SCADA
27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI
28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap
29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA
Pycomm
30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing
Studio 5000
31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm
TABLE VCYBER CRIMINAL ATTACKS IN S3
Cyber Criminal Attacks Insider Attacks
S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18
S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30
1 3 4 5 6 7 8 1114 23 25 26 27 31
To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination
Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian
Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601
in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304
Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which
1httpswwwcvedetailscomcveCVE-2008-2160
controlled the ICS the attackers changed the simulation tagassociated with water level in T101
Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on
VII RESULTS
Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017
A S3-2016 results
We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly
Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected
It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime
Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
TABLE IIIATTACKS LAUNCHED DURING S3-2016
SNO Target Method Attack Tool
1 Tank fill levelLIT101
Use HMI access Close MV101 and Stop P101 andP102
HMI
2 HMISCADA ARP spoofing Attack HMI DoS attack Ettercap
3 PLC Manual access Removed the cable at the ring atlevel 0
Manual
4 HMISCADA DoS on HMI by droppingall packets between PLC andSCADAHMI
DoS attack on SCADA wide DoSattack took a while to restore SWaTto its normal state
Ettercap
5 Tank fill levelLIT101
Use HMI access Attack on LIT101 ManualHMI
6 Valve MV301 Use SCADA access Attack on MV301 manually openfrom the SCAD workstation
ManualSCADA
7 Pump P101 Use SCADA access Attack pump manually open it fromthe SCADA workstation
ManualSCADA
8 Historian DoS attack using CPPPO andloop
Attack between HMI and PLC CPPPO
9 Valve MV101 Use SCADA access MV101 attacked using SCADAchanged the valve state from Opento Closed
ManualSCADA
10 Pump P101 Use SCADA access LIT301 set point changed ManualSCADA
11 Tank fill levelLIT301
Using SCADA access LIT301 set point altered ManualSCADA
12 Chemical dosingP201
Control MV101 and AIT503 setpoints of LIT301 to ensure flowthis triggered chemical dosing
Dosing pump attack on P201 ManualSCADA
13 HMISCADALIT101
Functional block introduce newconstant tag tie that to output tagcould only do zero
LIT101 set to zero from PLC Studio5000
14 Chemical dosingpump P205
Use SCADA access Manipulation of the chemical dosingpump (P205)
ManualSCADA
15 HMISCADA DoS on HMI using Level 1 net-work
Attack on HMI EttercapPycomm
16 Historian SYN flood ENIP port at PLC1 DoS to PLC by SYN flooding (attackon HMI)
Ettercap
17 Chemical dosingpump P203
HMI-based direct manipulation Attack on P203 while the four dosingpumps are running
ManualHMI
18 HMISCADALIT101
Re-program PLC to fix LIT101value to an arbitrary value
Attack on LIT101 Studio5000
416 are cyber criminal attacks in S3-2016
Control of the chemical dosing system through a Pythonscript (Pycomm) The objective of this attack (attack 15 inTable IV) was to change chemical dosing at the end of the de-chlorination system (Stage 4) First the attackers compromisedVirtual Network Computing (VNC) Then they used a Pythonscript (Pycomm) and Wireshark to gain access to the HMIAfter gaining access to the HMI through the compromisedVNC the cybercriminal attacker used Wireshark to capturethe packets flowing between the HMI and PLC4 The con-troller tags were retrieved by an analysis of the packets Theattackers changed the data associated with these tags to controlthe chemical dosing function using the Pycomm framework
Control of PLC through the Bridged Man-in-the-Middle(MiTM) at Level 0 the objective of this attack was (attack 16in Table IV) to change the commands and values that PLC1receives and sends First the attackers configured a bridgebetween the RIO and PLC1 using Netfilterqueue andScapy The attack was launched at two network levels Ananalysis on the network traffic revealed the packets that theattackers should edit As the target of this attack was thewater level in T101 the attackers set it to a constant valueto hide from PLC1 the rise in water level in T101 Before apacket was forwarded Netfilterqueue rerouted it into aqueue which can be read and modified by the Python script
TABLE IVATTACKS LAUNCHED DURING S3-2017
SNo Target Method Attack Tool
1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI
2 Historian ARP and drop Change the value stored at the Historian Ettercap
3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000
4 Tank fill level LIT301 420to 320
Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm
Manual HMI
5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI
6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2
Manual SCADA
7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC
Manual
8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)
Manual
9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script
10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000
11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI
12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap
13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000
14 RIODisplay Disconnect specific IO PIN basedon manual
Disrupt the sensor reading send to PLC throughremote IO (RIO)
Manual
15 Chemical dosing pumpP403 AIT501
Based on captured traffic betweenHMI and PLC4
Change chemical dosing function VNC Python script PycommWireshark
16 PLC LT101 from 742mm to500mm
Level 0 MITM Change the commands and values that the PLCreceives and sends
Aircrack Airodump AireplayNetfilterqueue Scapy
17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap
Compromise historian data Ettercap Aircrack
18 Pressure sensor DPIT30130MV301-4
SMB to EW get project files runFT
Disrupt valves operation of Ultrafiltration andBackwash (P3)
SMB
19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc
20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite
Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy
21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000
22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy
23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy
24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig
25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin
Manual
26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued
Manual SCADA
27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI
28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap
29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA
Pycomm
30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing
Studio 5000
31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm
TABLE VCYBER CRIMINAL ATTACKS IN S3
Cyber Criminal Attacks Insider Attacks
S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18
S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30
1 3 4 5 6 7 8 1114 23 25 26 27 31
To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination
Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian
Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601
in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304
Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which
1httpswwwcvedetailscomcveCVE-2008-2160
controlled the ICS the attackers changed the simulation tagassociated with water level in T101
Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on
VII RESULTS
Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017
A S3-2016 results
We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly
Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected
It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime
Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
TABLE IVATTACKS LAUNCHED DURING S3-2017
SNo Target Method Attack Tool
1 HMISCADA LIT401 HMI simulation insider attack Change the value of LIT401 in the HMI Manual HMI
2 Historian ARP and drop Change the value stored at the Historian Ettercap
3 Valve MV201 Reprogram PLC Change the status of the MV201 Studio 5000
4 Tank fill level LIT301 420to 320
Manual Lower the water tank level from 820mm to420mm without raising any alarm LIT301decreased till 320mm
Manual HMI
5 Pump P101 Manual mode of pump Alternate the state [OnOff] of the pump P101 Manual HMI
6 Chemical dosing P205 Manually dosing chemical pump Change the chemical dosage of sodiumhypochlorite (NaOCl) in P2
Manual SCADA
7 PLC Disconnect cable Disrupt sensor values from remote inputoutput(RIO) to the PLC
Manual
8 RIO Display Disconnect IO PIN manual Disrupt the sensor reading send to PLC throughRemote IO (RIO)
Manual
9 Chemical dosing P404 MiTM Python script to control Increase chemical dosage in pre-treatment Python script
10 LIT101 (476mm to 540mm ) Reprogram PLC Falsify water level display at SCADA Studio 5000
11 Pump P101 HMI simulation insider attack Alternate the state [OnOff] of the pump P101 Manual HMI
12 HMISCADA AIT 504 ARP+rewriting Increase AIT504 Ettercap
13 PLC LIT401 Reprogram PLC Falsify water level display at SCADA Studio 5000
14 RIODisplay Disconnect specific IO PIN basedon manual
Disrupt the sensor reading send to PLC throughremote IO (RIO)
Manual
15 Chemical dosing pumpP403 AIT501
Based on captured traffic betweenHMI and PLC4
Change chemical dosing function VNC Python script PycommWireshark
16 PLC LT101 from 742mm to500mm
Level 0 MITM Change the commands and values that the PLCreceives and sends
Aircrack Airodump AireplayNetfilterqueue Scapy
17 Historian LT101 tag Aircrack WiFi ARP spoofingEttercap
Compromise historian data Ettercap Aircrack
18 Pressure sensor DPIT30130MV301-4
SMB to EW get project files runFT
Disrupt valves operation of Ultrafiltration andBackwash (P3)
SMB
19 MV201 LT101 metasploit+vnc Change the water level of the tank LIT101 Metasploit+vnc
20 Pump P501 Rogue AP disassociated Telnetwith default credentials to turnoff original AP Scapy rewrite
Disrupt pump control operation KisMAC Password crackingtool 3vilTwinAttacker TelnetScapy
21 PLC LIT101 Reprogram PLC Change level indicator value Studio 5000
22 Pump P101 LIT301 Using back-door connection Establish back-door connection Mimikatz malicious VBAMacro SOCKS proxy
23 HMISCADA P201 Netfilterqueue Scapy Change the display value of the HMI Netfilterqueue Scapy
24 Historian LIT101 Overwrote specific data stored at the Historian Microsoft PsExec ipconfig
25 RIODisplay Manual Control of the RIO through disconnected Ana-logue InputOutput pin
Manual
26 Valve MV201 Manual Permanently closed the motorised valve re-gardless of commands issued
Manual SCADA
27 RIODisplay AIT202203 Manual Change the pH value shown at HMI Manual HMI
28 PLC MV201 P101 ARP poisoning MiTM Increase the pressure at P1 Ettercap
29 Tank fill level LIT101 Lower Falsify the water level reading of the tankdisplayed at SCADA
Pycomm
30 Chemical dosing PLC2 Use Studio 5000 Change the level of the chemical used fordosing
Studio 5000
31 Pressure MV302 P3012 Using Pycomm script Change the pump state sent to the PLC Pycomm
TABLE VCYBER CRIMINAL ATTACKS IN S3
Cyber Criminal Attacks Insider Attacks
S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18
S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30
1 3 4 5 6 7 8 1114 23 25 26 27 31
To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination
Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian
Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601
in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304
Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which
1httpswwwcvedetailscomcveCVE-2008-2160
controlled the ICS the attackers changed the simulation tagassociated with water level in T101
Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on
VII RESULTS
Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017
A S3-2016 results
We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly
Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected
It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime
Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
TABLE VCYBER CRIMINAL ATTACKS IN S3
Cyber Criminal Attacks Insider Attacks
S3-2016 4 16 1 2 3 5 6 7 8 910 11 12 13 14 1517 18
S3-2017 2 9 10 12 13 1516 17 18 19 20 2122 24 28 29 30
1 3 4 5 6 7 8 1114 23 25 26 27 31
To prevent all packets from entering the queue in order notto disrupt other processes iptables was used to identifythe targeted packets entering the queue Using Scapy and acustom dissector the attacker edited the payload of the targetedpacket which was then forwarded to its original destination
Control of Historian through the Aircrack WiFi The objectiveof this attack was (attack 17 in Table IV) to compromise thedata stored in the Historian Attackers performed crack WiFipassword ARP poisoning and MiTM payload manipulationusing Aircrack and Ettercap As PLC1 was operating inthe wireless mode the cybercriminal attacker used Aircrackto obtain the password for connecting to the ICS Access Point(AP) ARP poisoning was executed to reroute traffic betweenPLC1 and the Historian through the attackerrsquos rogue terminalThe attackers then used an Ettercap filter to manipulate thenetwork packets The attackers changed the tag correspondingto LIT101 to an arbitrary value before releasing the packetsto the Historian
Control of pressure through the Server Message Block (SMB)The objective of this attacks was (attack 18 in Table IV) todisrupt the state of four motorized valves in Stage 3 to affectthe differential pressure in UF Vulnerability CVE-2008-21601
in Factory Talk software from Rockwell and in MicrosoftrsquosServer Message Block (SMB) was used by the attackersto obtain files from the HMI As the HMI was runningWindows CE it has a vulnerability that allows an attackerrsquosterminal to execute arbitrary code on the HMI Thus theattackers were able to retrieve the files to create a copy ofthe workstation From the copied workstation the attackersmanually changed the state of the valves in Stage 3 suchthat the differential pressure across the UF unit as measuredby DPIT301 became dangerously high The attackers closedvalves MV301 MV302 and MV303 and opened MV304
Control of water level in the tank through the MetasploitVNC Scanner Objective of this attack was (attack 19 inTable IV) to change the water level in tank T101 The attackersused Metasploit VNC authentication None scanner to ob-tain access to the VNC server without password protection andto check for nodes running a VNC Server Once the scannerdetected the VNC Server running without any authenticationthe attackers penetrated into the server through a VNC Clientconnection As the VNC Server was hosting the HMI which
1httpswwwcvedetailscomcveCVE-2008-2160
controlled the ICS the attackers changed the simulation tagassociated with water level in T101
Control of a pump through a rogue router The objective ofthis attack (attack 20 in Table IV) was to disrupt the controlof pump P501 The attackers used Evil twin (rogue accesspoint) method using KisMAC a password cracking tool3vilTwinAttacker Telnet and Scapy The attackersused KisMAC to scan for wireless networks in the ICS Oncethe targeted wireless network was identified the attackers useddictionary attack to crack the password After the passwordwas cracked the attackers created a rogue wireless routerwith a similar SSID and configuration They then sent a de-authentication packet to disassociate PLC5 and the originalrouter The attackers used Telnet to log into the originalrouter and shut it down Scapy was then used to modify thepackets to turn the pump on
VII RESULTS
Tables VI and VII summarize the response of WD andWDH to the attacks launched during the two S3 events Recallthat both WD and WDH contain exactly the same set ofinvariants In WD the invariants are coded and placed insidethe PLCs whereas in WDH the invariants are coded and placedat the Historian WDH did not exist during S3-2016 and hencethe response of WDH is available only for attacks launchedduring S3-2017
A S3-2016 results
We note from Table VI that 10 out of 18 attacks weredetected immediately while the remaining eight attacks werenot detected Six of the eight undetected attacks did not leadto process anomaly during the observation period and hencedid not violate any invariant This outcome is expected as theinvariants in WD are designed to detect process anomaly
Consider attack 2 ARP spoofing in Table III This is aDoS attack on HMI It leads to defacing the screen on theHMI or displaying incorrect information thereby preventingan operator from knowing the actual plant state Howeverthe attack does not cause process anomaly and hence is notdetected as it does not violate any invariant Similar logic canbe used to explain why the other attacks in Table VI are notdetected
It is important to note that a DoS attack when given enoughtime to evolve and be launched at an appropriate state of theplant may impact physical process behavior In such a caseone or more invariants may detect the attack One such attackis 16 in Table VI This attack prevented the Historian fromreceiving data from PLC1 However if this attack was leftactive for a longer period it would prevent PLC1 from sendingappropriate commands to the actuators eg to MV101 orP101 In turn this would have led to process anomaly Notenough data is available to conclude with certainty whether ornot this attack would be detected by WD if active for sufficienttime
Two single point [2] attacks were not detected by WD Inone attack (attack 6 in Table III) the adversary altered the status
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
of valve MV301 Under normal circumstances this valve isopened during the backwash process However the attackeropened it when there was no backwash Hence the attackdid not affect the physical process except in changing thevalve status No invariant was violated due to this attackbecause the backwash process ie Stage 6 is not includedin this case study The second single point attack (attack 17 inTable III) was performed on chemical dosing pump P203 whilethe other pump P204 was running Note that under normalcircumstances only one of these two pumps is supposed to berunning while the other remains as a backup Subsequently theattacker shut down pump P204 This attack was not detectedbecause there were no invariants that related to the chemicalproperties of water
Although the overall performance of WD was below 100it did detect all attacks within its scope except two (attacks 6and 17 in Table III) as mentioned earlier
B S3-2017 results
Table VI indicates that 21 out of 31 attacks were detectedby WD while 24 out of 31 attacks were detected by WDHConsidering only the attacks within its scope as mentionedin Section V-A WD detected 21 out of 28 attacks (75)Similarly WDH detected 24 out of 31 attacks (7741) withinits scope mentioned in Section V-B Three attacks on theHistorian are not in the scope of WD All attack targets relatedto RIODisplay (in Table II and in Table IV) are not detectedby both WD and WDH This is because registers inside aPLC save the previous values received from the sensors andthe PLC continues to execute the control code The invariantsalso use the same values stored in the PLC registers and hencedo not raise an alert
In general PLCs send to the Historian via the SCADAworkstation the data received from the sensors When a PLCdoes not have updated values during the attack period it isobvious that the Historian also receives the same stale valuesThis is the reason why WDH also did not detect attacksrelated to RIODisplay Note that the RIODisplay attacks werelaunched and remained active only for a few seconds Duringthis period the PLC did not update the current sensor valuescoming through the RIO If the same attack is performed for alonger duration the PLC would update the data received fromthe sensors Doing so would likely lead to WD and WDHdetecting the RIO attacks
Attacks launched on the Historian were detected by WDHbut not by WD This variance is due to the fact that data inthese attacks is manipulated at the Historian Thus invariantsin a PLC do not have access to the manipulated data andhence the invariants in WD do not raise any alert All attackstargeting a PLC are detected by WD and WDH
WD Detection of physical process attacks All attacks onvalves pressure sensor and level sensors were detected Threeout of four attacks on the chemical dosing process pumpswere detected An example of a detected attack is when theattackers took control of pump P301 (attack 20 in Table IV)
TABLE VIPERFORMANCE OF WD AND WDH
S3-2016 S3-2017
WD WD WDH
Detected 1 5 7 910 11 1213 14 18
3 4 7 9 1011 12 13 1516 18 19 2021 22 23 2628 29 30 31
2 3 4 7 9 1011 12 13 15 1617 18 19 20 2122 23 24 26 2829 30 31
Not detected 2 3 4 68 15 1617
1 2 5 6 8 1417 24 25 27
1 5 6 8 14 2527
through a Python script (Pycomm) to raise the pressure in theUF unit measured by sensor DPIT301 to a dangerous levelWD immediately raised an alarm This invariant ensured thatpump P301 must be OFF when the pressure at DPIT301 wasabove a threshold During the attack the invariant was violatedas the pump was not turned off while DPIT301 indicatedreadings that were above the threshold Consequently an alarmwas raised immediately In certain cases multiple alarmswere raised due to the violation of one or more invariantsFor example when level sensor LIT101 was compromisedthe invariants corresponding to this sensor were violated andraised alarms
WD Detection of sensor data attack WD detected attacks onHMISCADA and PLC values because these attacks directlycompromised the physical processes These attacks eithercompromised chemical dosing water tank levels or pumpstatus through hacking of the HMISCADA or PLC Hencethe robustness of WD in detecting unusual physical processbehavior was found effective in these attacks On the otherhand WD was unable to detect insider attacks that pulled outRIO cables This is because WD triggers an alarm only whenthe invariants are violated Under normal circumstance for aperiod of time a PLC continues to execute its control codeand any invariant code based on the last known state andorvalues Thus the invariants located inside the PLCs are unableto observe this anomalous behavior
WDH Detection of physical process attacks WDH detected14 out of 16 physical process attacks
WDH Detection of sensor data attacks WDH detected theattacks on HMISCADA and PLC values because these attacksdirectly compromised the physical processes albeit with aslightly lower detection rate when compared with the rate ofdetecting physical process attacks As with WD WDH did notdetect any attack launched against the Remote IO by pullingthe cables that connect it to the corresponding PLC WDHfared better in the detection of attacks against the Historian asit was directly accessing data on the Historian server
If the Historian itself or data that is input to the Historian iscompromised WDH takes the decision based on the input itreceives A clever and powerful attacker can attack the physical
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
TABLE VIIRESULTS FROM S3 2017
Target of Attack Noofattacks
WD WDH
Physical Process Attacks
State of motorised valves 2 100 100
State of water pumps 4 75 75
Pressure in UF 2 100 100
Water tank level 4 100 100
Chemical dosing 4 75 75
Sensor Data Attacks
Data in historian 3 0 100
Data in HMISCADA 3 67 67
Tampering PLC communi-cations
5 100 100
Tampering Remote IO 4 0 0
Total Attacks 31 6774 7741
process and modify values entering the Historian and thusdeceive WDH In general such a situation may arise in allbehavioral intrusion detection systems where the detector takesthe decision based on incorrect input data
Indeed data that appears to be ldquolegitimaterdquo could lead theWDH into believing that there is nothing wrong with thephysical process though there actually is However doingso requires the attacker to continuously manipulate a largenumber of state variables For example consider an attackwhere the attacker turns a pump say P101 ON when it shouldbe OFF and (continually) sends the state of the pump as OFFto the Historian and the corresponding PLC If the pump isOFF then the level of the source and destination tanks must berespectively decreasing and increasing at rates determined bythe pump characteristics Creating ldquolegitimate-lookingrdquo datathus requires an attacker to manipulate several state variablesas explained next (a) Two state variables that correspondto tank levels Two sensors (in SWaT) measure these statevariables (see Figure 2) Thus the attacker must have accessto these level sensors (b) If pump P101 is actually ON whilethe Historian receives its state as OFF then FIT201 must showno flow Thus the attacker will also need to manipulate FIT201to avoid detection This argument can be carried forward tosubsequent stages to show that many sensors will need to bemanipulated by an attacker to ldquohiderdquo a simple attack such asldquochange the state of a pumprdquo In summary yes incorrect dataat the Historian could prevent detection though doing so wouldbe a significant challenge for the attacker due primarily to thedistributed nature of the invariants
VIII DISCUSSION
A Challenges faced
We faced several challenges during S3 For example aftereach teamrsquos performance the operator was required to bringSWaT back to a predefined normal state It was necessary to
keep SWaT in a normal state before another team launched at-tacks Bringing SWaT to its normal state required (a) resettingnetwork communications to ensure that all the communicationchannels are operating as expected (b) the operator to ensurethat all physical processes in SWaT are stable with respectto the control logic (c) the operator to bring back SWaT tothe normal state of that particular device such as a pump or amotorized valve in the case of any physical or manual attacksby the previous team and (d) that the Historian and SCADAservers were reverted to their original state ie the state thatexisted prior to the launch of attacks
B Research questions
RQ1 How do attackers compromise the security of an ICS InSection VI we presented and categorized the attacks based onattacker profiles An attacker can launch physical attacks wheninside the plant such as manually operating a motorized valveor tampering with network cabling Several attacks launchedby the attack teams had not been launched by the authorsin their evaluation of WD [1] and WDH Thus S3 raisedour confidence in the effectiveness of the attack detectionmechanisms based on invariants derived from plant designs
RQ2 How effective is WD in detecting attacks launched byindependent attack teams As mentioned earlier while bothWD and WDH were found to detect a number of attacksthey did fail in several cases Given that the invariants derivedare intended to detect process anomalies it is clear that suchmechanisms must be used in conjunction with other attackdetection tools such as those in [24] [27] [21]
C Assessment by the authors and by independent teamsTable VIII lists the number of attacks launched by the
authors in an experimental evaluation performed prior to S3-2016 [1] Note that the WD detection rate observed by theauthors (89) was higher than the combined rate observedduring the two S3 events (6326) The difference in perfor-mance is due to different attack vectors used in the three setsof experiments WDH detection rate observed during S3 eventis (7741) which is much higher than the WD detection rateSome of these attack vectors are explained in Section VI andthe remaining may be found in [20]
TABLE VIIIPERFORMANCE OF WD AS EVALUATED BY THE AUTHORS AGAINST THOSE
BY PARTICIPANTS IN S3
Experiments by Attacks
Launched Detected (WD) Detected (WDH)
Authors 37 33 (89) NA
S3-2016 18 10 (555) NA
S3-2017 31 21 (677) 24 (774)
NA WDH did not exist at the time of experimentation by the author andduring S3-2016
The data in Table VIII is indicative of the value of orga-nizing S3 events Specifically in the case described in this
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
paper the two S3 events led to an increased confidence inthe effectiveness of the invariant-based approach in detectingcyber attacks The hackfests also led to the creation of newtypes of attack vectors that were not used earlier to assess theperformance of WD and WDH in detecting cyber attacks
D False alarms
The performance of any attack detection method ought tobe assessed using its detection accuracy ie how many of thelaunched attacks it detects as well as the rate at which falsealarms are raised During S3 each team attempted to launchseveral attacks The attacks listed in Tables III and IV are theones that were successful in realizing the stated attacker intentand were scored by the judges The remaining attacks werenot recorded and hence any alarm generated by such attackswas not considered Some of these unrecorded alarms couldbe false though no specific claims can be made about theirnature
Since S3-2017 the authors have observed no false alarmsfrom WD during normal operation of SWaT WDH has beenin operation since a few weeks prior to S3-2017 Againduring the normal operation of SWaT no alarm has beengenerated by WDH This observation should not be construedto imply that an invariant-based attack detection mechanismwill not generate any false alarmndash in fact it could Howeverif the invariants generated are complete in the sense that theyaccurately capture all aspects of process behavior and theirimplementation is correct and tuned properly the likelihoodof false alarms is low
Even though SWaT is a relatively new plant (2-years sinceits inauguration at the time of writing this paper) we doobserve intermittent failures in a few motorized valves Forexample sometimes MV101 in Stage 1 takes much longer toopen than expected by its controlling PLC1 The PLC itselfdetects such cases In such a case WD or WDH dependingon the time it takes for the valve to finally open will raisean alarm We do not consider this as a false positive simplybecause whether an anomalous behavior is due to a naturalcause or a cyber attack cannot be distinguished by WD orWDH While such distinction is important to make additionalresearch is needed to distinguish process anomalies due tocyber attacks and those arising due to natural componentfailures
E Benefits of S3
S3 exposed the organisers participants and researchers tohow an attacker might design and launch attacks on ICS Bene-fits of S3 include the following 1) An improved understandingof how an ICS operates and the consequent formulation ofnew research directions 2) Opportunity for participants fromindustry and academia to learn from the event and focus onthe limitations of their work 3) An aid to the ICS managementteam to observe the defense teams thus leading to possibleadoption of technology embedded in WD or WDH
F Placement of WD
The placement of WD is another question that ought tobe looked into carefully In this work WD is placed insidePLCs However an exceptionally large number of invariantsmay prevent adding code to the existing control code in a PLCThis may happen due to the computational load requirementson a PLC This aspect led us to create WDH that is placedon the plant network and gets its data from the Historian toevaluate the invariants
G Forensics
One advantage of the invariant-based approach for attackdetection appears while determining the area of impact ofan attack When a single invariant is violated it indicatesclearly the source of process anomaly For example an alertis generated if valve MV101 is closed when the water intank T101 is at or below the L level marker While this alertdoes not indicate how an attacker entered the system or ifthe valve or the level sensor is defective it does assist inlocalising the reason for the alert The analysis becomes abit more complex when multiple invariants raise alerts Thisaspect of an invariant-based detection mechanisms remains tobe analyzed in further detail
H Attacker capabilities
We do not have any validation of the professionalism of theS3 attack teams As mentioned earlier [20] [35] [36] attackteams were from a variety of backgrounds including fromthe industry and academia from Europe and Asia During S3-2017 one team consisting of four membersndashall from outsideof Singaporendash focuses on ethical hacking and cyber-warsinvolving critical infrastructure This team is part of a globalalliance The other teams consist of hackers interested inknowing how vulnerabilities in software can be exploitedand passes this information to others for improving systemssecurity Coverage of attacks launched by the attack teams andattacker profiles is discussed in Section IV and summarizedin Tables I II V and VII
I Attack trees
It is possible to use attack trees [37] [42] to model attackslaunched during the two hackfests reported in this paper Doingso would enable mapping each attack to a specific path inthe attack tree and reveal which attack paths in SWaT weretraversed Such modeling and analysis has not been attemptedin this work and is a possible subject for future research
IX RELATED WORK
S3 is a Capture-The-Flag [15] event on ICS TraditionalCTF events generally attract the attention of both industrialand academic teams and currently enjoy increasing popularityas indicated in [15] The number of such events is graduallyincreasing [13] [16] Such events aid in learning about secu-rity vulnerabilities how these could be exploited nature ofattacks and strength of the deployed [18] [33] [45] defensemechanisms To the best of our knowledge S3 is the first CTF
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
style event of its kind in ICS that involves participants from theindustry and academia and focuses on an operational watertreatment testbed
The study reported here focuses on cyber attacks on ICS thatresult in deliberate data and command manipulation Injectionof such attacks in ICS has been studied by several researchersAttacks have been modeled as noise in sensor data [28] [47]Authors previously presented cyber physical attacker model [2]to aid in the design of cyber physical attacks on ICS Attackermodels designed specifically for ICS include a variety ofdeception attacks including surge bias and geometric [11]Such models have been used in experiments to understandthe effectiveness of statistical techniques in detecting cyberattacks
There exist several techniques other than the type usedin WD for the detection of process anomalies CPAC [19]presents stateful detection mechanisms to detect attacksagainst control systems The Weaselboard [31] uses PLC back-plane to get the sensor data and actuator commands and analy-ses them to prevent zero day vulnerabilities WeaselBoard [31]has a dedicated device and detects changes in control settingssensor values configuration information firmware logic etc
The invariants in WD use data from multiple stages to en-able distributed detection of cyber attacks Such sensor fusionhas been proposed by several researchers In safety criticalcyber physical systems this was reported in [26] In [38] itis shown how safety critical systems are interconnected andtheir complexity Model based attack detection schemes inwater distribution systems was presented in [7] It uses theMatlab system identification tool to get a model from thedata generated in a water distribution system The data drivenmodel is helpful in detecting process anomalies
Monitoring the physics of the system has been studiedin [22] Cardenas et al [44] have experimented with the useof CUSUM in detecting stealthy attacks Hsio et al [23] haveproposed a distributed security monitoring solution to detectattacks on an ICS There exists literature on the design ofrobust ICS [28] [46] These works focus on attack modellingand the design of controllers and monitors for secure ICS
X CONCLUSION
There exist a number of devices for defending networksand ICS against cyber attacks Firewalls attempt to preventattackers from entering an ICS Intrusion Detection Systems(IDSs) attempt to detect if an unauthorized user has entered theplant network The approach used in WD is orthogonal to thatused in most commercially available firewalls and IDS WDuses a design-centric approach to detect process anomaliesin contrast to network traffic anomalies that are the focus ofseveral IDS Thus WD is effective in detecting attacks by anexternal or an internal agent One could consider WD as alast-mile defense
While in the study reported here WD has been foundeffective in detecting attacks that lead to process anomaly itdoes fail in detecting attacks such as a replay attack where aplant operator views the system state that is different from the
actual state This ineffectiveness of WD ought to be consideredwhen using such a system in critical infrastructure
It is interesting to observe that there exist attacks that aredetected by both WD and WDH though vice-versa is not trueFor example attack 17 in Table IV was detected by WDHbut not by WD This observation suggests that when feasibleboth systems ought to be deployed simultaneously
The invariants used in WD and WDH were derived andcoded manually For a system such as SWaT the manualapproach is feasible as the plant has 42 sensors and actuators ascompared to perhaps hundreds or more in commercial plantsThus there needs to be an automated way of generating andcoding the invariants
The attacks launched by teams during the hackfests couldlater serve as a source for assessing the effectiveness of attackdetection mechanisms developed by other researchers Detailsof all attacks launched during the hackfests are therefore madepublic and available in [9] [20] [41]
It should be obvious that any attack detection mechanismincluding WD is one component of a holistic defense systemagainst cyber attacks on any critical infrastructure This paperdoes not address an important question What action should betaken and how when an alarm is raised by WD or WDHrdquoThis remains an open question
ACKNOWLEDGMENTS
A number of people were involved in the planning executionand post-data analysis during the two hackfests reported in thispaper Our thanks are due to Nils Tippenhauer Martin Ochoaand the staff of iTrust for organizing and judging the eventsKaung Myat Aung for invaluable assistance in the actual con-duct of the events Gyanendra Mishra for implementing WDHthe entire team of authors of the S3-2017 report [20] namelyFrancisco Furtado Lauren Goh Sita Rajgopal Elaine CheungEricson Thiang Toh Jing Hui and Ivan Lee to the SUTD-MIT International Design Center for partially supporting S3-2017 and to all the participants who traveled long distancesto come to Singapore to participate in the two hackfests Lastbut not the least thanks to the reviewers for their commentsthat helped improve the original manuscript
REFERENCES
[1] S Adepu and A Mathur Distributed detection of single-stage multipointcyber attacks in a water treatment plant In Proceedings of the 11th ACMon Asia Conference on Computer and Communications Security ASIACCS rsquo16 pages 449ndash460 2016
[2] S Adepu and A Mathur Generalized attacker and attack models forcyber physical systems In 2016 IEEE 40th Annual Computer Softwareand Applications Conference (COMPSAC) pages 283ndash292 June 2016
[3] S Adepu and A Mathur An investigation into the response of a watertreatment system to cyber attacks In 2016 IEEE 17th InternationalSymposium on High Assurance Systems Engineering (HASE) pages141ndash148 Jan 2016
[4] S Adepu and A Mathur Using Process Invariants to Detect CyberAttacks on a Water Treatment System pages 91ndash104 2016
[5] S Adepu and A Mathur Water-defense -a method to detect multi-pointcyber attacks on water treatment systems US provisional applicationno 623146 March 2016
[6] S Adepu S Shrivastava and A Mathur Argus An orthogonal defenseframework to protect public infrastructure against cyber-physical attacksIEEE Internet Computing 20(5)38ndash45 Sept 2016
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-
[7] C M Ahmed C Murguia and J Ruths Model-based attack detectionscheme for smart water distribution networks In Proceedings of the2017 ACM on Asia Conference on Computer and CommunicationsSecurity pages 101ndash113 ACM 2017
[8] Allen-Bradley Logix5000 Controllers Structured Text Program-ming Manual Publication 1756-PM007D-EN-P Rockwell AutomationNovember 2012
[9] D Antonioli H R Ghaeini S Adepu M Ochoa and N O Tip-penhauer Gamifying education and research on ICS security Designimplementation and results of S3 CoRR abs170203067 2017
[10] The Bro network security monitor httpswwwbroorg[11] A A Cardenas S Amin Z-S Lin Y-L Huang C-Y Huang and
S Sastry Attacks against process control systems Risk assessmentdetection and response In ACM Symp Inf Comput Commun Security2011
[12] Check Point Critical Infrastructure amp ICSSCADA httpwwwcheckpointcomproducts-solutionscritical-infrastructureindexhtml
[13] N Childers B Boe L Cavallaro L Cavedon M Cova M Egele andG Vigna Organizing large scale hacking competitions In Proveedingsof conference on Detection of Intrusions and Malware and VulnerabilityAssessment (DIMVA) 2010
[14] P Cobb German steel mill meltdown Rising stakes in the internet ofthings 2015
[15] CTFtime httpsdefconorg Accessed 2016-10-19[16] DEF CON conference httpsdefconorg Accessed 2017-10-19[17] ICS-CERT Advisories httpsics-certus-certgovadvisories[18] C Eagle and J L Clark Capture-the-flag Learning computer security
under fire Technical report DTIC Document 2004[19] S Etigowni D J Tian G Hernandez S Zonouz and K Butler
Cpac securing critical infrastructure with cyber-physical access controlIn Proceedings of the 32nd Annual Conference on Computer SecurityApplications pages 139ndash152 ACM 2016
[20] F FURTADO L GOH S RAJAGOPAL E CHEON E THIANG T JHui and I LEE Swat security showdown (s3-17) event report Technicalreport iTrust Singapore University of Technology and Design 2017
[21] H R Ghaeini and N O Tippenhauer Hamids Hierarchical monitoringintrusion detection system for industrial control systems In Proceedingsof the 2Nd ACM Workshop on Cyber-Physical Systems Security andPrivacy CPS-SPC rsquo16 pages 103ndash111 2016
[22] D Gollmann and M Krotofil Cyber-Physical System Security pages195ndash204 Springer Verlag 2016
[23] S-W Hsiao Y S Sun M C Chen and H Zhang Cross-levelbehavioral analysis for robust early intrusion detection In Intelligenceand Security Informatics (ISI) 2010 IEEE International Conference onpages 95ndash100 IEEE 2010
[24] ICS2 On Guard httpics2comproductsics2-on-guard-2[25] httpsics-certus-certgov[26] R Ivanov M Pajic and I Lee Attack-resilient sensor fusion for
safety-critical cyber-physical systems ACM Transactions on EmbeddedComputing Systems (TECS) 15(1)21 2016
[27] KICS Kaspersky Lab httpsicskasperskycom[28] C Kwon W Liu and I Hwang Security analysis for cyber-physical
systems against stealthy deception attacks In American Control Con-ference (ACC) 2013 pages 3344ndash3349 2013
[29] R Lipovsky New wave of cyber attacks against Ukrainian powerindustry January 2016 httpwwwwelivesecuritycom20160111
[30] A P Mathur and N O Tippenhauer SWaT A water treatment testbedfor research and training on ICS security In 2016 International Work-shop on Cyber-physical Systems for Smart Water Networks (CySWater)pages 31ndash36 April 2016
[31] J Mulder M Schwartz M Berg J R Van Houten J Mario M A KUrrea A A Clements and J Jacob Weaselboard Zero-day exploitdetection for Programmable Logic Controllers Technical report techreport SAND2013-8274 Sandia National Laboratories 2013
[32] ODVA EthernetIP technology overview httpswwwodvaorgHomeODVATECHNOLOGIESEtherNetIPaspx
[33] J Radcliffe Capture the flag for education and mentoring A casestudy on the use of competitive games in computer security train-ing httpwwwsansorgreading-roomwhitepaperscasestudiescapture-flag-education-mentoring-33018 2007
[34] M Rocchetto and N O Tippenhauer On attacker models and profilesfor cyber-physical systems In Proceedings of the European Symposiumon Research in Computer Security (ESORICS) 2016
[35] S3-2016 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2016s3
[36] S3-2017 SWaT Security Showdown (S3) httpsitrustsutdedusgscy-phy-systems-week2017-2s317-event
[37] V Saini Q Duan and V Paruchuri Threat modeling using attack treesJ Comput Sci Coll pages 124ndash131 2008
[38] J A Stankovic Research directions for cyber physical systems inwireless and mobile healthcare ACM Trans Cyber-Phys Syst pages11ndash112 Nov 2016
[39] K Stouffer and J F K Scarfone Guide to Industrial Control Systems(ICS) Security NIST Special Publication 800-82 pages 1-155 June2011
[40] SWaT Secure Water Treatment Testbed 2015 httpsitrustsutdedusgwp-contentuploadssites3201511Brief-Introduction-to-SWaT 181115pdf
[41] SWaT dataset and models httpsitrustsutdedusgdataset[42] C-W Ten C-C Liu and M Govindarasu Vulnerability assessment
of cybersecurity for SCADA systems using attack trees In PowerEngineering Society General Meeting 2007 IEEE pages 1ndash8 June2007
[43] D Urbina J Giraldo N O Tippenhauer and A Cardenas Attackingfieldbus communications in ICS Applications to the SWaT testbed InSingapore Cyber-Security Conference (SG-CRC) pages 75ndash89 2016
[44] D I Urbina J A Giraldo A A Cardenas N O TippenhauerJ Valente M Faisal J Ruths R Candell and H Sandberg Lim-iting the impact of stealthy attacks on industrial control systems InProceedings of the 2016 ACM SIGSAC Conference on Computer andCommunications Security CCS rsquo16 pages 1092ndash1105 2016
[45] G Vigna Teaching network security through live exercises In Securityeducation and critical infrastructures pages 3ndash18 Springer 2003
[46] A Wasicek P Derler and E Lee Aspect-oriented modeling of attacksin automotive cyber-physical systems In Design Automation Conference(DAC) 2014 51st ACMEDACIEEE pages 1ndash6 June 2014
[47] S Weerakkody Y Mo and B Sinopoli Detecting integrity attackson control systems using robust physical watermarking In IEEE 53rdAnnual Conference on Decision and Control (CDC) pages 3757ndash3764Dec 2014
[48] S Weinberger Computer security Is this the start of cyberwarfareNature 174142ndash145 June 2011
BIOGRAPHY
Sridhar Adepu is a PhD student in Information SystemsTechnology and Design pillar at the Singapore University ofTechnology and Design His research focuses on verificationsafety security and reliability of Cyber-Physical Systems
Aditya Mathur is a Professor of Computer Science at PurdueUniversity and Head of Pillar Information Systems Technologyand Design at the Singapore University of Technology and De-sign Aditya is Center Director of iTrust a center for researchin cyber security Design of secure public infrastructure is afocus of his current research
- I Introduction
- II Preliminaries and Background
-
- II-A Industrial Control Systems
- II-B SWaT Architecture and components
- II-C An illustrative attack on SWaT
-
- III Overview of WD
-
- III-A State-Dependent (SD) invariants
- III-B State-Agnostic (SA) invariants
-
- IV SWaT Security Showdown (S3)
-
- IV-A S3-2016
- IV-B S3-2017
- IV-C Attack targets
-
- V Preparation for S3
-
- V-A Scope of WD
- V-B Scope of WDH
-
- VI S3 Attacks
-
- VI-A S3-2016 Attacks
- VI-B S3-2017 Attacks
-
- VII Results
-
- VII-A S3-2016 results
- VII-B S3-2017 results
-
- VIII Discussion
-
- VIII-A Challenges faced
- VIII-B Research questions
- VIII-C Assessment by the authors and by independent teams
- VIII-D False alarms
- VIII-E Benefits of S3
- VIII-F Placement of WD
- VIII-G Forensics
- VIII-H Attacker capabilities
- VIII-I Attack trees
-
- IX Related Work
- X Conclusion
- References
-