fourth annual secure and resilient cyber …...this report was compiled from sessions at the mitre...

This report was compiled from sessions at the MITRE Fourth Annual Secure and Resilient Cyber Architectures Invitational, 2014 Approved for Public Release. Distribution Unlimited. Case Number 15-0704.

©2015 The MITRE Corporation. All Rights Reserved.

Fourth Annual Secure and Resilient Cyber Architectures Invitational _____________________________________________________

Overview The purpose of the Annual Secure and Resilient Cyber Architectures Invitational (previously referred to as a workshop) is to bring the community together to work collectively in order to accelerate recognition and adoption of cyber resilience. The first workshop in October 2010 established the initial community and shared architectural technical and policy perspectives on cyber resiliency. The second workshop, held in May 2012, was focused on collaborating to develop a communal view of resiliency frameworks, engineering principles and metrics. The third annual workshop, held in June 2013, focused on collaborating on favorable conditions for use of specific resiliency techniques, assessing the use of techniques in enterprise architectures and developing use cases. The 4th Annual Secure and Resiliency Architectures Invitational, cooperatively supported by The MITRE Corporation and the National Security Agency’s Research and Information Assurance Directorates, focused on applying cyber resilience to space and critical infrastructure, designing a cyber resilience challenge and cyber resilience throughout the systems engineering life cycle. There were over 140 attendees that came from government, industry and academia. In addition there were vendor booths from 11 companies with cyber resiliency offerings. The invitational began with a few speakers to set the context followed by a day and a half of parallel track working group sessions. The invitational concluded with briefings on the results from each working group at the conclusion of the second day. This report presents a summary of the keynote speakers, and the results of the discussions and follow-on interactions during the four working group sessions. The report captures the points that participants considered the most salient portions of the discussion, items of consensus, questions raised during the discussions and comments on the next steps. All other materials from the invitational and briefings can be found at http://www.mitre.org/cyberworkshop. We welcome comments from readers through the contact email address: [email protected]. The Cyber Resiliency Invitational Committee March 2015

http://www.mitre.org/cyberworkshop

iii


Executive Summary

The Fourth Annual Secure and Resilient Cyber Architectures Invitational was held on May 28-29, 2014. The purpose of the Annual Secure and Resilient Cyber Architectures Invitational was to bring the community together to produce community cooperation around the theme of Moving Cyber Resilience into Practice. Day 1 featured keynote addresses by government and commercial leaders. The briefings that day included resiliency perspectives on security operations, attack techniques, the financial community’s perspective on cyber resiliency, and resiliency lessons learned from tabletop exercises. The end of day 1 and all of day 2 consisted of facilitated working groups in four tracks:

• Track 1: Applying Cyber Resilience to Space • Track 2: Resilient Critical Infrastructures – Applying Cyber Resilience and Overcoming

the Barriers to Acceptance • Track 3: Cyber Resiliency and its Roles in the Systems Engineering Lifecycle • Track 4: Designing a Cyber Resiliency Challenge for Integration and Demonstration

Track 1: Applying Cyber Resilience to Space

The session chair was Roberta Ewart, Chief Scientist, Space and Missile Systems Center (SMC), Air Force Space Command, Los Angeles AFB (LAAFB), CA. The session on applying cyber resilience to space had four goals:

• To share current state of the practice including updates to the 8500 series, focusing on the Risk Management Framework (RMF), not just on Department of Defense Information Assurance Certification and Accreditation Process (DIACAP) compliance.

• To gain an improved understanding of critical needs in relation to the state-of-the-art. • To identify development priorities and user considerations based on the above

information and understanding. • To develop and document community-vetted best practices for transition to Cyber

Enhanced Space Operations (CESO).

This session identified three barriers to overcome and three directions forward with four concrete recommendations. The barriers to overcome are:

• Lack of a common language – without a common language it is difficult to frame the problem so that it is understood by everyone across all domains; words mean different things in different domains.

• Conflicting policies – Polices that limit with whom information can be shared come into conflict with the policies requiring that certain types of information be shared.

iv


• Competing requirements – Balancing the requirements of minimizing cost, making fully resilient and widely capable systems and enterprises requires deep knowledge across multiple domains.

Three directions forward were identified: • Improve risk governance – this is important because cyber security is becoming

necessary to a wide range of systems that were previously isolated from cyber issues, e.g., cars.

• Identify mission dependencies and make segmentation decisions based on these dependencies.

• Identify resiliency tradeoffs – resiliency tradeoffs need to be evaluated at the Integrated Planning Process, Analysis of Alternatives, Science and Technology, and Operational Levels. The tradeoffs must be reevaluated and updated on a continuing basis as the threat evolves and mission needs change.

Four main recommendations were made: • Decide on a common language. • Clarify mission dependencies and tradeoffs at all levels for consistent risk governance. • Implement segmentation – not everything needs to access the Internet. • Use the input from the breakout session for the CESO tiger team.

Track 2: Resilient Critical Infrastructures – Applying Cyber Resilience and Overcoming the Barriers to Acceptance

The session chair was Nick Multari, Pacific Northwest National Laboratory. This session had three goals:

• Reach a common understanding of terms in relation to critical infrastructures because resilience means different things to different people.

• Define what resilience would ideally bring to the environment for each critical infrastructure type.

• Examine the barriers that need to be addressed in order to ease the concerns of decision makers and identify the steps to address these barriers.

This breakout session adopted the following definition of resilience for a critical infrastructure: the extent to which a complex cyber system can continue to deliver critical services in the face of impediments. Three common themes were found across the four critical infrastructure types examined (communications, finance, energy and transportation):

• Availability is often the priority for the services provided by each domain because of the important role these domains play in supporting life and commerce in the U.S. To achieve this, in most cases confidentiality and integrity of data must also be preserved.

v


The means for ensuring the availability, confidentiality and integrity aspects of security are often opposed to each other.

• There is a need for trusted, shared situational awareness. Data and analysis that support decision making and resilient responses must be trusted (making it yet another attack surface), and must be approached community-wide.

• Because of the high premium on availability, resilient responses must enable critical services to persist even when some parts of the system are compromised or lost.

In addition, the group identified four common barriers to acceptance of resilience within critical infrastructure which require policy changes at the national and international level:

• Critical infrastructure domains do not currently have the means to absorb the cost of realizing resilience. Overcoming this barrier will require government and private R&D investments that drive resilience technologies toward standardized, interoperable COTS and government-off-the-shelf solutions.

• There is a need for economic incentives around resilience. • The interdependence of critical infrastructure domains, leads to a situation where

resilience in one domain depends on resilience in the other domains. Overcoming this barrier must be approached with a multi-domain shift toward resilience supported by cross-domain coordination and a cultural shift that instills the value of resilience into the full technology development cycle.

• There are significant trust and sharing concerns. In the highly competitive environments of most critical infrastructure domains, sharing between entities eliminates competitive advantage, and in some cases is illegal. There are also trust concerns arising from reliance on non-owned components and services and geopolitical issues for multinational companies.

With these themes as the background, the session produced the following five recommendations: • New international laws, standards, and agreements are needed to clarify how resilience

technologies can be used and shared. • Critical infrastructures need to mutually arrive at resilience because of their

interdependence. • Economic incentives need to be provided to motivate resilient operations. • COTS solutions for resilience need to be developed and made available in the

marketplace; R&D must continue in resiliency technologies. • A well-trained workforce needs to be developed and maintained.

Track 3: Cyber Resiliency and its Roles in the Systems Engineering Lifecycle

The session chairs were Ron Ross, National Institute of Standards and Technology (NIST) and Richard Graubart, The MITRE Corporation.

This track focused on cyber resiliency and how it should fit into the broader concept of system security engineering. Thus the track needed participants to achieve a common understanding of

vi


what is system security engineering, what is cyber resiliency and why it is needed. This track also needed participants to provide recommendations regarding what needs to be done differently to incorporate resiliency into the system lifecycle. The goals of this track were to:

• Identify where and how to better incorporate cyber resiliency into the systems engineering life cycle processes.

• Recommend concrete actions that could be taken to better achieve this outcome over the next year.

There are nine recommendations that came out of the session. • Create a publically releasable set of cyber resiliency requirements that could be employed

(with some tailoring) to a broad set of systems. • Establish a common set of terminology for cyber resiliency. The most logical place for

this at the current time is NIST SP 800-160. • Identify and articulate the various roles and responsibilities with regards to cyber

resiliency. The most logical place at the current time for this is in NIST SP 800-160. • Develop an authoritative cross government means of assessing threats which

organizations could consider when planning for resiliency. • Develop a cyber resiliency overlay to NIST SP 800-53. • Educate senior officials on the difference between traditional cyber security and cyber

resiliency. • Develop guidance explaining the relationship between the traditional cyber security

activities and cyber resiliency activities. • Supplement traditional CONOPS with a cyber resiliency perspective. • Incorporate resiliency in major government system undertakings.

Track 4: Designing a Cyber Resiliency Challenge for Integration and Demonstration

The session chair was Harriet G. Goldman, The MITRE Corporation. This track focused on the question of how to define a “design challenge” for cyber resilience. The group characterized a “design challenge” as a statement of a problem that can be used as the basis for evaluating some combination of products, emerging technologies, architectural principles, and operating procedures. A cyber resilience problem is the problem of how to achieve specific cyber resilience goals and objectives (defined in "Cyber Resiliency Engineering Framework,” D. Bodeau and R. Graubart, available at: http://www.mitre.org/sites/default/files/pdf/11_4436.pdf) in the context of a set of assumptions about the technical, operational, and threat environments. The group sought to:

• Characterize experimental environments to be used in a design challenge. • Identify issues that must be addressed for the results of an evaluation in an experimental

environment to be meaningful and useful.

http://www.mitre.org/sites/default/files/pdf/11_4436.pdf

vii


• Define two or three vignettes, which include high-level assumptions about the technical, operational, and threat environments. A vignette can be used in multiple experiments or demonstrations; when used, it is made more specific, including for example specific adversary tactics, techniques, and procedures (TTPs).

This track focused on the question of how to define a “design challenge” for cyber resilience. The group

• Identified three major types of experimental environment: conceptual, table-top, and integration. Of these, integration experiments or demonstrations are the most convincing, but also require the most resources. Conceptual experiments and table-top exercises play an important role in determining whether an integration experiment is viable.

• Defined an incremental approach to experimentation and identified issues that must be addressed for the results of an evaluation in an experimental environment to be meaningful and useful.

• Defined two vignettes, which include high-level assumptions about the technical, operational, and threat environments. These vignettes can be used in multiple experiments or demonstrations; when used, they can be made more specific, including for example specific adversary tactics, techniques, and procedures (TTPs).


Table of Contents Overview 1

Executive Summary ................................................................................................................... iii

Track 1: Applying Cyber Resilience to Space ........................................................................... iii

Track 2: Resilient Critical Infrastructures – Applying Cyber Resilience and Overcoming the Barriers to Acceptance .................................................................................................... iv

Track 3: Cyber Resiliency and its Roles in the Systems Engineering Lifecycle ........................ v

Track 4: Designing a Cyber Resiliency Challenge for Integration and Demonstration ........... vi

1 Introduction ............................................................................................................................. 1

2 Keynotes 2

2.1 Attack Techniques: Gary Gagnon .................................................................................... 2

2.2 Enabling Secure and Resilient Infrastructures in the Public and Private Sectors: Brigadier General Gregory Touhill .................................................................................. 2

2.3 Managing Cyber Resilience in Financial Market Infrastructures: Ken Buckley ............. 2

2.4 Cyber Tabletop Exercises & Lessons Learned For Resiliency: David Dumas ............... 3

3 Applying Cyber Resilience to Space ....................................................................................... 4

Chair: 4

3.1 Goals ................................................................................................................................ 4

3.2 Context ............................................................................................................................. 4

3.3 Information and discussion .............................................................................................. 4

3.3.1 Future Horizons ......................................................................................................... 4

3.3.2 Applying Cyber Resilience to Space Capabilities ..................................................... 5

3.3.3 Cyber Enhanced Space Operations Tiger Team ........................................................ 5

3.3.4 Space Cyber: an Aerospace Perspective .................................................................... 5

3.3.5 AFRL Spacecraft Cyber Resiliency Project Cyber S&T in the Space Domain ........ 6

3.4 Discussion ........................................................................................................................ 6

3.5 Summary .......................................................................................................................... 8

3.5.1 Barriers ....................................................................................................................... 8

3.5.2 Path Forward .............................................................................................................. 8

3.5.3 Recommendations ...................................................................................................... 9

4 Resilient Critical Infrastructures: Applying Cyber Resilience and Overcoming the Barriers to Acceptance .......................................................................................................................... 0

4.1 Goals ................................................................................................................................ 0

4.2 Context ............................................................................................................................. 0

i


4.3 Discussion ........................................................................................................................ 1

4.3.1 Session Design ........................................................................................................... 1

4.3.2 Formation of Companies............................................................................................ 2

4.3.3 Common Themes for Resilience ................................................................................ 2

4.3.4 Barriers to Acceptance ............................................................................................... 3

4.3.4.1 Political Barriers ................................................................................................ 3

4.3.4.2 Operational Barriers .......................................................................................... 3

4.3.4.3 Economic Barriers ............................................................................................. 4

4.3.4.4 Technical Barriers .............................................................................................. 4

4.4 Summary .......................................................................................................................... 5

4.4.1 Path Forward .............................................................................................................. 5

4.5 Conclusions ...................................................................................................................... 5

5 Cyber Resiliency and its Roles in the Systems Engineering Lifecycle ................................... 7

Chairs: ......................................................................................................................................... 7

5.1 Goals ................................................................................................................................ 7

5.2 Context ............................................................................................................................. 7

5.2.1 Context: System Security Engineering ...................................................................... 7

5.2.2 Context: Cyber Resiliency ......................................................................................... 8

5.3 Discussion ...................................................................................................................... 12

5.3.1 Old Problems Reoccurring ....................................................................................... 12

5.3.2 Seniors Lack an Understanding of the Difference between Traditional Cyber Security and Cyber Resiliency ................................................................................. 13

5.3.3 Need to Move from a Compliance Mindset to a Risk Management Mindset ......... 13

5.3.4 Need for a Cyber Resiliency Center of Excellence. ................................................ 13

5.4 Recommendations for Moving Forward ........................................................................ 13

5.4.1 Need for Cyber Resiliency Requirements ................................................................ 13

5.4.1.1 Common Set of Terms ..................................................................................... 14

5.4.1.2 Roles and Responsibilities ............................................................................... 14

5.4.1.3 Resiliency Metrics ........................................................................................... 14

5.4.1.4 Cyber Resiliency Controls ............................................................................... 15

5.4.1.5 Educate Senior Officials on Difference between Traditional Cyber Security and Cyber Resiliency ........................................................................................................ 15

5.4.1.6 Balance between Traditional Cyber Security and Cyber Resiliency ............... 15

5.4.1.7 Post-Exploit CONOPS .................................................................................... 16

ii


5.4.1.8 Incorporation of Resiliency in Major Government System Undertakings ...... 16

5.5 Summary ........................................................................................................................ 16

6 Designing a Cyber Resilience Challenge for Integration and Demonstration ...................... 17

6.1 Goals .............................................................................................................................. 17

6.2 Context ........................................................................................................................... 17

6.3 Discussion ...................................................................................................................... 18

6.3.1 Experimentation ....................................................................................................... 18

6.3.2 Experimentation Approach ...................................................................................... 19

6.3.3 Defining Usable Vignettes ....................................................................................... 20

6.3.3.1 Architecture ..................................................................................................... 20

6.3.3.2 Threats ............................................................................................................. 21

6.3.3.3 Solutions .......................................................................................................... 21

6.3.3.4 Measures of Effectiveness ............................................................................... 22

6.3.4 Representative Vignettes ......................................................................................... 22

6.3.4.1 Vignette: Attack against an Enterprise System ............................................... 22

6.3.4.2 Vignette: Attack against a Transactional System ............................................ 23

6.3.4.3 Proto-Vignette: Attack against a Data Storehouse .......................................... 24

6.4 Summary ........................................................................................................................ 24

7 References ............................................................................................................................. 24

8 Acronyms .............................................................................................................................. 26

iii


List of Figures Figure 1. Spectrum of DHS Critical Infrastructure Areas, Mapped by General Qualities of Open vs. Controlled Access and Service-oriented vs. Material-oriented ................................................. 1

Figure 2. Aspects of the System Engineering Life Cycle ............................................................... 8

Figure 3. MITRE Cyber Resiliency Engineering Framework (CREF) .......................................... 9

Figure 4. Cyber Attack Life Cycle .................................................................................................. 9

iv


List of Tables Table 1. Cyber Attack Lifecycle Stages - Description ................................................................... 9

Table 2. Effects a Defender Can Have On an Adversary ............................................................. 10

Table 3. Effects of the Resiliency Techniques on the Adversary at each Stage of the Cyber Attack Lifecycle ............................................................................................................................ 11

1


1 Introduction The Fourth Annual Secure and Resilient Cyber Architectures Invitational was a two-day event designed to bring the community together to work collectively in order to accelerate recognition and adoption of Cyber Security Resilience. The purpose of the Invitational was to bring the community together to produce community cooperation around the theme of Moving Cyber Resilience into Practice. The bulk of the first day was intended to set the theme for the event. This was done by the four keynote presentations. The keynote speakers included representatives from MITRE, DHS, the Federal Reserve Board, and Verizon. These talks are summarized in Section 2. The end of day 1 and all of day 2 consisted of facilitated working groups in four tracks. Sections 3 – 6 are each dedicated to a separate track:

• Track 1: Applying Cyber Resilience to Space • Track 2: Resilient Critical Infrastructures – Applying Cyber Resilience and Overcoming

the Barriers to Acceptance • Track 3: Cyber Resiliency and its Roles in the Systems Engineering Lifecycle • Track 4: Designing a Cyber Resiliency Challenge for Integration and Demonstration

2


2 Keynotes The first day started with four keynote speakers: Gary Gagnon set the stage with a discussion of MITRE’s Adversarial Tactics, Techniques and Common Knowledge database. Following Gagnon’s discussion were Brigadier General Gregory Touhill speaking about public and private infrastructures and Ken Buckley speaking about the financial market infrastructure. The fourth keynote was delivered by David Dumas and focused on cyber tabletop exercises and the resulting resiliency lessons.

2.1 Attack Techniques: Gary Gagnon

Gary Gagnon, Senior Vice President and Chief Security Officer of The MITRE Corporation, provided information on MITRE’s Adversarial Tactics, Techniques and Common Knowledge database that is focused on providing higher fidelity descriptions of adversary behaviors in the post exploit phases of the cyber attack life cycle without referencing specific tools. When viewed through this lens, there are only 70 techniques adversaries are using and so the problem becomes tractable.

2.2 Enabling Secure and Resilient Infrastructures in the Public and Private Sectors: Brigadier General Gregory Touhill

Brigadier General Gregory Touhill, DHS Deputy Assistant Secretary for Cyber Security Operations, spoke about security and resiliency in public and private sector infrastructures. He set the stage with the following three points:

• The adversary is not going to wait until it is convenient. He illustrated this with the vignette that he “got sworn in at 7 a.m. and at 7:10 we had Heartbleed.”

• Culture trumps strategy, so you have to change culture. He has seen cultures change over the last few years but not enough yet.

• Cyber security is less a technical issue, and more a risk issue. Board room vs. server room – problems should be elevated to the board room not just dismissed as a server room issue.

2.3 Managing Cyber Resilience in Financial Market Infrastructures: Ken Buckley

Ken Buckley, Associate Director, Division of Reserve Bank Operations and Payment Systems, Board of Governors of the Federal Reserve, spoke on managing cyber resiliency in financial market infrastructures. The Federal Reserve Board transfers on the order of trillions of dollars/day. This is all about risk management. He spoke about post 9-11 guidance that articulated sound practices to improve financial stability and reduce systemic risk. This guidance targets a two hour Recover Time Objective for both physical and cyber disruptions and an end-of-day resumption/completion of settlements. These cyber resiliency activities are industry wide, and the response to DDoS attacks is a good model for industry-government cooperation to provide effective deterrents. Systemic risks in financial organizations include:

• Recognized challenges in identifying, isolating and recovering from cyber attacks.

3


• Business continuity policies aimed at the continuation of operations. This is at odds with the need to shut down part or all of an IT system in order to prevent further damage.

• Actions require consultation with regulators, overseers, central banks and national authorities. This consultation can inhibit prompt action.

Security challenges in financial organizations include: • Complex trust models with external parties including multilateral contracts, business

process participants and information sources • Validating the external environment including information handling policies and

procedures and verifying the control environment of external parties • Trust in the internal control environment including authentication of external information

sources, protecting data at rest and in transit, and reducing the attack surface

This situation calls for changing the approach from “Defending Yesterday” to more “Intelligence Driven.” In the “Defending Yesterday” approach, everything must be patched and protected, selected resources are monitored and then defenders fight what gets through. In the “Intelligence Driven” approach there is a rebalancing of initiatives; valuable assets are patched and the focus is on detecting and responding more efficiently to use limited resources most effectively. Realizing that there is no way to guarantee against a determined adversary, this approach also prepares for the inevitable breach.

2.4 Cyber Tabletop Exercises & Lessons Learned For Resiliency: David Dumas

David Dumas, a senior network security engineer in Security Operations at Verizon, talked about cyber tabletop exercises and lessons learned for resiliency. Cyber Resiliency Tabletop exercises are important preparations for the inevitable attack. The key elements in setting up a tabletop exercise are, establishing the objectives, clarifying responsibilities and roles, and identifying ways to assess performance. NIST SP 800-84, Guide to Test, Training and Exercise Programs for IT Plans and Capabilities provides a solid foundation on which to base exercises. The four exercise scenarios discussed were: a privacy breach in the U.S. and internationally, a large malware infection, an APT attack with loss of intellectual property, and a loss of critical internal infrastructure. The results of the resiliency exercises identified four potential areas of process change and three potential areas for training and resources. In the area of process change, it is important to identify the most critical components and single points of failure, store an offline or paper copy of the contact lists for critical information, review business continuity plans for how they rely on network availability, and identify who would lead the response to a large incident for the enterprise and the initial state from which to restart. In the area of training and resources, it is important to begin cross-training in order to mitigate single points of failure in staff, determine alternate staffing for a large cyber disaster both inside and outside the enterprise, and consider retention bonuses for key personnel during the recovery of the business.

4


3 Applying Cyber Resilience to Space Chair: Roberta Ewart, Chief Scientist, Space and Missile Systems Center (SMC), Air Force Space Command, Los Angeles AFB (LAAFB), CA.

3.1 Goals

This session had several goals: • To share current state of the practice including updates to the 8500 series, focusing on a

risk management framework, not just on Department of Defense Information Assurance Certification and Accreditation Process (DIACAP) compliance.

• To gain an improved understanding of critical needs in relation to the state-of-the-art. • To identify development priorities and user considerations based on the above

information and understanding. • To develop and document community-vetted best practices for transition to Cyber

Enhanced Space Operations (CESO).

3.2 Context

The expected outcome from this session was a set of recommendations that would help close the gap between cyber resiliency and space systems. These recommendations can be used by the CESO Tiger Team and would identify common themes and challenges in bringing cyber resiliency to space systems. This session brought together individuals from Space and Missile System Center (SMC), Air Force Space Command (AFSPC), National Aeronautics and Space Administration (NASA), Lockheed Martin, The MITRE Corporation, Raytheon, Aerospace, and National Reconnaissance Office (NRO). Being one of the first opportunities for this diverse group to meet, the focus was on discovering common challenges instead of on a path forward.

3.3 Information and discussion

The breakout session started with several talks to provide the context. The first was given by Mark Maybury and provided the context of the environment and how it is changing. The next three talks were given by Roberta Ewart, Richard Boller and Joe Betser. These briefings spoke to the challenges of incorporating Cyber into the Space Domain.

3.3.1 Future Horizons

Mark Maybury’s presentation, Future Horizons, defined cyberspace as the interdependent network of information technology infrastructures, and includes the Internet, telecommunications networks, computer systems, and embedded processors, controllers, individuals, organizations and missions. He then discussed some predictive models – Moore’s Law (memory doubles every 18 months), Metcalf’s Law (the power of the network is exponentially related to the number of devices on the network), and Kurzweil’s Law (technology adoption rate is accelerating). Mark Maybury added his own law to relate the level of autonomy vs human control to the level of confidence or trust. As the level of autonomy in a device increases the amount of human trust in that devices goes from a low level to something that may be considered as “over-trusting.”

5


The main points of this talk were: • Government is no longer the driving force in cyber – that role has been transferred to the

banking and commercial sector. • Autonomy is going to help because it provides faster methods of dealing with the

adversary. • Autonomy can also hurt because the adversary can take advantage of autonomous

mechanisms. • The best supply chain managers are pharmaceuticals – this is a place to learn supply

chain risk management. • It should be easy to do the right thing.

3.3.2 Applying Cyber Resilience to Space Capabilities

Roberta Ewart’s presentation, Applying Cyber Resilience to Space Capabilities, provided background on new national security challenges in cyber operations, the Air Force cyber vision and long-term vision of AFSPC. This presentation was followed by a discussion on operational and doctrinal gaps in resilience.

3.3.3 Cyber Enhanced Space Operations Tiger Team

Richard Boller presented a briefing on the Cyber Enhanced Space Operations Tiger Team. The team’s objective is to determine what science and technology should be brought to bear to enhance future and possibly legacy space systems in the area of cyber. The discussion that followed this presentation included questions about how and with whom to share actionable threat intelligence, since this type of information is often classified. Questions raised in this presentation were:

• What and where is or should be the seam between space and cyber capabilities and operations?

• What space systems require cyber protection or could otherwise benefit from cyber technologies?

• What studies have previously been done? • What are the current and potential future cyber threats to space systems? • Where will AFSPC have to cross space and cyber responsibility boundaries to apply

cyber technologies to space systems, and what process/OPR will take responsibility for crossing boundaries?

• Should this be an on-going effort of AFSPC’s S&T Program?

3.3.4 Space Cyber: an Aerospace Perspective

Joe Betser presented the third topic: Space Cyber: An Aerospace Perspective. This presentation focused on the idea that information assurance is necessary for cyber resiliency but not sufficient. The traditional categories of information availability, integrity, confidentiality, authentication and non-repudiation must be expanded to include engineering technologies, practices and procedures to protect, detect, react, defend, prevent, restore and assure. In addition, the rest of cyber operations that include mission training, Concept of Operations, and

6


Tactics, Techniques and Procedures (TTPs) must be expanded in order for the mission to operate through an attack.

3.3.5 AFRL Spacecraft Cyber Resiliency Project Cyber S&T in the Space Domain

Roberta Ewart presented Major Calvin Roman’s briefing, AFRL Spacecraft Cyber Resiliency Project Cyber S&T in the Space Domain. This briefing focused on the trust issues for both hardware and software. The key areas were supply chain risk management, technologies and techniques for cyber hardening the spacecraft and its ecosystem, vulnerability assessments and security policy enforcement.

3.4 Discussion

Roberta Ewart used the construct of Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities and Policy, to frame observations from the three previous presentations. Doctrine is defined as goals and principles. The team made the following observations in the area of Doctrine:

• In order to operate through an attack, it has become a high priority to be able to tolerate network intrusions.

• The gap between space and cyber is similar to that between core command and cyber. • There are cases where it is helpful to have a gap between space and cyber capabilities and

operations.

Challenges identified in the area of Doctrine were: • Goals and principles need to cover the spectrum from cradle to grave for the system of

systems. • There is a need to share cyber information with international and commercial interests,

but this conflicts with doctrine principles – Interoperability is important. • There is a need for a resolution process on conflicts of risk.

Organization is defined as “how we build ourselves.” It was observed that resilience crosses organizational boundaries. Challenges identified in the area of organization were focused on disaggregation in the domains of space and cyber and were framed as questions.

• What and where should the seam between space and cyber capabilities and operations be?

• Where will AFSPC have to cross space and cyber responsibility boundaries to apply cyber technologies to space systems?

• What process or organization will take responsibility for crossing boundaries?

Training is necessary to conduct operations and promote resilience. In general, incorporating autonomous and scalable technologies that operate with humans on the loop and humans in the loop, needs experience that is on the order of years. This requires a large investment in training.

• Leadership needs training in the areas of threat attribution and evolution (e.g., what types of threats are relevant to which organizations and how are threats changing) and in defining operational resilience.

7


• Cyber risk management requires more sophisticated decisions, risk based decisions and machine-parameterized decisions.

• Automation can be used to jump start training and situational awareness. Challenges identified in the area of training were:

• The threat continues to move and evolve. • Where to use “Human on the Loop” vs. “Human in the Loop”? • How to automate where possible but leave the critical to human expertise/experience?

Within the area of Materiel, money is often a focus of solution, as well as a limiting factor. Autonomy is being pushed because of speed and money reduction. Challenges identified in the area of Materiel were:

• How the Risk Management Framework is changing the landscape with regard to legacy systems, waivers, tailoring and overlays, early lifecycle and supply chain.

• Trust must be built from untrustworthy components. • Resiliency can be in conflict with both the capability, as well as the cost of a system.

The observations and challenges in the area of Leadership overlapped with those in the area of Training. Incorporating resilience, like resolving any complex issue, requires the right blend of people. While automation is faster and may reduce some errors, humans add a level of human judgment and a higher semantic cognition than some machines. It was observed that one place to implement automation is continuous monitoring because of the amount of data and the ongoing nature of the job. Challenges identified in the area of Personnel were:

• Augmenting and improving the human decision making process. • New flexibility to interpret risk makes things more complex and increases the need for

expert personnel. • Bringing actionable engineering intelligence without compromising the security

classification. • Asymmetry of the challenge – the attacker has lots of time to prepare but the defender

doesn’t know until the exploit happens – this is a very short part of the lifecycle.

The area of Facilities focuses on the existing footprints and the need for different, or differently built, facilities. Three observations in the area of facilities were made:

• Solutions need to consider embedded cyber as well as networks, software, hardware and firmware.

• Ground segments are generally the most vulnerable – easier to get to and contain the command and control system – but the other segments are vulnerable as well.

• Outages can be the result of malicious activities.

8


The challenge identified in this area was the move to convert to more Internet Protocol related technologies. Not all policy can be adopted. Two observations with regard to policy are:

• Assistant Secretary of Defense for Research and Engineering (ASD R&E) Strategic Guidance1 now ranks cyber just below nuclear threats. The focus needs to move from countering terrorism and insurgence to cyber capabilities across joint operations.

• Policy is designed to address threats without specifying the threat. An example is complex password policy was designed to address a specific threat but no one had to know the specific threat to implement it.

The challenges identified in the area of policy were:

• A need to adopt common federal cyber security terminology so everyone is speaking the same language. NIST’s Risk Management Framework was suggested as it is already used by civil and intelligence communities.

• Incorporate the restore and evolve portions of resiliency to expand resilience beyond survivability.

• It’s hard to talk problems because not everyone has the correct clearance.

3.5 Summary

The results of this breakout section were divided into three areas: Barriers, Path Forward and Recommendations.

3.5.1 Barriers

Barriers to moving forward on cyber resiliency in space systems include the lack of a common language, conflicting policies and conflicting requirements. The lack of a common language is a problem for two reasons. First, without a common language it is difficult to frame the problem so that it is understood by everyone across all domains. Second, words mean different things in different domains. For example resilience can refer to the ability to operate through an attack but it can also refer to the evolvability and reusability of the components. Policies that limit with whom information can be shared come into conflict with the policies requiring that certain types of information be shared. The policies limiting information sharing also may make it difficult to bring the problem, in its full context, to the personnel with the greatest ability to solve the problem. Policies that require sharing may inadvertently provide too much information to the adversary. Similarly conflicting requirements (e.g., minimizing cost, making fully resilient and widely capable systems and enterprises) are impossible to meet simultaneously. Properly balancing these requirements requires deep knowledge across multiple domains.

3.5.2 Path Forward

There are three main thrusts to the path forward in closing the gap between cyber and space. They are:

1 See http://www.acq.osd.mil/chieftechnologist/publications/docs/ASD(R&E)_Strategic_Guidance_May_2014.pdf

9


• Better risk governance: Risk governance is important because cyber security is becoming necessary to a wide range of systems that were previously isolated from cyber issues, e.g., cars. Currently decision makers are not thinking of how to manage the risk in cyber dependent systems.

• Identifying mission dependencies and making segmentation decisions based on these dependencies: Not everything needs to be on the Internet or even networked together. Mission dependencies are critical in determining how to segment the environment.

• Identify resiliency tradeoffs: Resiliency Tradeoffs need to be evaluated at the Integrated Planning Process, Analysis of Alternatives, Science and Technology, and Operational Levels. The tradeoffs must be reevaluated and updated on a continuing basis as the threat evolves and mission needs change.

3.5.3 Recommendations

This was the first opportunity for representatives from across the domains and organizations to discuss the issues. There was much agreement on what the issues are and the need to continue meeting to work on the solutions space. There are four main recommendations:

• Decide on a common language. • Clarify mission dependencies and tradeoffs at all levels for consistent risk governance. • Implement segmentation – not everything needs to access the Internet. • Use the input from the breakout session for the CESO tiger team.

0


4 Resilient Critical Infrastructures: Applying Cyber Resilience and Overcoming the Barriers to Acceptance

Chair: Nick Multari, Pacific Northwest National Laboratory

4.1 Goals

This session had several goals: To reach a common understanding of terms in relation to critical infrastructures (CIs), define what the resilience would ideally bring to the environment for each CI examined, and examine the barriers to achieving resilience and the steps required to address the barriers. The team’s plan was to develop a consensus roadmap for realizing resilience in critical infrastructure that includes:

• Large research and development (R&D) challenges that we recommend as priorities for funding agencies

• Operational challenges that could be addressed by existing and future consortia, ISACs, and other federated groups

• Issues that require legal, policy, or regulatory development • Social and economic needs and impacts of resilience.

In order to achieve this roadmap, the group was divided into smaller teams, with each team developing an example CI that illustrated the need for resilience, the attributes of that resilience, and barriers to acceptance. The goal was for each team to drive toward identifying key advances that would be required for acceptance of resilience in their domain.

4.2 Context

Critical infrastructure (CI) is a cornerstone of healthy and safe communities. Critical infrastructures typically consist of enterprise-type cyber systems, physical components or industrial control systems, and interfaces between the two. This growing reliance on infrastructures and components that are vulnerable to and can be used to propagate cyber-attacks means cyber resilience concepts should be applied to CI environments.

One of the biggest issues is that most of the CIs in the U.S. are operated by non-government entities. However, many are regulated to some degree to make sure that they conform rigidly to requirements that lead to deterministic or at least bounded behaviors. Typically, deterministic behavior is desired since deviations from the expected can have catastrophic results.

While many CI owners understand and agree that there is a need for resilient systems, there are also many barriers to acceptance. These barriers include political, operational, economic, and technical issues. An example of such a barrier that is both technical and operational is the use of non-determinism by some resilient methodologies resulting in the addition of new risks into the system operations.

1


4.3 Discussion

4.3.1 Session Design

The CI breakout session provided an opportunity to flesh out four example companies representing a spectrum of CI types. The teams fleshed out their respective company’s specific mission and goals, cyber-physical components, threats, regulatory issues, the role of cyber resilience, and ultimately the barriers to acceptance of resilience in their domain area. Figure 1 is a projection of the 16 U.S. Department of Homeland Security (DHS)-defined CI domains into a 2D space in which the horizontal axis is a measure of “hardness” of what is delivered by the CI domain (services being on the “soft” end, and food/water being on the “hard” end), and the vertical axis is a measure of the degree of exposure that the underlying infrastructure has to the general public. These are not definitive or quantitative. They were meant to give a qualitative partitioning to the space of CI domains to drive the selection of four or five for development into scenarios that span a wide spectrum of CI types. At the planning stages, we provisionally selected communications, energy, transportation, manufacturing, and finance as the representative covering set. Other considerations for the final topics were the need to have all discussions in an unclassified setting, and the need to have multiple representatives in our session from each CI domain. The actual attendance of the breakout session determined the domains that were selected. The end result was four CIs selected, including communications, energy, transportation, and finance.

Figure 1. Spectrum of DHS Critical Infrastructure Areas, Mapped by General Qualities of Open vs.

Controlled Access and Service-oriented vs. Material-oriented

This qualitative mapping provided the basis for selecting CI domains (highlighted) that represent different areas of this space.

2


The scenarios developed by the breakout teams were the basis for identifying barriers to acceptance that are common to different CI types rather than those that are specific to the CI types. Scenarios were also the basis for identifying gaps that could be addressed by technical, operational, policy, social, or economic R&D. Finally, these scenarios provided a foundation for comparing the existing state of practice for resilience in CI types to the ideal future state.

4.3.2 Formation of Companies

The first step in this session was the creation of companies representing a cross range of CI domains. The created companies, their mission and scope are:

• Transportation: Worldwide Network of Transportation o Mission: multimodal transportation services for package delivery; focused on

normal packages, not oversized or hazardous o Scope: preferred provider of transportation services worldwide

• Communications: BigCom o Mission: worldwide voice and data telecom infrastructure and service provider;

international business headquartered in the U.S. o Scope: premier provider of telecom infrastructure and services

• Energy: Gotham Energy Services o Mission: generation and distribution, including network of pipes that bring in fuel

products to be distributed and sold o Scope: primary provider for large metropolitan area; customer base is 40%

industrial and 60% residential • Finance: American Trust Company

o Mission: custodian of U.S. securities; book entry trust agent of record o Scope: customer base is 60 exchanges, FX traders, commodities, future.

Captured 90% of the book entry market. The IT architecture is private, connected to customers and data services, requires 100% availability, and has a high expectation of integrity and confidentiality. Primary provides transactional services.

4.3.3 Common Themes for Resilience

The breakout teams summarized discussions during their “out-briefs” to the larger group and discussed a number of themes related to resilience. Two general conclusions emerged.

• The attributes of resilience are highly similar across the CI domains explored in the session.

• The implementation of resilience appears to be very individualized.

Each CI needs to continue operating even if only in a degraded mode. As a result, availability of service is a top priority. This, however, does not reduce the need for integrity and confidentiality. Integrity of data and sensing for situational awareness were considered highly important. Confidentiality needs depended upon the CI. Whenever customers were involved, the protection of those records was deemed important.

3


Means to achieve resilience varied with some commonalities. For example, redundancy, backups, and hot spares were common throughout the CI domains examined although potential vulnerabilities are present in each of these solutions. Another tactic used by some CI domains to improve availability was the isolation of critical resources when feasible. Removing a critical device from an externally accessible path was deemed important to be sure of its integrity and availability. Interestingly, diversity was not seen as a feasible solution due to its potential high cost in terms of acquisition and maintenance. A final means of achieving resiliency is adaptive systems. These systems have the capability to change in some manner based upon events occurring within the environment. It was noted that while most people think of adaptive systems as automated, it was agreed that adaptive response does not have to be automatic. In fact, in the context of this discussion, adaptive referred to any action that changes a system including manual as well as automated modifications. As such, the group agreed that adaptive systems can make the acquisition process easier in that it is easier to change a system than to procure a new one. Finally, there was some concern that an adaptive system could result in significant certification problems.

4.3.4 Barriers to Acceptance

4.3.4.1 Political Barriers

The various teams developed a number of political and regulatory issues that presented barriers to achieving resiliency in their specific CI. These included the following:

• Lack of standards for resilience implementations. As a result, no CI was sure whether their specific implementation would be acceptable to government as a whole and the regulatory body specifically. This resulted in uncertainty.

• Laws can also restrict what could be done to improve resiliency. This was specifically identified by the “multinational” companies. Their concern was that export controls could prevent the export of resiliency-improving technologies to other countries in which they had sites.

• Conversely, multinational corporations must also deal with non-U.S. laws, companies, and personnel. As a result, U.S.-driven policies and practices addressing resiliency may not apply, be practical, or be legal in other countries. Cultural or social issues may also arise.

• Lack of policy/authority for sharing technologies was identified by entities with close partner relationships. They stated that many resilience technologies need to operate not only in their environments but also those of their partners, vendors, and other that they interface with (e.g., Target).

• Concern was raised about antitrust laws. Legal clarity is needed to understand whether sharing of technologies could be seen as anticompetitive.

4.3.4.2 Operational Barriers

The teams identified a number of operational concerns that could also derail acceptance of resiliency technologies.

4


• Resilience may increase overall risk. A number of reasons for this counterintuitive barrier were identified. First, if new regulations are required and adopted to accommodate resiliency, new vulnerabilities and faulty interpretations may arise. In addition, many CI domains work with unregulated/uncontrolled non-CI companies. These relationships may arise as a result of business operations or be specific to the implementation of resiliency technologies. In either case, these companies may take risks with their environments that increase the risk to the regulated environment. Finally, to enable resiliency technologies to address risk, an enumeration of current risks is needed. However, this list of risks in itself is a risk as such a list identifies all vulnerabilities and is extremely sensitive.

• Many CI domains depend upon technologies beyond their boundary of control. An example is the global positioning system (GPS) that is used by a number of CI domains. As a result, although the CI may have taken measures to improve internal resiliency, these external systems may not have done likewise making the internal improvements of limited value. The same issue arises when one CI must interface with another that has also not taken steps to improve their resiliency.

• Resilient systems require trained, knowledgeable operational personnel to drive the resilient responses. As a result, the lack of such personnel can have detrimental effects.

4.3.4.3 Economic Barriers

A number of economic barriers were also identified. • The cost of resiliency is an unknown and potentially unacceptably high. This is

especially a concern for highly competitive or narrow margin domains. In addition, there is the question of who assumes the cost for resilience given the above political and operational concerns. No government agency or partner provides incentives to defray any of the costs. Since customers are not demanding it (and possibly not even understanding the need for resilience) and there is no competitive advantage enumerated (e.g., no return on investment (ROI) determination), it is seen as a non-competitive cost.

• Resiliency technologies are not free. To be resilient, resources are needed, including resources that would otherwise be used to provide services to customers.

• Costs associated with resilience must be justified based upon a mature risk management structure. Since none exists to date, cost justifications are difficult.

4.3.4.4 Technical Barriers

Finally a number of technical barriers were identified. • There are no reasonable commercial-off-the-shelf (COTS) solutions that provide for

resiliency. There are point solutions but none that holistically address the overall problem.

• Many solutions rely upon the restoration of a “gold copy” of the software. However, this is not a viable long-term solution because the gold copy can increase the attack surface by introducing another target, the verification of the gold copy is a constant requirement and a gold copy does nothing to protect the integrity of system operational data.

5


• Determining when resilient actions are needed requires analytics capabilities. This requires determining which systems are critical to the primary CI functions and to track changes that are occurring in real time, both of which are difficult to determine and track. In addition, there was concern that some of the analytics required may increase the probability of breaches in privacy.

4.4 Summary

Internal and external barriers exist across the spectrum of CI domains and they span the political, operational, economic, and technical realms. Policy is needed to clarify roles, expectations, oversight for resilience, and create incentives to help make the economic case close. Operational issues will require coordination across all interacting CIs and must also address partners, vendors, international laws and regulations, and export laws. All economic concerns must be resolved and COTS products made available for companies to voluntarily implement resilient technologies within their CI. Finally, technical concerns must also be addressed, including support of a robust R&D agenda to address the identified issues.

4.4.1 Path Forward

The group reached consensus on the following recommendations: • New international laws, standards, and agreements are needed to clarify how resilience

technologies can be used and shared. • CIs need to mutually arrive at resilience because of their interdependence. • Economic incentives need to be provided to motivate resilient operations. • COTS solutions for resilience need to be developed and made available in the

marketplace; R&D must continue in resiliency technologies. • A well-trained workforce needs to be developed and maintained.

4.5 Conclusions

This breakout session explored a sample of CI domains that had different underlying features (Figure 1). As expected, resilience has some common themes across CI.

• Availability of the services provided by each domain is a high priority because these domains support life and commerce in the U.S. In terms of resilience, this means that often availability is the dominant need. However, to achieve this in most cases confidentiality and integrity of data must also be preserved. The means for ensuring the availability, confidentiality and integrity aspects of security are often opposed to each other.

• There is a need for trusted, shared situational awareness. Data and analysis that support decision-making and resilient responses must be trusted and must be approached community-wide.

• Resilient responses must enable critical services to persist even when some parts of the system are compromised or lost because of the high premium on availability.

6


Therefore, the working definition of resilience for CIs is the extent to which a complex cyber system can continue to deliver critical services in the face of impediments. Impediments, in this case, are the collection of natural faults and actions of adversaries that drive systems toward a state of lost functionality.

Another major finding is that barriers to acceptance have some common themes. • CI domains do not currently have the means to absorb the cost of realizing resilience.

Overcoming this barrier will require government and private R&D investments that drive resilience technologies toward standardized, interoperable COTS and government-off-the-shelf solutions. The other main component of overcoming this barrier is the need for economic incentives around resilience.

• The interdependence of CI domains, leads to a situation where resilience in one domain depends on resilience in the other domains. This means that lack of resilience in any CI domain or the support services on which domains rely could jeopardize the resilience of all CI domains. Overcoming this barrier must be approached with a multi-domain shift toward resilience supported by cross-domain coordination and a cultural shift that instills the value of resilience into the full technology development cycle.

• There are significant trust and sharing concerns. In the highly competitive environments of most CI domains, sharing between entities eliminates competitive advantage, and in some cases is illegal. There are also trust concerns arising from reliance on non-owned components, services and geopolitical issues for multinational companies.

Policy changes at the national and international levels are required to overcome these barriers. To address the comment raised at the outset of our breakout session, we recommend a follow-up workshop held at the CI-level view to augment the industry-focused view that was explored in this meeting.

Resilience is a complex but essential ideal to enable persistence of critical U.S. infrastructure in the hostile global realities of networked systems. There is a need for continued cross-domain coordination, international and domestic policy developments, government and private science and technology investments, and cultural shifts to move from the current state to a more ideal resilient future.

7


5 Cyber Resiliency and its Roles in the Systems Engineering Lifecycle

Chairs: Ron Ross, National Institute of Standards and Technology (NIST), Richard Graubart, The MITRE Corporation

5.1 Goals

This track focused on cyber resiliency and how it should fit into the broader concept of system security engineering. The track needed participants to:

• Achieve a common understanding of what is system security engineering and what is cyber resiliency and why it is needed.

• Provide recommendations regarding what changes are needed to incorporate resiliency into the system lifecycle.

5.2 Context

Two technical areas, system security engineering and cyber resiliency, form the basis for this breakout session. Therefore, the primary focus of the track was two-fold:

• Identify where and how to better incorporate cyber resiliency into the systems engineering life cycle processes.

• Recommend some concrete actions that could be taken to better achieve this outcome over the next year.

5.2.1 Context: System Security Engineering

The first step to achieving a common understanding of system security engineering, is a common definition of what is a system. A system, for the purposes of this track, is a combination of interacting technology, physical, human, process, and natural elements organized to achieve a purpose [ISO15288:2008]. An alternative, more technical view is that a system is combination of interacting elements2 organized to achieve one or more stated purposes.

System security engineering is in essence a specific implementation of systems engineering. Thus, the first step is to understand the various aspects of the system engineering life cycle. That is reflected in Figure 2 below, one of the slides presented during the track.

2 A system element is defined 1) as a member of a set of elements that constitute a system, 2) may be a system in and of itself, or 3) realized by hardware, software, data, humans, processes, procedures, facilities, materials, and naturally occurring entities, or any combination.

8


Figure 2. Aspects of the System Engineering Life Cycle

Draft NIST SP 800-160, System Security Engineering, discusses how systems security engineering should be incorporated into the process shown in Figure 2. Part of the level setting in this track involved introducing participants to this document. The document has multiple purposes:

• Provide a comprehensive statement of the systems security engineering discipline. • Foster a common mindset to deliver security for any system, regardless of its scope, size,

complexity, or system life cycle stage. • Advance the field of systems security engineering by promulgating it as a discipline that

can be applied and studied. • Demonstrate how systems security engineering processes can be effectively integrated

into systems engineering processes. • Serve as a basis for the development of educational and training programs.

5.2.2 Context: Cyber Resiliency

While there are many possible definitions of the term “cyber resiliency,” for the purposes of this track we used the following definition: Cyber resiliency is the ability of missions to continue in the presence of adversary activities or attacks on the supporting cyber resources the mission needs to function. Setting the context also involved explaining the reason behind the cyber resiliency concept – the nature of today’s adversaries is such that it is simply not realistic to assume that tradition boundary based defenses (e.g., firewalls) can keep the adversaries out of the organization.

As part of the context, the MITRE Cyber Resiliency Engineering Framework (CREF) [CREF11], was re-introduced; see Figure 3 below.

9


Figure 3. MITRE Cyber Resiliency Engineering Framework (CREF)

The discussion of cyber resiliency included a discussion of the concept of the Cyber Attack Lifecycle (CAL) which define stages an adversary goes through to achieve their objectives. The CAL3 provides a framework for understanding and analyzing how distinct adversary activities contribute to an attack. Understanding the various stages of the CAL, shown in Figure 4, allows organizations to better understand that there are various places in the CAL where they can counter adversary actions. Table 1 describes the CAL stages of a malware-based cyber attack.

Figure 4. Cyber Attack Life Cycle

Table 1. Cyber Attack Lifecycle Stages - Description

Stage Description

Recon The adversary identifies a target and develops intelligence to inform attack activities. The adversary develops a plan to achieve desired objectives.

Weaponize The adversary develops or acquires an exploit (e.g., a “0-day”), places it in a form that can be delivered to and executed on the target device, computer, or network.

Deliver The exploit is delivered to the target system. (e.g., tailored malware is included in a spearphishing email attachment or compromised components inserted in the supply chain are integrated into a target network).

Exploit The initial attack on the target is executed. (e.g., A vulnerability is exploited, and malware is installed on an initial target system).

3 There are multiple versions of the Cyber Attack Lifecycle, also referred to as the Cyber Kill Chain. The one depicted here is consistent with what is described as a cyber campaign in NIST Special Publication 800-30 Rev 1.

10


Stage Description

Control The adversary employs mechanisms to manage the initial targets, perform internal reconnaissance, and compromise additional targets.

Execute The adversary executes the plan and achieves desired objectives (e.g., exfiltration of sensitive information, corruption of mission-critical data, fabrication of mission or business data, degradation or denial of mission-critical services).

Maintain The adversary ensures a sustained, covert presence on compromised devices, systems, or networks. To do so, the adversary may erase indications of prior presence or activities.

Another part of the cyber resiliency context discussion focused on the various effects a defender can have on the adversary. The description of the effects are shown in Table 2 below.4 The italicized text indicates the general category of the effect on the adversary. The Bolded text indicates the key word that describes the specific effect.

Table 2. Effects a Defender Can Have On an Adversary

Defender Actions

Effect on Adversary

Divert the adversary’s efforts Deter The adversary ceases or suspends activities, or redirects activities toward different targets Misdirect The adversary reveals capabilities, intent, targeting, TTPs, or strategy, w/o achieving intended effects.

Obviate the adversary’s efforts so that their resources cannot be applied or are wasted. Prevent The adversary’s efforts are wasted, as no intended effects can be achieved. Preempt The adversary’s resources cannot be applied and/or the adversary cannot perform activities

Impede the adversary so that only by investing more resources or taking additional actions can they achieve their goals Degrade The adversary achieves some but not all of the intended effects, or achieves all intended effects but only

after taking additional actions. Delay The adversary achieves the intended effects, but may not achieve them within the intended time period.

Detect the adversary’s activities so that the adversary’s ability to act stealthily is removed Limit the adversary’s effectiveness Contain The value of the activity to the adversary, in terms of achieving the adversary’s goals, is reduced. Curtail The time period during which the adversary’s activities have their intended effects is limited. Recover The adversary fails to retain mission impairment due to recovery of the capability to perform key mission

operations. Expunge The adversary loses a capability for some period of time

Expose the adversary so that they lose the advantage, as defenders are better prepared Analyze The adversary loses the advantages of uncertainty, confusion, and doubt; the defender can recognize

adversary TTPs Publicize The adversary loses the advantage of surprise; the adversary’s ability to compromise one organization’s

systems to attack another organization is impeded.

Finally, linking all of this back to resiliency, Table 3 depicts the effects each of the fourteen resilience techniques can have on the adversary and maps that across the various stages of the CAL.

4 Tables 2 and 3 are included as used by the track participants. However, more recent versions are available in the Cyber Resiliency Engineering Aid available at http://www.mitre.org/sites/default/files/publications/pr-15-1334-cyber-resiliency-engineering-aid-framework-update.pdf.

11


Table 3. Effects of the Resiliency Techniques on the Adversary at each Stage of the Cyber Attack Lifecycle5

Cyber Resiliency Technique

Recon Weaponize Deliver Exploit Control Execute Maintain

Adaptive Response Contain Curtail

Impede Curtail Prevent Recover

Contain Curtail

Curtail Impede Recover

Contain Curtail

Analytic Monitoring

Detect Analyze

Prevent Detect Analyze

Detect Analyze

Detect Analyze

Detect Analyze

Coordinated Defense

Delay Impede Detect Impede

Detect Impede

Deception Prevent Impede Divert

Deceive Detect

Analyze

Deter Impede Deceive Analyze

Deter Divert

Deceive Analyze

Deter Divert

Deceive Analyze

Deter Divert

Deceive Detect

Analyze

Deter Divert

Deceive Degrade Detect

Analyze

Deter Deceive Detect

Analyze

Diversity Impede Impede Prevent Contain

Degrade Prevent

Degrade Contain Recover

Recover Degrade Contain Recover

Dynamic Positioning

Divert Detect Curtail

Prevent Divert

Detect Impede Curtail

Expunge Recover


Expunge Recover


Expunge Recover

Dynamic Representation

Obviate Analyze

Obviate Detect

Analyze Expunge

Detect Analyze Recover

Obviate Analyze Expunge

Non-Persistence Impede Prevent Curtail Expunge

Curtail Expunge

Curtail Curtail Expunge

Privilege Restriction

Impede Prevent Degrade Delay

Contain

Prevent Degrade Delay

Contain


Contain


Contain Realignment Impede Prevent

Impede Impede Prevent

Impede Prevent Impede

Prevent Impede

Prevent Impede

Redundancy Degrade Curtail

Recover

Segmentation Contain Impede Impede Contain

Impede Delay

Contain Detect

Impede Delay

Contain Detect

Recover

Impede Delay

Contain

Substantiated Integrity

Prevent Detect

Detect Curtail

Expunge

Curtail Recover Expunge

Detect Curtail

Expunge Unpredictability Deter

Delay Impede Delay Detect

Delay Delay Detect

Detect

5 Effects in italics are second-order rather than primary effects, and are typically associated with a few approaches to implementing the technique.

12


5.3 Discussion

Having established a common understanding of cyber resiliency and system security engineering, the group was able to discuss what needs to be done differently to incorporate cyber resiliency in the system security lifecycle. The discussion was somewhat free form and far ranging, and the participants included personnel from DoD, Military Services, NIST, MITRE, various other FFRDCs and national labs, and FRB. The following are some of the observations from the group as a result of the discussion and presentations.

5.3.1 Old Problems Reoccurring

Many of the issues with regards to resiliency and systems engineering are the same as those for cyber security and systems engineering. These fell into four categories. The first category is the disconnect between the view of “C suite”, the engineers, and the operators. Because the operators are at the pointy end of the spear with regards to dealing with the attacker, they need maximum flexibility regarding tools to respond to attacks. However, they often were not involved in the overall system security engineering decisions. On the other side of this disconnect, seniors may not necessarily always appreciate the nature and demands cyber security and cyber resiliency may impose.

The second category concerned the ongoing nature of traditional cyber security and cyber resiliency. To be effective both must be applied and reassessed across the entire system life cycle. This is not always clear to seniors; they sometimes question why a security problem is still present if they focused on the security problem at some early stage of the life-cycle. To be effective the importance of traditional cyber security and cyber resiliency must be articulated and pushed at the highest levels of the organization. Cyber security and cyber resiliency are two of many competing issues (e.g., cost) that an organization must deal with, but if the importance and the nature of the issue are not articulated for seniors they are not given sufficient weight.

Thirdly, as alluded to in the previous paragraph, cyber resiliency is just one element of the larger system engineering and system security engineering processes. Neither cyber security nor cyber resiliency is an end in and of itself. It is there to support some larger engineering and mission purpose.

The final category concerns the way perspective on resilience varies depending on role. Communication between the C suite, operators and system engineers is often suboptimal in several ways. Those in the C suite are going to have a different view than those designing a system and still a different view from those operators trying to secure the system on a daily basis. Those involved in developing systems (and setting system requirements) are often not communicating with those who are the users of those systems; so the systems are not necessarily developed with the security needs of the operators in mind. Finally, it is often the case that neither the system developers nor the operators have an appreciation for the mission concerns of those in the C suite.

13


5.3.2 Seniors Lack an Understanding of the Difference between Traditional Cyber Security and Cyber Resiliency

The seniors are not security professionals and often have only a tangential interest in cyber security. They are not aware of the changing security threat landscape. Their view may be that they are funding various traditional cyber security capabilities (e.g., firewalls and anti-virus technology deployment), why is that not enough?

As a result seniors can miss the basic point that the security of the system may be breached by today’s adversaries. Moreover, they may not fully appreciate how dependent they are on cyber components working properly and the true harm that an adversary can cause. They likely have not considered the question of “the system has failed, now what do we do?”

5.3.3 Need to Move from a Compliance Mindset to a Risk Management Mindset

Systems are too complex, and environments too varied to apply standard, cookie-cutter, check the box solutions. This is especially true regarding cyber resiliency where cutting edge mitigations are applied in an attempt to thwart advanced adversaries. In such an environment, there is insufficient history of applying mitigations to constitute a body of standard practice that is required for standardized compliance approaches.

Moreover, from a cost perspective, it is not practical or feasible to apply all possible security resiliency mitigations to address the threat. What is required is balancing the nature of the threat, opportunities, likelihood that an adversary can exploit weaknesses in the defenses, and the probable impact of the adversaries attack succeeding.

The baselines in documents such as NIST SP 800-53 [NIST13] and CNSSI 1253 [CNSSI14] provide a starting point for selecting security mitigations. This being said, decision makers must understand that they are just starting points, that they must select security mitigations (both cyber resiliency and traditional cyber security ones) on a risk management basis.

5.3.4 Need for a Cyber Resiliency Center of Excellence.

It was clear that the participants believe that cyber resiliency is a critical need for combating the APT. It was also clear from the track that various organizations have attempted to address different aspects of the problem. Some have looked at developing frameworks, some have developed requirements, etc. Having some center of cyber resiliency excellence that provides current thinking, recommended frameworks, mitigations, and cyber resiliency requirements would be a critical means of enhancing cyber resiliency.

5.4 Recommendations for Moving Forward

Based on the presentations and discussions the working group produced an action plan highlighting what needs to be done, in order to better integrate resiliency into the system security engineering process.

5.4.1 Need for Cyber Resiliency Requirements

One of the concerns noted during the track was the lack of existing, repeatable requirements language for addressing cyber resiliency. There are controls in NIST SP 800-53 that address

14


cyber resiliency. These generally are at too high a level of abstraction to be useful for those individuals responsible for creating RFPs. It was noted that there have been various proprietary efforts to generate some cyber resiliency RFP requirements language. None of these are publically available and some are tailored to the specific needs of organizations or particular systems. What is missing and what the working group proposed is the creation of a publically available set of cyber resiliency requirements that could be employed, with some tailoring, to a broad set of systems. Such cyber resiliency requirements could be fed into the stakeholder requirements definition process and the acquisition (RFP) process.

5.4.1.1 Common Set of Terms

Currently no common, established set of terminology exists for cyber resiliency. Nor is there a common, authoritative cyber resiliency framework. There are various resiliency related frameworks (MITRE, Aerospace, SEI) each with their own terminology. Because these are all produced by private organizations, they lack the authoritative nature of one produced by the government. The working group advocated the development of an authoritative, common terminology and frameworks for cyber resiliency. The proposed cyber resiliency appendix to NIST SP 800-160 might be an appropriate location for it.

5.4.1.2 Roles and Responsibilities

The working group noted a need to identify and articulate the various roles and responsibilities with regards to cyber resiliency. The first step is to identify the key cyber resiliency stakeholders with regards to cyber resilience. These include, but are not limited to, the system engineer, system security engineer, risk management executive, program managers, system operators, and those responsible for acquisition and procurement. The next step is to articulate the cyber resiliency responsibilities for each of these roles. The final step is to identify any dependencies between the various roles. For example, acquisition personnel who are responsible for purchasing components that provide (or in some instances fail to provide) the necessary security, and resiliency functions are likely to have a direct impact on those individuals charged with defending the system. The proposed cyber resiliency appendix of NIST SP 800-160 would be the optimum location to place cyber resiliency roles and responsibilities.

5.4.1.3 Resiliency Metrics

The ability to measure the relative resilience of a system is important, especially from the perspective of those developing and designing systems. Having some means to measure the relative resilience would also be helpful in selecting products and requirements. It is very challenging to create a generalized way of assessing resilience. Usually resilience is relative to operating scenarios and use cases. Resiliency is more than simply measuring how long after an incident a system takes to return to normal operating parameters (even if it was so simple there is the question of whether one measures resilience post incursion or post detection). Because advanced cyber threats are persistent, it is reasonable to expect that the adversary would work to keep an organization from recovering from an attack; the more capable the threat actor, the longer time and more effort required to recover. Thus it would be useful to have some approved means of assessing threats which organizations could consider in planning for resiliency. There are some such measures that have already been considered ([DSB13], [CyberPrep09]) but no

15


cross government, authoritative ones have been established to date. Creating such measures would be a step forward.

5.4.1.4 Cyber Resiliency Controls

To ensure cyber resiliency is implemented in government systems it is necessary to ensure that security controls supporting cyber resiliency are in NIST SP 800-53. This document already has over 150 controls that are cyber resiliency related, (see [CyberRel13]) but it is likely that additional controls may need to be added over time. Not all of the existing 150+ “resiliency” controls should be required for any given system. Unfortunately there is currently no authoritative process for selecting which controls are needed under which circumstances. Having a cyber resiliency overlay document to reference would be very useful to organizations. It should include some guidance regarding which resiliency controls, or class of resiliency controls, to reference under which circumstances.

5.4.1.5 Educate Senior Officials on Difference between Traditional Cyber Security and Cyber Resiliency

As noted earlier in this section, senior officials (C-suite occupants) often are unaware of the purpose or need for cyber resiliency. They likely believe that their continual funding of security mechanisms such as firewall, anti-virus software, and authentication (e.g., PKI) mechanisms will adequately safeguard their systems. They are often unaware of their systems’ vulnerability to penetration by the APT; and that such traditional cyber security mechanisms are necessary, but not sufficient, to ensure security operations. These officials need to be educated on the nature of the threat, the distinction between traditional cyber security defenses and cyber resiliency. Specifically, these officials need to prepare to address the problem “the security of the system has failed, now what do we do?” To help address this problem the working group recommended that a series of articles be written and placed in appropriate periodicals to help raise awareness of the problem for such seniors.

5.4.1.6 Balance between Traditional Cyber Security and Cyber Resiliency

Cyber resiliency is not intended as a replacement for traditional cyber security, it is intended to supplement it. Organizations only have a limited amount of money to devote to securing their systems. What is unclear is how much money and effort should be placed on traditional cyber security (i.e., focus on keeping out the adversaries), and how much should be placed on cyber resiliency (i.e., focus on responding to system compromises by adversaries that have penetrated the protections of traditional security measures). Placing too much on former (i.e., traditional cyber security) with the goal of closing every possible avenue of entrée into a system is clearly wasteful. But placing too much on the latter (i.e., cyber resiliency) at the expense of the former could result in an insufficient security foundation on which to build cyber resiliency. Obviously there is no single answer or simple formula for achieving this, but general guidance regarding the balance between these two important objectives can still be produced. The security community in general, and those individuals responsible for building secure systems in particular, would benefit by the develop of such guidance explaining the relationship between the two activities.

16


5.4.1.7 Post-Exploit CONOPS

A security Concept of Operations (CONOPS) is written to explain how the organization intends to operate the system from the perspective of information security. Generally those are written with the traditional cyber security perspective (e.g., keep adversaries out), and focus on explaining how to employ the various security mechanisms in support of operations. What is needed is a supplement to the CONOPS reflecting the cyber resiliency perspective. Such a CONOPS supplement would focus on actions that an organization would take to “fight through a security incident.” It would include procedures to employ during an attack (e.g., use of tools/applications that the adversary would not have seen before). The supplement would also provide guidance on the use of mechanisms (e.g., honeynets, moving target defense tools) that are intended to implement the various cyber resiliency techniques.

5.4.1.8 Incorporation of Resiliency in Major Government System Undertakings

There are multiple government systems, and systems of systems, being built. Some of these are major efforts (e.g., Joint Information Environment – JIE) that are intended to support multiple government organizations over a period of years, if not decades. Given the importance, prominence, and anticipated long life of such systems, the working group noted that it is essential that such systems incorporate cyber resiliency.

5.5 Summary

The following is a summary of the eight recommendations that came out of the session. • Create a publically releasable set of cyber resiliency requirements that could be employed

(with some tailoring) to a broad set of systems. • Establish a common set of terminology for cyber resiliency. The most logical place for

this at the current time is NIST SP 800-160. • Identify and articulate the various roles and responsibilities with regards to cyber

resiliency. The most logical place at the current time for this is in NIST SP 800-160. • Develop an authoritative cross government means of assessing threats which

organizations could consider when planning for resiliency. • Develop a cyber resiliency overlay to NIST SP 800-53. • Develop guidance explaining the relationship between the traditional cyber security

defenses and cyber resiliency, to give senior officials a better understanding. • Supplement traditional CONOPS with a cyber resiliency perspective. • Incorporate resiliency in major government system undertakings.

17


6 Designing a Cyber Resilience Challenge for Integration and Demonstration

Chair: Harriet G. Goldman, The MITRE Corporation

6.1 Goals

This track focused on the question of how to define a “design challenge” for cyber resilience. The group characterized a “design challenge” as a statement of a problem that can be used as the basis for evaluating some combination of products, emerging technologies, architectural principles, and operating procedures. A cyber resilience problem is the problem of how to achieve specific cyber resilience goals and objectives6, in the context of a set of assumptions about the technical, operational, and threat environments. The group sought to:

• Characterize experimental environments to be used in a design challenge. • Identify issues that must be addressed for the results of an evaluation in an experimental

environment to be meaningful and useful. • Define two or three vignettes, which include high-level assumptions about the technical,

operational, and threat environments. A vignette can be used in multiple experiments or demonstrations; when used, it is made more specific, including for example specific adversary tactics, techniques, and procedures (TTPs).

6.2 Context

A cyber resiliency design challenge can be used to produce one or more demonstrations, analyses, or experiments. Such efforts can serve one or more purposes:

• Demonstrate and evaluate the benefits of individual solutions and combinations of solutions, for improving cyber resiliency, cyber security, and/or cyber defensibility.

• Identify operational and architectural assumptions underlying solutions. • Identify and resolve barriers to interoperability and synergy. • Identify capability gaps in existing and emerging solutions.

Cyber resiliency design challenges are intended to complement DARPA Grand Challenge work, which focuses on autonomous service resiliency. While future cyber resiliency design challenges may be able to leverage vignettes defined by DARPA Grand Challenges, the group recommended that cyber resiliency challenges include rather than avoid hands-on, operator interaction. In the future, a cyber resiliency design challenge might be able to use a “ground truth” data set from a DARPA Grand Challenge, thus enabling participants in the cyber resiliency challenge to compare architectures. However, the group affirmed that cyber resiliency design challenges should focus on ways to evolve existing architectures, rather than define “de novo” or “blue-sky” architectures.

6 See (Bodeau & Graubart, Cyber Resiliency Engineering Framework (MTR110237, PR 11-4436), 2011) (Bodeau & Graubart, Cyber Resiliency Assessment: Enabling Architectural Improvement (MTR 120407, PR 12-3795), 2013) for more information on cyber resiliency goals and objectives.

18


6.3 Discussion

The group’s discussion centered on three topics: • What types of experimentation are most useful? What are the benefits and challenges of

experimentation? • What approach should be taken to experimentation? • What must be identified in order to define usable vignettes for experimentation?

The group then defined two vignettes, and sketched a third.

6.3.1 Experimentation

The group identified three major types of experimental environments: conceptual, table-top, and integration. (The group noted that simulation can also provide an experimental environment [EVALNET2012], but viewed simulation as out of scope for its discussions.) Integration could involve a laboratory environment, or an experimentation environment such as DETER.7 Ultimately, an integration environment could take the form of a “plug-fest,” but such an effort would be late in a process of developing experiments. Each environment presents some challenges. For example, a table-top experiment requires an understanding of the operational environment, including roles, responsibilities, processes, and governance; table-top experiments are most effective when representatives from the real world are included. The group identified several key challenges and potential benefits of integration experiments:

• Benefit: The experience gained during integration can help to characterize relationships among alternative solutions (products, architectures, emerging technologies, procedures, and combinations thereof).

• Benefit: Experimentation can push participants toward increased technical integration via common / exposed interfaces, improved interoperability, and common data formats. These in turn will support more effective systems engineering.

• Benefit: Analysis of a series of integration experiments can result in a description of which cyber resiliency techniques are most effective under different conditions, as well as identification of representative issues of complexity, stability, and scalability.

• Challenge: It is unrealistic to expect turn-key solutions – integration will be needed. The level of effort for an integration experiment can be difficult to predict.

• Challenge: Integration efforts can run into issues of complexity, which also raise issues of stability and scalability. There is a risk that proponents of specific solutions will blame such issues on other solutions, rather than using the integration experiment to identify challenges they must address.

• Challenge: An integration experiment cannot be expected to address all possible engineering trade-offs. However, trade-offs from each experiment can be documented.

In addition, those engaged in an experiment or demonstration of any type must consider issues of fidelity: How well does the experimental or demonstration environment reflect a true operational environment, in which loads or stresses can vary? Finally, the design of the experiment or

7 See http://deter-project.org/

19


demonstration should either allow for the representation of partial results (e.g., partial achievement of desired resilience outcomes, partial success for the adversary), or explicitly state that partial results cannot be represented.

6.3.2 Experimentation Approach

The group defined an incremental approach to experimenting, which could be applied to any of these experimental environments. The increments are described in four experiments listed below. In each experiment, the resilience would be assessed on how well the system or mission achieved the resilience goals8:

• Experiment 1: How resilient is the system/mission? In this experiment, the system is constructed and operated using a well-defined reference architecture. This experiment produces a baseline.

• Experiment 2: How resilient is the system/mission, once we use existing capabilities with resilience in mind? For example, cyber defenders could analyze data provided by existing non-security sensors (e.g., network performance monitors), or could define courses of action using resource reallocation mechanisms designed for performance optimization to ensure continuity of mission-critical functions.

• Experiment 3: How resilient is the system/mission, if we add a couple of products and tweak the architecture? For example, the experiment could look at segregating some legacy components, defending that enclave at the boundary, and adding other tools for parts of the system not in that enclave.

• Experiment 4: How resilient is the system/mission, if we add some bleeding-edge capabilities?

At each stage, those engaged in the experiment or demonstration should: • Characterize the relationships among the solutions included in the experiment. Solutions

can be mutually interfering or compatible. Compatible solutions can be non-interfering, interoperable, or synergistic. Solutions can be compatible at different architectural layers (e.g., data, service, and network). Compatibility can be expected to depend on the maturity of the solution space.

• Identify areas of lessons-learned, beyond vignette-specific measures of effectiveness (MOEs). These can include:

o What are the associated costs? � Procurement and maintenance. Costs can be estimated in dollar terms, but

estimates should be clear regarding which elements are required vs. which are optional. Estimates should reflect the total cost of ownership, thus avoiding (for example) the illusion of low cost associated with free and open source software (FOSS).

8 See (Bodeau & Graubart, Cyber Resiliency Engineering Framework (MTR110237, PR 11-4436), 2011) (Bodeau & Graubart, Cyber Resiliency Assessment: Enabling Architectural Improvement (MTR 120407, PR 12-3795), 2013) for more information on cyber resiliency goals.

20


� Operational impacts. These impacts include staffing requirements and procedural complexity. Operational impacts should be identified in at least three areas: mission operations, cyber defense, and system/network administration.

� Architectural changes and technical ramifications. These can include, for example, requirements for additional storage, processing, and bandwidth.

o What are the resiliency benefits? o What issues would need to be resolved, and what POET (political, operational,

economic, and technical) factors would need to be addressed?

6.3.3 Defining Usable Vignettes

The group noted several considerations for defining vignettes that could be used in experimentation or demonstration. These are the architecture, threats, solutions, and measures of effectiveness (MOEs).

6.3.3.1 Architecture

The architecture represented in the experimental or demonstration environment needs to be representative of real-world, mission systems (e.g., use real-world apps and data stores). Those engaged in the experiment or demonstration need to define one or more reference architectures in which some parts can be changed, while others (e.g., representing legacy investments or systems owned by external organizations such as critical infrastructure providers) are fixed. In the context of experimentation or demonstration, a reference architecture is not “to-be” or ideal; it is simply representative. The design challenge will populate each part of the reference architecture with something that exists, or identify gaps. The group observed that the definition of reference architectures will be sensitive to sectors (e.g., defense, Government, critical infrastructure sectors). When defining cloud architectures, the group recommended following the NIST model [NIST, 2011]. The group also recommended applying a relevant reference architecture to evolving architectures, rather than recommending specific reference architecture (e.g., one based on JIE). In defining a reference architecture, those engaged in experimentation or demonstration will need to think about the relationship between detection and resilience. However, the group recommended against using an experiment or demonstration to attempt to define an improved detection architecture. The group noted that experiments can not focus solely on the network layer, particularly for detection. In addition, those engaged in experimentation or demonstration will need to think about the role of sensors and actuators, asking (1) What should be monitored? (2) How can the architecture leverage sensors that aren’t security-specific (e.g., performance monitoring)? (3) What can and should be changed? And (4) What could serve as actuators for such changes (e.g., resource reallocation mechanisms)?

21


6.3.3.2 Threats

In defining vignettes, some classes or examples of threats are needed, to enable “before and after” experiments and demonstrations. However, the group observed that in developing vignettes, the value of describing the threats in detail is limited. Cyber resiliency is needed, whether the threat is known to be malicious or whether it could be attributed to error (e.g., network connectivity taken out by a backhoe). Even for adversary attacks, some vignettes could ignore potential consequences of adversary actions (e.g., exfiltration) if not of significant concern to the missions or organizations being represented (e.g., missions in which data is transient, but must not be maliciously modified). Thus, rather than describing adversary characteristics, the group recommended that vignettes center around the potential adverse effects of greatest concern. Vignettes should reflect use cases in which some capabilities are identified as needing resilience properties, independent of the types of adversary activities that might be involved (for example, a resilient high-integrity database). For each vignette, a few desired resilience properties and corresponding measures of effectiveness (MOEs) such as the minimum level of performance for specific services should be defined. The effects of concern can then be situated in the context of a campaign or intrusion set, rather than of a specific activity within a campaign. The group recommended defining several representative campaigns, with different goals and corresponding cyber effects.9 Goals to be considered should include:

• Deny mission capabilities at will (cyber effects: degradation and interruption of service) • Degrade the integrity of specific data, in such a way that the attack effects cannot be

easily detected (cyber effects: modification and fabrication of mission data) • Stage adversary command and control (C2), for example as part of an attack that subverts

a supply chain (cyber effects: modification and fabrication of system components) • Exfiltrated data, with subsequent effects on business or mission (cyber effects:

unauthorized use and interception)

6.3.3.3 Solutions

While being representative of real-world environments, an experimental architecture would be intended to bring together existing, leading-edge, and (maybe) bleeding-edge technologies together to achieve overall (mission-level or system-level) resilience, rather than looking at the resilience of any specific component. The group noted that leading-edge technology may be too hard for some participants in a design challenge, and may not be easily integrated into legacy. One goal of experimentation might be to look at how to provide resilience for legacy systems (e.g., by changing front ends). The group recommended that any reference architecture should avoid dependence on any single point solution.

9 The cyber effects – degradation, interruption, modification, fabrication, unauthorized use, and interception – were defined by Musman et al. (Musman, Temin, Tanner, Fox, & Pridemore, 2009). These effects were discussed in a paper provided as read-ahead to participants (Bodeau & Graubart, Characterizing Effects on the Cyber Adversary: A Vocabulary for Analysis and Assessment (MTR 130432, PR 13-4173), 2013).

22


Solutions should be mapped to the cyber resiliency objectives they help to achieve. The group recommended focusing on solutions that help achieve the objectives: Constrain, Reconstitute, and Understand. Solutions can also be mapped to attack techniques represented in the experiment or demonstration, describing the intended and observed effects of each solution on each attack technique.

6.3.3.4 Measures of Effectiveness

MOEs are evidence about claims of effects. In the context of a cyber resiliency experiment, MOEs can be designed to answer one or more of the following questions:

• How effectively does the mission or system achieve its resilience objectives? The group recommended that MOEs should focus on the objectives: Constrain, Reconstitute, and Understand. These objectives should be translated into outcomes, in terms of the vignette used in the experiment.

• What effects do the cyber resilience solutions have on the adversary, and how strong are those effects? MOEs should map to intended effects on adversary activities, such as degrade or curtail [CharEff 2013].

Evidence can take many forms, e.g., • Mission performance measurements. • Occurrence or nonoccurrence of some event (e.g., exfiltration, data corruption). • Increased time between adversary accomplishments of activities in adversary’s planned

sequence.

Evidence must be situated in terms of assumptions about and constraints on the evaluation environment (e.g., the baseline behavior of the system, the expertise of the Red Team). The group observed that MOEs or other evidence obtained from experiments are not ends in themselves. The value of evidence or MOEs is to support reasoning about improvement.

6.3.4 Representative Vignettes

The group defined two representative vignettes, and sketched a third. A vignette consists of • Stage-setting: Describe the overarching story about mission or business operations and

the adversary campaign that will be represented. • Action: Describe the actions that will be represented, using the structure of a multi-act

play. • Resilience approaches: Identify possible technologies or processes to be evaluated

experimentally.

6.3.4.1 Vignette: Attack against an Enterprise System

A contractor in the defense industrial base (DIB) possesses a set of critical information desired by a competitor. The adversary has significant sponsorship, tools, staff, and desire to gather data. The adversary has been previously thwarted by security controls. The adversary has developed a plan to use ineffective training to dupe staff to defeat access controls and move around within the target company’s enterprise information and communications technology (ICT).

23


The play is structured in four acts: • Act 1: Initial target selection – Use LinkedIn to identify staff • Act 2: Morph malware to be undetectable by the company’s anti-virus software, put it in

a pdf announcing company picnic, and send it to identified staff via LinkedIn. The result is that malware is on staff’s home systems.

• Act 3: Use captured credentials from malware to access enterprise systems; apply mapped attack techniques. The adversary needs to get past protections on the virtual private network (VPN), and then can start moving laterally, and escalating privileges.

• Act 4: Exfiltrate data. The experiment must explain why existing security practices (e.g., data loss prevention or DLP) do not suffice.

Resilience approaches that might be explored in experiments include virtualized remote access so that no presence on the enterprise system includes the untrustworthy home system, and technologies to make data unusable outside the enterprise perimeter.

6.3.4.2 Vignette: Attack against a Transactional System

The attack is against the financial sector. Russian crime organizations are sending phishing emails to bank customers and using banks to distribute payments to partners or mules (i.e., someone who transfers stolen money for others, in person, via a courier service or electronically). The play is structured in four acts:

• Act 1: Threat actors steal credentials, e.g., sending phishing emails via Google to bank customers.

• Act 2: Credentials are distributed to criminal actors. • Act 3: Criminal actors coordinate use of credentials by mules to withdraw money or

transfer money to target accounts. • Act 4: Financial and reputational impacts are experienced by the bank.

Several resilience approaches can be explored in the context of the play, e.g., • Act 1: Make it harder to steal the credentials in the first place (e.g., using secure

browser). • Act 3: Delay transaction commitments so that coordinated actions can be detected using

big data analytics. This is an example of Analytic Monitoring applied to business data, and supports the objectives of Constrain and Understand.

• Act 3: Insert an out-of-band challenge for a sampling of transactions. This is an example of Diversity combined with Unpredictability, with a strong procedural component, and also supports the objectives of Constrain and Understand. This could involve changes to workflow, and could require policy changes.

• Act 4: Forensic analysis, redress via law enforcement. Forensic analysis is an example of Analytic Monitoring, and supports the Understand objective.

• Act 4: Seek redress via law enforcement. Cooperation with an external entity involves Coordinated Defense, and supports the Recover objective.

24


6.3.4.3 Proto-Vignette: Attack against a Data Storehouse

This involves attacks against data “in the cloud.” The group identified two variants: a private cloud and a third-party provider. The vignette could be adapted from one that has been previously used in demonstrations [CyberRel 10].

6.4 Summary

This track focused on the question of how to define a “design challenge” for cyber resilience. The group

• Characterized a “design challenge” as a statement of a problem that can be used as the basis for evaluating some combination of products, emerging technologies, architectural principles, and operating procedures.

• Identified three major types of experimental environment: conceptual, table-top, and integration. Of these, integration experiments or demonstrations are the most convincing, but also require the most resources. Conceptual experiments and table-top exercises play an important role in determining whether an integration experiment is viable.

• Defined an incremental approach to experimentation and identified issues that must be addressed for the results of an evaluation in an experimental environment to be meaningful and useful.

• Define two vignettes, which include high-level assumptions about the technical, operational, and threat environments. These vignettes can be used in multiple experiments or demonstrations; when used, they can be made more specific, including for example specific adversary tactics, techniques, and procedures (TTPs).

7 References [CNSSI14] Security Categorization and Control Selection for National Security Systems, Committee For National Security Systems Instruction 1253, March 2014. [CharEff13] D. Bodeau and R. Graubart, "Characterizing Effects on the Cyber Adversary: A Vocabulary for Analysis and Assessment (MTR 130432, PR 13-4173)," November 2013. [Online]. Available: http://www.mitre.org/sites/default/files/publications/characterizing-effects-cyber-adversary-13-4173.pdf. [CyberAssess13] D. Bodeau and R. Graubart, "Cyber Resiliency Assessment: Enabling Architectural Improvement (MTR 120407, PR 12-3795)," May 2013. [Online]. Available: http://www.mitre.org/sites/default/files/pdf/12_3795.pdf. [CREF11] D. Bodeau and R. Graubart, "Cyber Resiliency Engineering Framework, Version 1.0," September 2011.

http://www.mitre.org/sites/default/files/publications/characterizing-effects-cyber-adversary-13-4173.pdf

http://www.mitre.org/sites/default/files/publications/characterizing-effects-cyber-adversary-13-4173.pdf

http://www.mitre.org/sites/default/files/pdf/12_3795.pdf

25


[CyberRel13] Graubart, R., Bodeau, D., Cyber Resiliency and NIST SP 800-53R4, MITRE Technical Report 130531, MITRE Corporation, September 2013, Public Release 13-4047. [DSB13] Defense Science Board, Rask Force Report, Resilient Military Systems and the Advanced Cyber Threat, January 2013. [CyberRel10] H. Goldman, R. McQuaid and J. Picciotto, "Cyber Resilience for Mission Assurance," in Proceedings of the 2010 IEEE Conference on Technologies for Homeland Security, Waltham, MA, 2010. [CyberPrep09] Graubart, R., Bodeau, D, and Fabius Greene, J., Improving Cyber Security and Mission Assurance Via Cyber Preparedness (Cyber Prep) Levels, MITRE Corporation, 2009, Public Release 09-4656. [ISO15288:2008] International Organization for Standardization/International Electrotechnical Commission, ISO/IEC 15288, IEEE Std 15288-2008 Systems and Software Engineering—Systems Life Cycle Processes, 2008. [EvalImp09] S. Musman, A. Temin, M. Tanner, D. Fox and B. Pridemore, "Evaluating the Impact of Cyber Attacks on Missions (PR Case No. 09-4577)," 2009. [Online]. Available: http://198.49.146.10/work/tech_papers/2010/09_4577/09_4577.pdf. [NIST13] National Institute of Standards and Technology Special Publication 800-53, Revision 4, Security and Privacy Controls for Federal Information Systems and Organizations, April 2013. [NIST11] NIST, "The NIST Definition of Cloud Computing, NIST SP 800-145," September 2011. [Online]. Available: http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf. [NIST14] National Institute of Standards and Technology Special Publication 800-160 (Initial Public Draft), System Security Engineering, An Integrated Approach to Building Trustworthy Resilient System, May 2014.

26


[EvalNet2012] S. Hassell, P. Beraud, A. Cruz, G. Ganga, S. Martin, J. Toennies, P. Vazquez, G. Wright, D. Gomez, F. Pietryka, N. Srivastava, T. Hester, D. Hyde and B. Mastropietro, "Evaluating Network Cyber Resiliency Methods using Cyber Threat, Vulnerability, and Defense Modeling and Simulation," 2012. [CyberAssess13] D. Bodeau and R. Graubart, "Cyber Resiliency Assessment: Enabling Architectural Improvement (MTR 120407, PR 12-3795)," May 2013. [Online]. Available: http://www.mitre.org/sites/default/files/pdf/12_3795.pdf.

8 Acronyms AFRL Air Force Research Laboratory AFSPC Air Force Space Command APT Advanced Persistent Threat CAL Cyber Attack Lifecycle CESO Cyber Enhanced Space Operations CI Critical Infrastructure CNSSI Committee on National Security Systems Instruction CONOPS Concept of Operations COTS Commercial Off-The-Shelf CREF Cyber Resiliency Engineering Framework DDoS Distributed Denial-of-Service DHS Department of Homeland Security DIACAP Department of Defense Information Assurance Certification and

Accreditation Process DLP Data Loss Prevention DoD Department of Defense FFRDC Federally Funded Research and Development Center FOSS Free and Open Source Software FRB Federal Reserve Board GPS Global Positioning System ISAC Information Sharing and Analysis Center IT Information Technology JIE Joint Information Environment MOE Measure of Effectiveness NASA National Aeronautics and Space Administration NRO National Reconnaissance Office NIST National Institute of Standards and Technology PKI Public Key Infrastructure POET Political, Operational, Economic, and Technical R&D Research and Development RFP Request for Proposal RMF Risk Management Framework

27


ROI Return on Investment S&T Science and Technology SMC Space and Missile System Center SP Special Publication TTPs Tactics, Techniques, and Procedures VPN Virtual Private Network

fourth annual secure and resilient cyber …...this report was compiled from sessions at the mitre...

Documents