controlled chaos · @swagitda_ “chaos isn’t a pit. chaos is a ladder.” ― petyr baelish,...

CONTROLLED CHAOSThe Inevitable Marriage of DevOps & Security

Kelly Shortridge (@swagitda_) Velocity Berlin 2019

@swagitda_

Hi, I’m Kelly

2

@swagitda_

“Chaos isn’t a pit. Chaos is a ladder.”

― Petyr Baelish, Game of Thrones

3

@swagitda_

Infosec has a choice: marry DevOps or be rendered impotent & irrelevant

4

@swagitda_

Infosec won’t survive in a silo. It must be embedded in software delivery.

5

@swagitda_

DevOps can learn to carve its own path to secure software delivery

@swagitda_

How can controlling chaos create a marriage of infosec and DevOps?

7

@swagitda_

1. Chaos Theory

2. Time to D.I.E.

3. A Phoenix Rises

8

Chaos Theory

@swagitda_

Chaos engineering = continual experimentation to test resilience

@swagitda_

“Things will fail” naturally extends into “things will be pwned”

11

@swagitda_

Security failure is when security controls don’t operate as intended

12

@swagitda_

What are the principles of chaotic security engineering?

13

@swagitda_

1. Expect that security controls will fail & prepare accordingly

14

@swagitda_

2. Don’t try to avoid incidents – hone your ability to respond to them

15

@swagitda_

Game days: like planned firedrills

16

@swagitda_

Prioritize security game days based on potential business impacts

17

@swagitda_

Decision trees: start at target asset, work back to easiest attacker paths

18

@swagitda_

Determine the attacker’s least-cost path (hint: it doesn’t involve 0day)

19

@swagitda_

Your goal is to raise the cost of attack, ideally beginning at design

20

Time to D.I.E.

@swagitda_

We need a model promoting qualitiesthat make systems more secure

22

@swagitda_

Enter the D.I.E. model by Sounil Yu: Distributed, Immutable, Ephemeral

23

@swagitda_

Distributed: multiple systems supporting the same overarching goal

24

@swagitda_

Distributed infrastructure reduces risk of DoS attacks by design

25

@swagitda_

A service mesh is like an on-demand VPN at the application level

26

@swagitda_

Attackers are forced to escalate privileges to access the iptables layer

27

@swagitda_

Immutable: infrastructure that doesn’t change after it’s deployed

28

@swagitda_

Immutable infra is more secure by design – ban shell access entirely

29

@swagitda_

Patching is no longer a nightmare with version-controlled images

@swagitda_

Ephemeral: infrastructure with a very short lifespan (dies after a task)

31

@swagitda_

Ephemerality creates uncertainty for attackers (persistence = nightmare)

32

@swagitda_

Installing a rootkit on a resource that dies in minutes is a waste of effort

33

@swagitda_

Optimizing for D.I.E. reduces risk by design & supports resilience

34

A Phoenix Rises

@swagitda_

Begin with “dumb” testing before moving to “fancy” testing

36

@swagitda_

D.I.E.ing is an art, like everything else

@swagitda_

Controlling Chaos: Distributed

38

@swagitda_

Distributed is mostly covered by the existing repertoire of chaos eng tools

39

@swagitda_

Repurpose these tools, but make attackers the source of failure

40

@swagitda_

Multi-region services present a fun opportunity to mess with attackers

41

@swagitda_

Shuffle IP blocks regularly to change attackers’ lateral movement game

42

@swagitda_

Test: inject failure into your service mesh to test authentication controls

43

@swagitda_

Controlling Chaos: Immutable

44

@swagitda_

Immutable infra is like a phoenix – it disappears & comes back a lot

45

@swagitda_

Volatile environments with continually moving parts raise the cost of attack

46

@swagitda_

Create rules like, “If there’s ever a write to disk, crash the node”

47

@swagitda_

Attackers must stay in-memory, which hopefully makes them cry

48

@swagitda_

Bonus: disallowing all local IO improves service reliability

49

@swagitda_

Metasploit Meterpreter + webshell:Touch passwords.txt & kaboom

50

@swagitda_

Build your Docker images with a garbage-filled “bamboozle layer”

51

@swagitda_

Mark garbage files as “unreadable” to craft enticing bait for attackers

52

@swagitda_

A potential goal: architect immutability turtles all the way down

53

@swagitda_

Test: inject attempts at writing to disk to ensure detection & reversion

54

@swagitda_

Treat changes to disk by adversaries similarly to failing disks: mercy kill

55

@swagitda_

Controlling Chaos: Ephemeral

56

@swagitda_

Most infosec bugs are stated-related – get rid of state, get rid of bugs

57

@swagitda_

Reverse uptime: longer host uptime adds greater security risk

58

@swagitda_

Test: change API tokens & test if services still accept old tokens

59

@swagitda_

Test: retrograde libraries, containers, other resources in CI/CD pipelines

60

@swagitda_

Test: inject hashes of old pieces of data to ensure no data persistence

61

@swagitda_

Leverage lessons from toll fraud –cloud billing becomes security signal

62

@swagitda_

Test: exfil TBs or run a cryptominerto inform billing spike detection

63

Conclusion

@swagitda_

Chaos/resilience are natural homes for infosec & represent its future.

65

@swagitda_

The future of infosec involves unified responsibility & accountability.

66

@swagitda_

Security can be innovative and fuel the engine of business as well.

67

@swagitda_

“You must have chaos within you to give birth to a dancing star.”

― Friedrich Nietzsche

68

@swagitda_

@swagitda_

/in/kellyshortridge

kelly@greywire.net

69