firewall on demand multidomain - internet2 · 4/9/2014 · bgp flowspec bgp flowspec was...

Jeffrey Haas JUNIPER

Firewall on Demand Multidomain S E C U R I T Y V I A B G P F L O W S P E C & A W E B P L A T F O R M

Leonidas Poulopoulos GRNET NOC

Wayne Routly DANTE

Internet2 Global Summit, Apr 9 2014 Firewall on Demand Multidomain

S E C U R I T Y V I A B G P F L O W S P E C & A W E B P L A T F O R M

Firewall on Demand

L e o n i d a s P o u l o p o u l o s

l e o p o u l @ n o c . g r n e t . g r

G R N E T N O C

( @ l e o p o u l )


GRNET NOC

Internet2 Global Summit, Apr 9 2014 Firewall on Demand Multidomain 3

Staff: 15

Network: 120 devices (40 routers/80 switches)

Juniper-based network

Presence: 90 cities

Clients: ~100

Upstream: GÉANT

UPSTREAMNREN

IX

DDoS attack traffic consumes

network capacity

DDoS attack launched from

compromised systems (bots)

DDoS attack targets

applications and services

Victim

DDoS Illustrated


DDoS facts

<1 1 3 10 17 24 40 49

100

60 60

309

0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 1 0 1 1 1 2 1 3 1 4

400 Gbps

Firewall on Demand Multidomain 5 Internet2 Global Summit, Apr 9 2014

Source: Arbor Networks Inc. & Cloudflare

Staying alive…


acls, firewall filters

RTBH

BGP flowspec

BGP FLOWSPEC IETF AND JUNIPER ROADMAP

Jeffrey Haas <[email protected]>

8 Copyright © 2014 Juniper Networks, Inc. www.juniper.net

BGP FLOWSPEC

BGP Flowspec was originally defined in RFC 5575 and has been

part of JUNOS since version 7.3. It permits layer 4 (TCP and

UDP) firewall filters to be distributed in BGP on both a intra-

domain and inter-domain basis.

Flowspec was originally defined to assist in mitigation of DDoS

attacks. Deployments may use native configuration to distribute

the filters. Several DDoS mitigation environments will generate

the filters in support of their detection and mitigation tools.


CURRENT IETF WORK

draft-ietf-idr-bgp-flowspec-oid

Formally permits IBGP origination of BGP flowspec routes without

requiring a longest-match for validation. In practice, operators

have been using policy knobs to permit similar behaviors for non-

eBGP originated flowsec.

draft-haas-idr-flowspec-redirect-rt-bis

Clarifies some issues in RFC 5575 for the “Redirect to VRF” Route-

Target. As currently documented, it’s not possible to have a fully

compatible BGP Flowspec implementation.


CURRENT IETF WORK

draft-ietf-idr-flowspec-redirect-ip adds some exciting features to

BGP flowspec:

Permit redirection of traffic to a specific IP address rather than

requiring tunneling via VRF.

Permit the copying of traffic in a similar fashion.

Some issues with the feature encoding and precedence of rules

are being worked out currently. New draft expected soon.

draft-ietf-idr-flow-spec-v6

Provide for support for IPv6 in flowspec. Necessary changes

include:

(Limited) Support for Next Header.

Flow Label support

Ambiguous case of Traffic Class with regard to ECN still under debate.


JUNIPER ROADMAP

15.1 – Flowspec ISSU/NSR support, draft-oid validation rules

15.2 (tentative) – Redirect-IP

Future: IPv6 Flowspec support


INTO THE REALM OF SPECULATIVE FICTION…

BGP Flowspec provides a convenient encoding mechanism to

permit Layer3+ traffic filters be distributed. Future facing work,

such as Software Defined Networking (SDN), Service

Chaining/Network Function Virtualization or Interface to the

Routing System (I2RS) may be able to leverage flowspec as a

mechanism to distribute custom forwarding behaviors.

BGP community flow vs. RTBH vs. ACLs


• Distributed across the

network • Closer to the source • Fine-grained even on

core/backbone networks • Multidomain easy

propagation towards the upstream via BGP

• Easy automation & integration

ACLS

• Flowspec: enhancement

of RTBH • Does not affect all traffic

to victim • Less coarse • More actions • Separate NLRI

BGP RTHB

Firewall on Demand


GRANULARITY: Per-flow level

ACTION: Drop, rate-limit, redirect

SPEED: 1-2 orders of magnitude quicker

EFFICIENCY: closer to the source, multi-domain

AUTOMATION: integration with other systems

MANAGEABILITY: status tracking, web interface

NEED FOR BETTER TOOLS TO MITIGATE TRANSIENT ATTACKS

FoD Architecture


User Interface

Django MVCLong Polling

(Gevent)

Job Queue (Celery/Beanstalk)

Caching Layer(Memcached)

Network Config to XML proxy (nxpy)

Python NETCONF client(ncclient)

NETCONFeBGP

eBGP

iBGP

iBGP

Shibboleth

• https://code.grnet.gr/projects/flowspy • http://flowspy.readthedocs.org

OPEN SOURCE

https://fod.grnet.gr

FoD Screenshots


…more during demo

How it works – Single domain


• Customer’s NOC logs in web tool (shibboleth) & describes flows and actions

• Destination validated against customer’s IP space

• A dedicated router is configured (NETCONF) to advertise the route via BGP flowspec

• Dynamic firewall filters are implemented on all routers

• Attack is mitigated upon entrance

• End of attack: Removal via the tool, or auto-expire

Web

NETCONF

eBGP

iBGP

UPSTREAM

GRNET

Client Client

IX

FoD

GRNET FoD usage examples


2.5years 20Tbytes 100rules 40users 20peers

What now? Idea!


BGP is by nature MULTIDOMAIN

Deploy FoD in a MULTIDOMAIN

Environment

GÉANT

and its peering NRENs

connect • communicate •

collaborate

Firewall on Demand – A

Multi-Domain

Implementation

Wayne Routly

Security Manager

DANTE

connect • communicate • collaborate

GÉANT : Who What How

21

Pan-European Network

…..Transit Network….ISP

30 Physical Pops

50,000 km network

infrastructure on 44 routes

100Gb/s

100s TB of Data

15+ Million IPs

100+ Workstations

Truly Global (50 million users)

10,000 institutions

Interconnects

European NRENs - 40

Commercial & Commodity Traffic


Today

Little bit of DDoS on the side…..

NTP, DNS, SMTP……. Amplification Attacks

2k DDoS Events (183 pm)

298 vs 929 ….. 1k in 2014, average 300

22


Today

DDoS Events – CyNet

Target: The University of Cyprus (www.ucy.ac.cy)

Port Ranges: 0, 2070 and 3475

Multiple Source IP’s and source AS’s.

Attack peak: Over 13G over 1G link

23

http://www.ucy.ac.cy/


Destination ports for 194.42.x.x

Date Seen Dst Port Flows (%) Packets (%) Bytes (%)

2013-09-02 04:58 2070 47268(37.8) 144.2 M(32.7) 182.4 G(35.3)

2013-09-02 04:58 0 46315(37.1) 260.0 M(59.0) 295.4 G(57.1)

2013-09-02 04:58 3475 29714(23.8) 31.3 M( 7.1) 39.2 G( 7.6)

2013-09-02 04:58 771 1348( 1.1) 4.3 M( 1.0) 243.6 M( 0.0)

2013-09-02 04:58 769 145( 0.1) 516000( 0.1) 29.0 M( 0.0)

2013-09-02 04:58 2816 55( 0.0) 199500( 0.0) 16.7 M( 0.0)

2013-09-02 04:58 1024 30( 0.0) 114500( 0.0) 6.4 M( 0.0)

Destination AS 3268 Traffic

Date Seen Dst IP Addr Flows (%) Packets (%) Bytes (%)

2013-09-02 04:58 194.42.x.x 124919(97.2) 440.6 M(99.2) 517.4 G(99.5)

2013-09-02 04:59 82.116.x.x 129( 0.1) 143000( 0.0) 154.3 M( 0.0)

2013-09-02 05:00 194.42.x.x 128( 0.1) 244000( 0.1) 12.3 M( 0.0)

2013-09-02 04:59 194.42.x.x 114( 0.1) 57000( 0.0) 10.5 M( 0.0)

2013-09-02 04:59 82.116.x.x 90( 0.1) 239500( 0.1) 311.4 M( 0.1) 2013-09-02 04:59 194.42.x.x 81( 0.1) 40500( 0.0) 8.7 M( 0.0)

Today

DDoS Events – CyNet [2]

24


Date first seen Duration Proto Dst IP

Addr Flows(%) Packets(%) Bytes(%) pps bps bpp 2013-03-13 09:34:05.770 5701.654 any 194.177.211.102 35531(

7.8) 36.1 M(11.3) 53.5 G(11.9) 6330 75.0 M 1481 2013-03-13 09:34:05.782 5701.975 any 194.177.211.100 34632(

7.6) 35.6 M(11.1) 52.6 G(11.7) 6236 73.8 M 1478 2013-03-13 09:33:46.665 5720.961 any 194.177.211.101 34469(

7.6) 35.3 M(11.1) 52.2 G(11.6) 6164 72.9 M 1478 2013-03-13 09:33:14.456 5797.618

any 194.63.239.233 49621(11.0) 31.8 M(10.0) 44.3 G(

9.9) 5478 61.1 M 1394 2013-03-13 09:33:17.612 5753.346

any 194.63.239.234 48220(10.6) 27.1 M( 8.5) 36.7 G(

8.2) 4705 51.1 M 1356 2013-03-13 09:33:12.442 5791.562 any 194.63.239.237 39278(

8.7) 26.1 M( 8.2) 36.5 G( 8.1) 4503 50.4 M 1400 2013-03-13 09:33:11.553 5800.394 any 194.63.239.232 42260(

9.3) 26.1 M( 8.2) 36.4 G( 8.1) 4495 50.2 M 1394 2013-03-13 09:33:16.562 5794.656

any 194.63.239.231 46109(10.2) 26.6 M( 8.3) 36.1 G(

8.0) 4593 49.8 M 1356 2013-03-13 09:33:15.479 5755.473 any 194.63.239.238 44189(

9.8) 26.1 M( 8.2) 35.3 G( 7.9) 4527 49.1 M 1356 2013-03-13 09:33:38.839 5733.229 any 194.63.239.236 39860(

8.8) 25.2 M( 7.9) 34.6 G( 7.7) 4393 48.2 M 1372 2013-03-13 09:33:56.632 5714.286 any 194.63.239.235 38534(

8.5) 23.2 M( 7.3) 31.4 G( 7.0) 4053 44.0 M 1356 Summary: total flows: 452861, total bytes: 449.6 G, total packets: 319.0

M, avg bps: 620.0 M, avg pps: 54994, avg bpp: 1409 Time window: 2013-03-13 09:33:09 - 2013-03-13 11:10:00 Total flows processed: 38411283, Blocks skipped: 0, Bytes read:

2304722444 Sys: 6.808s flows/second: 5641281.6 Wall: 6.723s flows/second: 5713256.9

Date first seen Dst IP Addr Flows (%) Packets (%) Bytes (%)

2013-03-13 09:34 194.177.211.x 35531( 7.8) 36.1 M(11.3) 53.5 G(11.9)

2013-03-13 09:34 194.177.211.x 34632( 7.6) 35.6 M(11.1) 52.6 G(11.7)

2013-03-13 09:33 194.177.211.x 34469( 7.6) 35.3 M(11.1) 52.2 G(11.6)

2013-03-13 09:33 194.63.239.x 49621(11.0) 31.8 M(10.0) 44.3 G( 9.9)

2013-03-13 09:33 194.63.239.x 48220(10.6) 27.1 M( 8.5) 36.7 G( 8.2)

2013-03-13 09:33 194.63.239.x 39278( 8.7) 26.1 M( 8.2) 36.5 G( 8.1)

DNS Amplification Attack

• Target: GRNET

• Port Ranges: 53 (DNS)

• Multiple Source IP’s & Source

AS’s.

• Attack peak: 20G over 10G link

Today

DDoS Events – GRNET

25

http://nfsen.geant.net/nfsen.php











connect • communicate •

collaborate

Uhm…..Now What


Today

Security Changes - Audits


Strategy

…security solutions that simplify the improvement of

the security status quo…

28


Requirements - Defining

It must be easy to use

It must be ENHANCE security

Must deliver MEASURABLE VALUE

REDUNDANCY must be incorporated

into existing processes

…accepted by all participants …. conform to BEST PRACTICES & STANDARDS

Must be SCALABLE.

29


GÉANT Security

Complete Security Solution - NSHaRP

It is a mechanism to quickly and effectively inform affected users of

incidents detected transiting the GÉANT network dynamically.

It adds value by serving as an extension to an NRENs CERT, by adding

visibility to incidents targeting or originating from your network

Innovative and Unique - Caters for different types of requirements

….is a process that will enhance GÉANT backbone security and will

extend the NRENs ability to protect their infrastructure….

30


Firewall on Demand …But Why?

… better tools to mitigate transitory attacks and anomalies

“Better” in terms of

Granularity: Per-flow level

– Source/Dest IP/Ports, protocol type, DSCP, TCP flag…

Action:

– Drop, rate-limit, redirect

Speed: More responsive

– (Seconds / Minutes vs. Hours / Days)

Efficiency:

– Closer to the source, Multi Domain

Automation:

– Integration with other systems (NSHaRP)

Manageability

31

connect • communicate • collaborate Customer

FoD

NREN A

NREN B

GEANT

Credit: Andreas Polyrakis, GRNET

LEVEL3 • NSHaRP Customer or GN NOC

logs into web tool and describes

flows and actions

• Flow destination is validated

against the customer’s IP space

• Dedicated router is configured to

advertise the route via BGP

flowspec

• iBGP propagates the tuples to all

GEANT routers.

• Dynamic firewall filters are

implemented on all routers

• Attack is mitigated (dropped,

rated-limited) upon entrance

• End of attack: Removal via the

tool, or auto-expire

Firewall on Demand … Tomorrow

32


Phase 1 - Test Flow Spec on GN

Athens Router

- Test Propagation to GN

Gateways

Phase 2 - Deploy Flow Spec

Server

- Web Interface

- Pilot

Phase 2 (b) - Processes

- API

- Production Service

Firewall on Demand … Roadmap

Today 6 Months 12 Months

33

GÉANT Tests

Firewall on Demand Multidomain 34 Internet2 Global Summit, Apr 9 2014

GÉANT

CARNet

Victim

GRNET

Flowspec

Flowspec

FoD

Flowspec

Attacker

Click Apply

6 seconds later…

FoD multidomain principles


FoD setup & deploy by every interested domain/NREN

Multidomain FoD deployed in GÉANT

Multidomain FoD authentication: eduGAIN

Multidomain FoD authorization: peer address space

GÉANT accepts BGP flowspec rules from domains

Policies/filters per peering based on rule dest. addr.

User belongs to a domain/institution/NREN :: Peer

Peer is assigned an administrative IPv4 address space

Rule creation with destination address/network only inside the user’s Peer assigned address space

FoD multidomain deployment scenarios


GÉANT NREN

Flowspec

Flowspec

Flowspec

RTBH

ACL

Possible mitigation

with RTBH, ACL

Victim

NREN

Flowspec

Flowspec

FlowspecFlowspec

FoD

Legitimate Traffic Flows

Malicious Traffic Flows

Flow spec rule propagation

BGP Peering

Flowspec Flow spec rules

FoD Firewall on Demand platform

m·FoD

Flowspec

Current Status


GRNET in production since end of 2011

Tests: Multihop BGP peering with PSNC Interest/Evaluation from BELNET

GÉANT BGP flowspec enabled in all core devices Successful tests between GRNET and GÉANT

Multiple scenarios tested Iperf between Croatia and Greece Gone in 6 seconds

In production by April 2015

Extensions


FoD {single,multi}-domain interfaces to

other tools/platforms

REST API

XMPP client/server

ØMQ extensions

Filter counters/graphs NETCONF

Juniper UtilityMIB

Ipv6 support (Whenever available)

Can I deploy/try/test it?


Open source project

FoD : https://code.grnet.gr/projects/flowspy

Docs: https://flowspy.readthedocs.org

Ask for a demo account

PEER WITH US!

Demo time…


attaaaaack!

Questions?


42: “The Answer to the Ultimate Question of

Life, The Universe, and Everything.” Douglas Adams, The Hitchhiker's Guide to the Galaxy

Jeffrey Haas JUNIPER

Thank you

Leonidas Poulopoulos GRNET NOC

Wayne Routly DANTE


firewall on demand multidomain - internet2 · 4/9/2014 · bgp flowspec bgp flowspec was...

Documents