security and virtualization in the data center - cisco - · pdf filesecurity and...

© 2009 Cisco Systems, Inc. All rights reserved. Cisco PublicBRKSEC-2205 1

Security and Virtualization in the Data Center

Teerapol TuanpusaCisco Systems ThailandEmail: [email protected]


Agenda

Trends in Server and DC Virtualization

Security for Data Center Layers

Device Virtualization & Security Services

Security Considerations for Server Virtualization


Trends and DriversServer Consolidation Virtualization vs 10GbE

Multi-Core CPU architectures allowing bigger and multiple workloads on the same machine

Virtualization is creating a market transition

Server virtualization driving the need for more I/O bandwidth per server

Growing need for network storage driving the demand for higher network bandwidth to the server

Unified Fabric is now standard for LAN and SAN convergence

Servers are becoming fluid objects in the network

VmVIRTUAL MACHINE

2008

x86


Top IT Priorities in 2010


Virtualization and Cloud – Current State

Virtualization Remain Top Spending Priority15% of Server Workloads Virtualized in 2009; Forecast to be 50%-60% in Next 5 Years

2010 tipping point: More virtual servers than physical servers

Cloud Computing is a RealityEnterprise private clouds

Service provider public clouds

Desktop Virtualization is Moving from Pilot to Production

Sources: Goldman Sachs CIO Survey, Goldman Sachs IT Spending Survey, Industry Analyst Reports


Data Center are Evolving

Mainframe

Data Center 1.0

IT R

elev

ance

and

Con

trol

Application Architecture EvolutionCentralized

Data Center 2.0

Client-Server and Distributed Computing

Decentralized

Data Center 3.0

Service Oriented and Web 2.0 Based

Virtualized

Consolidate

Virtualize

Automate


VM-Aware virtualization

Fabric virtualization

Storage virtualization

Convergedvirtualization

Network virtualization

In the Network

Nexus 1000V

VN-Link Per VM Services

VM Mobility

Network hosted Servers

Nexus 7000

Nexus 5000

FCoE, DCE

10/40/100 GbE

NX-OS

MDS Directors

Intelligent Storage Apps

Fabric SAN

Branch WAN Optimization

All Resources Connect to a Unified Fabric

Automated, Virtualized, Unified, Transparent

Unified Computing Solution

Catalyst LAN Switching

Security

Application Networking

Cisco Data Center 3.0 VisionFive-Phase Virtualization Technology Plan


Where Are We Now?

Securing virtualized environments is a big concern

Two forms of virtualization we are discussing. Both apply to the Data Center

Device virtualization

Server virtualization

Security requirements shouldn’t change with virtualization


Data CenterCore

VM VM VM

VMVMVM

Data Center Services Layer

Virtual Access

Access Layer

Data Center Aggregation Layer

Virtual Infrastructure

Services

Top of Rack/End of Row

Aggregation/Distribution

Data Center Terms


Data Center Security Challenges

Virtualization

Applications

Data Loss

Compliance

Availability


Cisco SAFE Security Architecture

DataCenter Campus WAN

Edge Branch InternetEdge

Ecomm-erce

CiscoVirtualOffice

VirtualUser

PartnerSites

Services

Policy and Device Management

Security Solutions PCI DLP Threat Control Etc.

SecureNetworkFoundation Routers Servers Switches

IdentifyMonitor

Correlate

HardenIsolate

Enforce

Visibility Control

Mobility, Unified Communications, Network Virtualization

Core Network Protection

Security Devices VPNs Monitoring

Admission Control Intrusion Prevention

Firewall Email Filtering


Stateful Packet FilteringAdditional Firewall Services for Server Farm specific protection

Server Load BalancingServer Load Balancing masks servers and applications

Application FirewallApplication Firewall mitigates XSS, HTTP, SQL, XML based attacks

Network Intrusion PreventionIPS/IDS: provides traffic analysis and forensics

Flow Based Traffic AnalysisNetwork Analysis for traffic monitoring and data analysis

XML based Application ControlXML Gateway to protect and optimize Web-based services

Stateful Packet FilteringInitial filter for all DC ingress and egress traffic. Virtual Context allow correlation to Nexus VDC.

Network Foundation ProtectionInfrastructure Security features are enabled to protect device, traffic plane, and control plane. Device virtualization provides control, data, and management plane segmentation

Data CenterCore

VM VM VM

VMVMVM


Virtual Access

Access Layer

Enhanced Layer 2 SecurityAccess List, Dynamic ARP Inspection, DHCP Snooping, IP Source Guard, Port Security, Private VLANs, QoS

Endpoint securityHost intrusion prevention protect server against zero day attacks

Layer 2 Flow MonitoringNetFlow, ERSPAN, SPAN


•Visibility•Even Correlation

HIPS, Firewalls,IPS, Netflow, Syslog

•Forensics•Anomaly Detection•Compliance

CSM CS-MARS

Security Management

Addressing theChallenges


FBI/CSI Risk Assessment*

Many enterprises network ports are open Usually any laptop can plug into the network and gain

access to the network

Of companies surveyed total loss was over $130 million

Average spending per employee $241 per year

28% said they had no idea how many times or if they were were attacked

More than 50% of loss are from internal

*CSI/FBI Computer Crime and Security Survey—2009http://www.ussecurityawareness.org/highres/free-resources.html


Hardening

Hosts and network gear is both a target and weapon Harden all the devices in your environment!! Develop consistent baselines for “images” and audit use The level of hardening to apply depends on the device location and

functionImportance to the businessLikelihood of being attacked (often based on ease-of-reach)

HostsPervasive: patch OS, patch apps, service hardening, file access, user auth, AV, file system integrity checkersOptional: FW, IPS, file system encryption

Network DevicesPervasive: admin AAA, secure command channel comms, audit trail, service hardeningOptional: authenticated routing, secure output comms, resource throttles, L2 hardening (no auto trunk, disable unused ports, PVLANs)Links: www.cisco.com/warp/public/707/21.html


Routers are Targets

Potentially a hacker’s best friend Protection should include:

- constraining telnet access (SSH is more preferred)- SNMP read-only (SNMPv3 is preferred)- administrative access with TACACS+ (CLI AAA)- turning off unneeded services- logging unauthorized access attempts- authentication of routing update (MD5)- turn on uRPF checking for anti spoofing- enable Control Plane Policing (CoPP) to prevent DoS attacks- http://www.cisco.com/warp/public/707/21.html


Output Packet Buffer

LocallySwitched Packets

Input to the Control

Plane

ProcessorSwitched Packets

Control Plane Policing (CoPP)

Secure routers against DoS attacks Apply QoS to processor switched packets Divide required protocols into priority groups

Control Plane Policing

(Alleviating DoS)

Silent Mode(Prevent Recon)

Control Plane Output from the Control Plane

Management SNMP, Telnet ICMP IPv6 Routing

UpdatesManagement

SSH, SSL …..

IncomingPackets

PacketBuffer

CEF/FIB Lookup


Switches are Targets

Protection needs are similar to routers

VLANs are an added vulnerability:- remove user ports from auto-trunking

- use non-user VLANs for trunk ports

- set unused ports to a non-routed VLAN

- http://www.sans.org/newlook/resources/IDFAQ/vlan.htm


No, You’reNot!

I’m YourEmail Server

IP Source Guard

Dynamic ARP Inspection

DHCP Snooping

Port Security

Email ServerInnocent User

I’m theUser

Security ServicesCatalyst Integrated Security: Overview

Attack Catalyst Feature

MAC Address Flooding Port Security

DHCP Rogue Server for Default Gateway

DHCP Snooping

ARP Spoofing or ARP Poisoning


IP Spoofing IP Source Guard


MAC/CAM AttacksMACOF Attack Tool?

[root@macattack]# macof –i eth036:a1:48:63:81:70 15:26:8d:4d:28:f8 0.0.0.0.26413 > 0.0.0.0.49492: S 1094191437:1094191437(0) win 51216:e8:8:0:4d:9c da:4d:bc:7c:ef:be 0.0.0.0.61376 > 0.0.0.0.47523: S 446486755:446486755(0) win 51218:2a:de:56:38:71 33:af:9b:5:a6:97 0.0.0.0.20086 > 0.0.0.0.6728: S 105051945:105051945(0) win 512e7:5c:97:42:ec:1 83:73:1a:32:20:93 0.0.0.0.45282 > 0.0.0.0.24898: S 1838062028:1838062028(0) win 51262:69:d3:1c:79:ef 80:13:35:4:cb:d0 0.0.0.0.11587 > 0.0.0.0.7723: S 1792413296:1792413296(0) win 512c5:a:b7:3e:3c:7a 3a:ee:c0:23:4a:fe 0.0.0.0.19784 > 0.0.0.0.57433: S 1018924173:1018924173(0) win 51288:43:ee:51:c7:68 b4:8d:ec:3e:14:bb 0.0.0.0.283 > 0.0.0.0.11466: S 727776406:727776406(0) win 512b8:7a:7a:2d:2c:ae c2:fa:2d:7d:e7:bf 0.0.0.0.32650 > 0.0.0.0.11324: S 605528173:605528173(0) win 512e0:d8:1e:74:1:e 57:98:b6:5a:fa:de 0.0.0.0.36346 > 0.0.0.0.55700: S 2128143986:2128143986(0) win 512

MACOF is one of a number of tools available with “DSNIFF”

Dynamically generates MAC addresses to fill the Switch CAM table…

Three main development platformsRed Hat Linux, Solaris and Open BSD(Also on Win2K/XP, FreeBSD, Debian, AIX, and HPUX)

http://www.monkey.org/~dugsong/dsniff


MAC/CAM Attacks Stopping MAC/CAM Attacks!!

The Port Security feature can be used to stop MAC Spoofing, MACOF or any other CAM attack variant tool… Port Security allows you to set a MAC address for a port or set a max number of MAC addresses it can learn on that switchport…

SwitchA

B

C1

2

3

MACOF

Switch(config-if)# switchport port-security ?aging Port-security aging commandsmac-address Secure mac addressmaximum Max secure addressesviolation Security violation mode<cr>


DHCP SnoopingProtection Against Rogue/Malicious DHCP Server

DHCP requests (discover) and responses (offer) tracked Limits DOS attacks on DHCP server by Port Security or Rate Limiting Deny responses (offers) on non trusted interfaces; stop malicious or errant

DHCP Server

DHCP Server1000s of DHCP

Requests to Overrun the

DHCP Server

2

1


Countermeasures for DHCP AttacksRogue DHCP Server = DHCP Snooping

By default all ports in the VLAN are untrusted

Client

DHCPServerRogue Server

Trusted

Untrusted

Untrusted

DHCP Snooping-Enabled

DHCP Snooping Untrusted ClientInterface Commandsno ip dhcp snooping trust (Default)ip dhcp snooping limit rate 10 (pps)

Cisco IOSGlobal Commandsip dhcp snooping vlan 4,104no ip dhcp snooping information optionip dhcp snooping

DHCP Snooping Trusted Serveror Uplink

BAD DHCP Responses:offer, ack, nak

OK DHCP Responses: offer, ack, nak

Interface Commandsip dhcp snooping trust


Countermeasures for DHCP AttacksRogue DHCP Server = DHCP Snooping

Table is built by “snooping” the DHCP reply to the client

Entries stay in table until DHCP lease time expires

Client

DHCPServerRogue Server

Trusted

Untrusted

Untrusted

DHCP Snooping-Enabled

DHCP Snooping Binding Tablesh ip dhcp snooping bindingMacAddress IpAddress Lease(sec) Type VLAN Interface------------------ --------------- ---------- ------------- ---- --------------------00:03:47:B5:9F:AD 10.120.4.10 193185 dhcp-snooping 4 FastEthernet3/18

BAD DHCP Responses:offer, ack, nak

OK DHCP Responses: offer, ack, nak


ARP Attack Tools

Many tools on the net for ARP man-in-the-middle attacks

Dsniff, Cain & Abel, ettercap, Yersinia, etc.

ettercap: http://ettercap.sourceforge.net/index.phpSome are second or third generation of ARP attack tools

Most have a very nice GUI, and is almost point and click

Packet insertion, many to many ARP attack

All of them capture the traffic/passwords of applications FTP, Telnet, SMTP, HTTP, POP, NNTP, IMAP, SNMP, LDAP, RIP, OSPF, PPTP, MS-CHAP, SOCKS, X11, IRC, ICQ, AIM, SMB, Microsoft SQL, etc.


ARP Attack Tools

Ettercap in action

As you can see runs in Window, Linux, Mac

Decodes passwords on the fly

This example, telnet username/ password is captured


ARP Attack Tools: SSH/SSL

Using these tools SSL/SSH sessions can be intercepted and bogus certificate credentials can be presented

Once you have excepted the certificate, all SSL/SSH traffic for all SSL/SSH sites can flow through the attacker


Countermeasures to ARP Attacks: Dynamic ARP Inspection

Uses the information from the DHCP snooping binding table

Looks at the MacAddress and IpAddress fields to see if the ARP from the interface is in the binding; if not, traffic is blocked

sh ip dhcp snooping bindingMacAddress IpAddress Lease(sec) Type VLAN Interface------------------ --------------- ---------- ------------- ---- --------------------00:03:47:B5:9F:AD 10.120.4.10 193185 dhcp-snooping 4 FastEthernet3/18


No, You’reNot!

I’m YourEmail Server

IP Source Guard


DHCP Snooping

Port Security

Email ServerInnocent User

I’m theUser

Security ServicesCatalyst Integrated Security: Overview

Attack Catalyst Feature

MAC Address Flooding Port Security

DHCP Rogue Server for Default Gateway

DHCP Snooping

ARP Spoofing or ARP Poisoning


IP Spoofing IP Source Guard


Private VLANsEnhance Access Control in DC and DMZ

PromiscuousPort

PromiscuousPort

Community‘A’

Community‘B’

IsolatedPorts

Primary VLAN

Community VLAN

Community VLAN

Isolated VLAN

Only One Subnet!

x x x x


NetFlow Telemetry

Applications:

Router:• Cache creation• Data export• Aggregation

Collector:• Collection• Filtering• Aggregation• Storage• File system management

Accounting/Billing

Network Planning

Data Presentation

PartnersCisco and PartnersCisco


Stateful Packet FilteringAdditional Firewall Services for Server Farm specific protection

Server Load BalancingServer Load Balancing masks servers and applications

Application FirewallApplication Firewall mitigates XSS, HTTP, SQL, XML based attacks

Network Intrusion PreventionIPS/IDS: provides traffic analysis and forensics

Flow Based Traffic AnalysisNetwork Analysis for traffic monitoring and data analysis

XML based Application ControlXML Gateway to protect and optimize Web-based services

Stateful Packet FilteringInitial filter for all DC ingress and egress traffic. Virtual Context allow correlation to Nexus VDC.

Network Foundation ProtectionInfrastructure Security features are enabled to protect device, traffic plane, and control plane. Device virtualization provides control, data, and management plane segmentation

Data CenterCore

VM VM VM

VMVMVM


Virtual Access

Access Layer

Enhanced Layer 2 SecurityAccess List, Dynamic ARP Inspection, DHCP Snooping, IP Source Guard, Port Security, Private VLANs, QoS

Endpoint securityHost intrusion prevention protect server against zero day attacks

Layer 2 Flow MonitoringNetFlow, ERSPAN, SPAN


•Visibility•Even Correlation

HIPS, Firewalls,IPS, Netflow, Syslog

•Forensics•Anomaly Detection•Compliance

CSM CS-MARS

Security Management

Addressing theChallenges


Data Center: Aggregation Layer Design


Device Virtualization:Nexus 7000 Virtual Device Contexts

Up to 4 separate virtual switches from a single physical chassis with common supervisor module(s)

Separate control plane instances and management/CLI for each virtual switch

Interfaces only belong to one of the active VDCs in the chassis, external connectivity required to pass traffic between VDCs of the same switch

VDCs


Virtual Device Contexts @ Nexus 7000

Kernel

Infrastructure

Protocol StackVDCA

Nexus 7000 Physical Switch

VDC A

Pro

cess

AB

C

Pro

cess

DE

F

Pro

cess

XY

Z

…

Protocol StackVDCB

VDC B

Pro

cess

AB

C

Pro

cess

DE

F

Pro

cess

XY

Z

…

Process “DEF” in VDC B Crashes

Process DEF in VDC A Is Not Affected and Will Continue to Run Unimpeded

A VDC Builds a Fault Domain Around All Running Processes Within That VDC—Should a Fault Occur in a Running Process, It Is Truly Isolated from Other Running Processes and They Will Not Be Impacted

ABCD

AB

C D


FIB TCAMSize 128K

ACL TCAMSize 64K

FIB TCAMSize 128K

FIB TCAMSize 128K

FIB TCAMSize 128K

VDC-1IP routes: 20K

ACL entries: 10K

VDC-2IP routes: 100KACL entries: 50K

ACL TCAMSize 64K

VDC-3IP routes: 100KACL entries: 50K

ACL TCAMSize 64K

ACL TCAMSize 64K

Linecard 1 Linecard 2

Linecard 3Linecard 4

1 : NVirtual Device Contexts Separate Resource Allocation Domains (Layer 3)


Data Center Core

Multiple Aggregation VDCs

Access

Enterprise Network

Virtual Device Context Example:Multiple Aggregation Blocks

Single physical pair of aggregation switches used with multiple VDCs

Access switches dual-homed intoone of the aggregation VDC pairs

Aggregation blocks only communicate through the core layer

Design considerations:Ensure control plane requirementsof multiple VDCs do not overload Supervisor or I/O Modules

Where possible consider dedicating complete I/O Modules to one VDC(CoPP in hardware per-module)

Ports or port-groups may be moved between aggregation blocks (DC pods) without requiring re-cabling


Core

Aggregation VDC

Access

Sub-AggregationVDC

6500Services Chassis

Enterprise Network

Virtual Device Context Example:

Multiple VDCs used to “sandwich” services between switching layers

Allows services to remain transparent (layer-2) with routing provided by VDCs

May be leveraged to support bothservices chassis and appliances

Design considerations:Access switches requiring services are connected to sub-aggregation VDC

Access switches not requiring servicesmay be connected to aggregation VDC

Allows firewall implementations not toshare interfaces for ingress and egress

Facilitates virtualized services byusing multiple VRF instances inthe sub-aggregation VDC

Services VDC Sandwich


Cat6k Cat6k

N7k2‐VDC1

N7k2‐VDC2

N7k1‐VDC1

N7k1‐VDC2

Po99

vrf2 vrf1vrf1 vrf2

10.8.0.x/24.2

10.8

.1.x

/24 10.8.2.x/24

10.8.3.x/24

.1.1 .1

.2.2(SVI 3) .2.1 (SVI 3)

Po99

10.8.162.3/24 10.8.162.2/2410.8.152.3/24 10.8.152.2/24

RID:8.8.8.1 RID:8.8.8.2

RID:4.4.4.1 RID:4.4.4.2RID:5.5.5.1 RID:5.5.5.2

RID:3.3.3.1 RID:3.3.3.2

10.8.152.5 10.8.152.6 10.8.162.610.8.162.5

Aggregation Layer with VDCs

Outside Virtual Device Context

InsideVirtual Device Context


Enterprise Network

VLAN 161

VLANs171,172

VLAN 163

VLAN 170

Web Server Farm

VLAN 162

Transparent FWSM Context

TransparentACE Context

AggregationVDC

Services

Sub-AggregationVDC

Access

VLAN 180

Data CenterCore

Client-Server Flow

Using Virtualization and Service Insertion to Build Logical Topologies

Logical topology exampleusing services VDC sandwich physical model

Layer-2 only services chassis with transparent service contexts

VLANs above, below, and between service modules are a single IP subnet

Sub-aggregation VDC is a layer-3 hop running HSRP providing defaultgateway to server farm subnets

Multiple server farm VLANS can beserved by a single set of VLANsthrough the services modules

Traffic between server VLANs does not need to transit services device, but may be directed through services using virtualization


FT VLANs

Enterprise Network

VLAN 161

VLAN 163

FT VLAN

Web/AppServer Farm

Transparent FWSM Contexts

TransparentACE Contexts

VRF VRF

VRF Instances

Aggregation VDC

Services

Sub-Agg VDC

Access

VLAN 180

Data Center Core

VLAN 153

VLAN 152

VRF VRF

VLAN 181

FT VLANs

FT VLAN

DB ServerCluster

VLAN 151

Client-Server Flow

Server to Server Flow

VLAN 162

Logical Topology to support multi-tier application traffic flow

Same physical VDC serviceschassis sandwich model

Addition of multiple virtual contexts to the transparent services modules

Addition of VRF routing instances within the sub-aggregation VDC

Service module contexts and VRFs are linked together by VLANs toform logical traffic paths

Example Web/App server farmand Database server cluster homedto separate VRFs to direct traffic through the services

Using Virtualization and Service Insertion to Build Logical Topologies


Aggregation Security Features

CoPPProtect the supervisor from DoS attacks preventing outages. Prevent Layer 2 broadcast storms and irrelevant traffic redirections to CPU

Broadcast SuppressionProtects the data center against broadcast storms at the port level that pose risks to bandwidth availability

Packet Sanity ChecksForwarding engine performs extensive checks on IPv4 and IPv6 packet headers to protect the network from illegal packets.

LinkSec Wire-rate link-layer cryptography is provided at all ports. Packets are encrypted on egress and decrypted on ingress so they are clear inside the device.


Additional Nexus 7000 Tidbits

Virtualization supportAAA configuration and operation are local to the VDC.

AAA authentication methods for the console login only apply to the default VDC.

AAA accounting log is on per VDC basis

Role Based AccessFour default roles

Network-admin

Permission to create/delete/assign resources to VDC.

Can create other roles and users.

Network-operator

Permission to run show command across all VDCs.

VDC-admin

Permission to manage a VDC, create other VDC roles and users

for that VDC.

VDC-operator

Local to a VDC and has show command privilege


Data Center: Security Services Insertion(and Others)


Security Services

Data CenterCore

VM VM VM

VMVMVMVirtual Access

Access Layer



Physical Solution Topology

ACE Module

ACE WAF

IDS/IPS IDS/IPS

ACE Module

ASA5580

ASA5580

Catalyst6500

Nexus7000

Nexus7000

Catalyst6500

Catalyst6500s

Catalyst4900s

Catalyst6500s VSS

Nexus5000s

Catalyst3100 VBS

Core Layer

ServicesLayer

AggregationLayer

AccessLayer

WAAS


Virtualized Data Center Infrastructure

CBS 31xx Blade

Nexus 5000 & Nexus 2000Top-of-Rack

Nexus 7000End-of-Row

Nexus 5000 &FCoETop-of-Rack

Access Layer

Nexus 700010GbE Agg

MDS 9500Storage

Catalyst 6500End-of-Row

CBS 31xxMDS 9124eNexus blade (*)

10GbE and 4/8Gb FC Server Access10Gb DCE / FCoE Server Access

1GbE Server Access

Nexus 700010GbE Core

Gigabit Ethernet

10 Gigabit Ethernet

10 Gigabit DCE

4/8Gb Fiber Channel

10 Gigabit FCoE/DCE

(*) future

SAN BSAN A

Cisco UCS

Cisco Catalyst 6500DC Services

Aggregation LayervPC

vPCCore Layer

One-Arm Service Switches


Security Service Integration

Deploy security services and appliances as transparently as possible.

Maintain predictable traffic flows to ensure availability

Need to think about scalability of current infrastructure when planning designs.

Create Security Zones based on Trust

Minimal impact to allowed functions while maintainingEnforcement, Isolation, Visibility

Business model, compliance, applications, can all drive policy

One model does not fit all but there are some design guidelines we can provide


(VDC max = 4)

(ASA max = 50 VCs)(FWSM max = 250)

(ACE max = 250 VCs)

(VS max = 4)

Nexus7000

ASA

ACE

IPS/IDS

(ACE 4710 = 20 VCs)

Active-Active Solution Virtual Components

Nexus 7000VDCs, VRFs, SVIs

ASA 5580Virtual Contexts

ACE Service ModuleVirtual Contexts, Virtual IPs (VIPs)

IPS 4270Virtual Sensors

Virtual Access LayerVirtual Switching SystemNexus 1000vVirtual Blade Switching


One Physical DeviceMultiple Virtual Systems

(Dedicated Control and Data Path)

ACE Module and Appliance: Virtual Partitioning

Single configuration file

Single routing table

Limited RBAC

Limited resource allocation

Distinct context configuration files

Separate routing tables

RBAC with contexts, roles, domains

Management and data resource control

Independent application rule sets

Global administration and monitoring

Supports routed and bridged contexts at the same time

25% 25% 20%15%15%100%

Cisco Application Infrastructure ControlTraditional Device

System Separation for Server Load Balancing and SSL


Firewall Service Module (FWSM)Virtual Firewalls

e.g., Three customers three security contexts—scales up to 250

VLANs can be shared if needed (VLAN 10 on the right-hand side example)

Each context has its own policies (NAT, access-lists, inspection engines, etc.)

FWSM supports routed (Layer 3) or transparent (Layer 2) virtual firewalls at the same time

Core/Internet

Cisco Catalyst 6500

FW SMVFW VFW VFW

MSFC

Core/Internet

Cisco Catalyst 6500

FW SMVFW VFW VFW

MSFC

VLAN 10 VLAN 20 VLAN 30

VLAN 11 VLAN 21 VLAN 31

VLAN 10

VLAN11 VLAN 21 VLAN 31

A B C A B C


Data Center Virtualized ServicesCombination Example

v5

v105

v6 v7

v107

v2081v2082v2083...

v206 v207

v206

BU-4BU-2 BU-3

v105

v108

BU-1

1

2

3

4

* vX = VLAN X**BU = Business Unit

VRF

VRF

VRFVRFVRF

v208

“Front-End” VRFs (MSFC)

Firewall Module Contexts

ACE Module Contexts

“Back-End” VRFs (MSFC)

Server Side VLANs

v207

3

4v8


Cat6k Cat6k

SS1 SS2

N7k2-VDC1

N7k2-VDC2

N7k1-VDC1

N7k1-VDC2

ASA2

ACE2

ASA2

ASA1ASA1

ACE1

OSPF NSSA Area 81

Po99

OSPF Area 0

vrf2 vrf1vrf1 vrf2

10.8.0.x/24.2

10.8

.1.x

/24

10.8.2.x/2410.8.3.x/24

.1.1 .1

.2.2(SVI 3) .2.1 (SVI 3)

Po99

10.8.162.3/24 10.8.162.2/2410.8.152.3/24 10.8.152.2/24

RID:8.8.8.1 RID:8.8.8.2

RID:4.4.4.1 RID:4.4.4.2RID:5.5.5.1 RID:5.5.5.2

RID:3.3.3.1 RID:3.3.3.2

10.8.152.5 10.8.152.6 10.8.162.610.8.162.5

Layer 2Service Domain

Active-Active Solution Logical Topology


N7k

2-V

DC

1

N7k

1-V

DC

1

N7k2-VDC2N7k1-VDC2

ASA2

ACE2

ASA2

ASA2ASA1

ACE1

SVI-161

SVI-151SVI-151

SVI-161

vrf2 vrf1vrf1 vrf2 Po99

Po99

10.8.162.3 10.8.162.2

10.8.152.3 10.8.152.2

hsrp.1

hsrp.1

hsrp.7 hsrp.7

10.8.162.5 10.8.162.610.8.152.610.8.152.5

IPS1 IPS2

163,164 163,164

162

161

164

SS2SS1

Server Farm

Traffic Flow & Service Pattern Active-Active:Client-to-Server


ServiceSwitch

Po2

IPS/IDS

1.........8

Service Integration Services Layer Analysis-IPS/IDS

Virtual IPS/IDS sensors leveragestatic EtherChannel IPS inline VLAN pairing to

single services chassis Src/Dest EtherChannel hash

maintains symmetric flows EtherChannel

scalability and availability


Data Center: Access Layer Design


Data Center Physical Access Layer

The physical data center access layer is fairly well understood.

The features and design options at this layer have evolved through the use of virtualization

Security features for the access layer have been available and deployed for quite some time

A few highlights for the physical access layer before we look at Virtual Access…


VM VM VM

VMVMVM

Virtual Access

Access Layer

Data CenterCore



Data Center Access Layer

Data Center Access


Security Considerations

In many cases server tiers/clusters are separated by VLANs

Servers are often Layer 2 adjacent

Must allow for mobilityDR

Maintenance

Security is key in maintaining availability of servers and applications connected here.


Make Use of Switch Security Features

Anti-spoofing featuresDynamioc ARP Inspection, IP Source Guard, DHCP Snooping

STP protection (BPDU Guard)

QoS

Broadcast Packet Suppression

PVLANs

Access Lists

SPAN, ERSPAN, NetFlow


Data Center: Virtual Access and Security Concerns


Server Virtualization

Benefits of Virtualization Power savings

Consolidation of resources

Server portability

Application failover

Uplink Ports

Virtual Ethernet (vnet) Adapters

Uplink Ports

Physical Adapters


Server Virtualization

Hypervisors: Type 1 or Type 2Type 1 hypervisors as shown below are built into a pre-hardened host. There is no distinct boundary between the host operating system and the hypervisor.

Type 2 hypervisors as shown below are installed as separate software on top of the existing host operating system

Primary role of the host OS or hypervisor is to work with the VMM to coordinate access to the physical host system's hardware resources (CPU, Device Drivers, etc)

Theoretically the hypervisor should have fewer security vulnerabilities because it runs minimal services and contains only essential code BUT maintaining security updates is still important!


Server Virtualization Security Concerns

Secure HypervisorMitigate risk towards the hypervisoran attacker gaining unauthorized access to the hypervisor and taking control of the physical server and related virtual servers

Rogue VMsHas a guest operating system been compromised?Virtual Server Mobility

Inter-VM traffic visibility and securityTraffic between two virtual machines can flow across the bus inside the hosting physical server and not be required to be switched on an external network where traditional tools can be usedVMware “virtual switch” lacks security features available in Cisco switching platforms

Shared File system between VMsVMFS and VMotionConsolidated SANs or NAS attached storage

vnet adapters

Uplink Ports

Physical Adapters


vnet adapters

Uplink Ports

Physical Adapters

Securing the Hypervisor…

Hypervisor has access to all resources

Manages all system resources

Manages LAN & SAN access

vSwitch lacks “standard” network functions

No visibility into VM-to-VM traffic on a port group

No visibility into VM-to-Hypervisor calls

!!!!

!!

!!


vnet adapters

Uplink Ports

Physical Adapters

Virtual Machine LAN Security

Be aware of security affinitiesWould you place all your applications on the same VLAN?

Challenging troubleshooting & monitoring environment

Recommendation: Do not consolidate servers with unlike security affinities onto a single VLAN

DMZ Web Server

ApplicationServers

DatabaseServer

!! !! !!


Virtual Machine VMotion Security

VMotion enables workload mobility & Disaster Recovery

Increases server utilization efficiency by balancing workloads between servers

VMs can move between ESX cluster members with the same configuration

Port-groups, VLANs, etc

Inconsistent security policies enforcement and visibility

Policies applied at the server port or VLAN cannot be consistently applied

Vmotion traffic sent in clear text. Take precautions for isolating

vnet adapters

Uplink Ports

Physical Adapters

vnet adapters

Uplink Ports

Physical Adapters

ESX Cluster

.11 .13

Permit .11 <-> .12Deny .11 <-> .13Deny .12 <-> .13

X

.12


Virtual Machine Exploits

Several Theoretical ExploitsGain Control of the HypervisorExploiting vMotion

Reconnaissance: Virtual Machine DetectionVME artifacts Malware that detects virtual machinesTools: (The Red Pill, Scoopy & Doo, VMDetect, etc)Virtual machine-based root kits

Theoretical attacks are interesting but lets focus on the simple things that cover 99% of the issues. Most people don’t even have the simple items covered!Lets worry about this before we worry about theoretical Hypervisor attacks.


Things to Ponder…

Traditional Security Problems Unchanged

Security Policies still need to be enforced

Virtualization introduces some new flavorsHypervisor is a new layer of privileged software

Potential loss of separation of duties

Limited visibility into inter-VM traffic

So What’s the Secret Ingredient?


There Is NO Secret Ingredient!

Security best practices still apply!

If you would not do it on a non-virtualized server, you probably should not do it on a virtualized server.

But we can address the virtualization concerns…


Server Virtualization Issues

1. vMotion moves VMs across physical ports—the network policy must follow

2. Impossible to view or apply network policy to locally switched traffic

3. Need shared nomenclature for security policies between network and server admin

PortGroup

vCenter

Physical Switch Interface


Cisco Nexus 1000VIndustry First 3rd Party Virtual Distributed Switch

Nexus 1000V provides enhanced VM switching for VMW ESX environments

Features VN-Linkcapabilities:

Policy-based VM connectivity

Mobility of network and security properties

Non-disruptive operational model

Ensures visibility and continued connectivity during VMotion

Enabling Acceleration of Server Virtualization Benefits

VMW ESX

Server 1

VMware vSwitch Nexus 1000VVMW ESX

VMware vSwitch Nexus 1000V

Server 2

Nexus 1000V

VM #4

VM #3

VM #2

VM #1

VM #8

VM #7

VM #5

VM #5

VM #2

VM #3

VM #4

VM #5

VM #6

VM #7

VM #8

VM #1

VM #1


Cisco Nexus 1000V – VM Security

Server

Private VLAN• Promiscuous port• Isolated port• Community port

Server

I

Server

ICisco Nexus 1000V

VM #1

VM #4

VM #3

VM #2

VM #4

VM #3

VM #2

VM #1

VM #4

VM #3

VM #2

VM #1

VMW ESX VMW ESX VMW ESX

I I

Security Features• Access Control List• Port Security• DHCP Snooping• IP Source Guard• Dynamic ARP Inspection

P CC

Cisco TrustSec• Admission control: 802.1X

• Hop-by-hop crypto: 802.1AE

• Security Group Tag


Separation of Duties: Network and Server Teams

A network feature macro

Example: Features are configured under a port profile once and can be inherited by access ports

Familiar IOS look and feel for network teams to configure virtual infrastructure

PromiscuousPort

10.10.10.10 10.10.20.2010.10.30.30

port-profile vm180vmware port-group pg180switchport mode accessswitchport access vlan 180ip flow monitor ESE-flow inputip flow monitor ESE-flow outputno shutdownstate enabled

interface Vethernet9inherit port-profile vm180


port-profile vm180vmware port-group pg180switchport mode accessswitchport access vlan 180ip flow monitor ESE-flow inputip flow monitor ESE-flow outputno shutdownstate enabled



Port Profiles


Port Profile: Network Admin View

n1000v# show port-profile name WebProfileport-profile WebProfiledescription:status: enabledcapability uplink: nosystem vlans:port-group: WebProfileconfig attributes:switchport mode accessswitchport access vlan 110no shutdown

evaluated config attributes:switchport mode accessswitchport access vlan 110no shutdown

assigned interfaces:Veth10

Support Commands Include:

Port management VLAN PVLAN Port-channel ACL Netflow Port Security QoS

Support Commands Include:

Port management VLAN PVLAN Port-channel ACL Netflow Port Security QoS


Port Profile: Server Admin View


Separation of Duties: Network and Server Teams1. Nexus 1000V automatically enables port groups in Virtual Center via API

2. Server Admin uses Virtual Center to assign vnic policy from available port groups

3. Nexus 1000V automatically enables VM connectivity at VM power-on

Workflow remains unchanged


VMotion

1. Virtual Center kicks off a VMotion (manual/DRS) & notifies Nexus 1000V

2. During VM replication, Nexus 1000V copies VM port state to new host

3. Once VMotion completes, port on new ESX host is brought up & VM’s MAC address is announced to the network

Mobile Properties Include:

Port policy

Interface state and counters

Flow statistics

Remote port mirror session

vnet adapters

Uplink Ports

Physical Adapters

vnet adapters

Uplink Ports

Physical Adapters

VMotion


Community

VLAN

Isolated VLAN

PromiscuousPort

VM Isolation: Cisco Private VLANs

Private VLANs provide layer 2 isolation for hosts in the same subnet

Traditional Cisco PVLANs are supported: Isolated & Community ports

Physical Infrastructure is PVLAN aware. You can carry PVLAN to physical devices ie: FWSM


PromiscuousPort

10.10.10.10

10.10.10.1

10.10.20.20

10.10.20.20

dcvsm(config)# ip access-list deny-vm-to-vm-trafficdcvsm(config-acl)# deny ip host 10.10.10.10 host 10.10.20.20dcvsm(config-acl)# permit ip any any

VM Isolation and Traffic Control

Port ACLs

Limit VM to VM traffic flows

Enforce the way you enforce between physical servers today

Use in conjunction with VLANs, PVLANs


PromiscuousPort

10.10.10.10 10.10.20.20

192.168.20.0

Isolating Production and Management Traffic

Isolate management traffic from production

Enforce physical separation and virtual separation

dcvsm(config)# ip access-list deny-vm-traffic-to-service consoledcvsm(config-acl)# deny ip 10.10.0.0 192.168.20.0dcvsm(config-acl)# permit ip any any


PromiscuousPort

10.10.10.10 10.10.20.20

ip arp inspection vlan 180!ip arp inspection filter staticIP vlan 180arp access-list staticIPpermit ip host 10.10.10.10 mac host 00:50:56:87:18:2dpermit ip host 10.10.20.20 mac host 00:50:56:87:18:3dpermit ip host 10.10.30.30 mac host 00:50:56:87:18:4d!errdisable recovery cause arp-inspectionerrdisable recovery interval 120!switchport access vlan 180switchport mode accessip arp inspection limit rate 100

ip arp inspection vlan 180!ip arp inspection filter staticIP vlan 180arp access-list staticIPpermit ip host 10.10.10.10 mac host 00:50:56:87:18:2dpermit ip host 10.10.20.20 mac host 00:50:56:87:18:3dpermit ip host 10.10.30.30 mac host 00:50:56:87:18:4d!errdisable recovery cause arp-inspectionerrdisable recovery interval 120!switchport access vlan 180switchport mode accessip arp inspection limit rate 100

10.10.30.30

Anti-Spoofing

Protection against man-in-the middle attacks

Dynamic ARP Inspection, DHCP Snooping, IP Source Guard


Services

IDS1Network Analysis Module

ERSPAN DST

ID:1ID:2

VM to VM Visibility

ERSPAN source requires use of ERSPAN destination

Only one IP address associated with the ERSPAN source/destination per switch

ERSPAN ID provides segmentation

Permit protocol type header “0x88BE” for ERSPAN GRE

ERSPAN frame considerations:

ERSPAN does not support fragmentation

Appends 50 Byte header to frame

Default 1500 MTU allows for 1468 byte frames

Max frame size supported 9,202 bytes

ERSPAN


ERSPANNexus 1000 Configuration

port-profile erspancapability l3controlvmware port-groupswitchport access vlan 3000no shutdownsystem vlan 3000state enabled

!monitor session 1 type erspan-source

description - to SS1 NAM via VLAN 3000source interface Vethernet8 bothdestination ip 10.8.33.4erspan-id 1ip ttl 64ip prec 0ip dscp 0mtu 1500no shut

monitor session 2 type erspan-sourcedescription - to SS1 IDS1 via VLAN 3000source interface Vethernet8 bothdestination ip 10.8.33.4erspan-id 2ip ttl 64ip prec 0ip dscp 0mtu 1500no shut


ERSPAN – IDS and NAM

Comprehensive view of VM traffic via ERSPAN to two network analysis devices simultaneously

NAM and IDS provide clarity. In this example, port scan of VM detected on IDS and visible on NAM


Example: Using ERSPAN to IDS for VMto VM Traffic

ERSPAN DSTIP: 10.8.33.4

10.8.180.230

Services

IDS1Network Analysis Module

ID:1ID:2

10.8.180.234


Out‐of‐BandNetFlow Collector

In‐BandNetFlow Collector

VM to VM Visibility

N1k requires Netflow source interface

Defaults to Mgmt0

Support v9 format

NetFlow


NetFlow

Maximum of one flow monitor per interface per direction is permitted

Maximum of two flow exporters per monitor are permitted

Port profiles afford easy deployment

flow exporter exporttest

description exportv9

destination <IP ADDRESS> use-vrf management

transport udp 3000

source mgmt0

version 9

template data timeout 1200

option exporter-stats timeout 1200

flow monitor NAMTest

description default flow to NAM

record netflow-original

exporter exporttest

timeout inactive 600

timeout active 1800

cache size 15000

port-profile vm180

vmware port-group pg180

switchport mode access

switchport access vlan 180

ip flow monitor NAMTest input

ip flow monitor NAMTest output


Features of the Nexus 1000V

Switching L2 Switching, 802.1Q Tagging, VLAN Segmentation, Rate Limiting (TX)

IGMP Snooping, QoS Marking (COS & DSCP), Class-based WFQ*

Security Policy Mobility, Private VLANs w/ local PVLAN Enforcement

Access Control Lists (L2–4 w/ Redirect), Port Security

Dynamic ARP inspection, IP Source Guard, DHCP Snooping

Provisioning Automated vSwitch Config, Port Profiles, Virtual Center Integration

Optimized NIC Teaming with Virtual Port Channel – Host Mode

Visibility VMotion Tracking, NetFlow v.9 w/ NDE, CDP v.2

VM-Level Interface Statistics

Policy-based SPAN & ERSPAN

Management Virtual Center VM Provisioning, Cisco Network Provisioning, CiscoWorks

Cisco CLI, Radius, TACACs, Syslog, SNMP (v.1, 2, 3)

Hitless upgrade

*In 1.4 Release, 4Q CY2010


Virtualization & Cloud Driving New Requirements in Data Center

VDC-1

VDC-2Hypervisor

App

OSApp

OSApp

OS

Dedicated Network Services

Firewall SLB/ADC WAN Opt

Virtual Service Nodes (VSNs)• Virtual appliance form factor• Dynamic Instantiation/Provisioning• Service transparent to VM mobility• Support scale-out• Large scale multi-tenant operation

• Application-specific services• Form factors:

• Appliance• Switch module

Virtual Network Services


Hypervisor

Traditional Service Nodes

Virtual Contexts

Deployment options for Virtual Services

VLANs

Hypervisor

Redirect VM traffic via VLANsto external (physical) firewall1

AppServer

DatabaseServer

WebServer

Apply hypervisor-based Virtual Firewall2

AppServer

DatabaseServer

WebServer

VSN

Virtual Service Nodes

VSN


Example Use Case: 3-tier Server Zones

WebServerWeb

Server

AppServerApp

Server

DBserverDB

server

Port 80 (HTTP)and 443 (HTTPS)of Web Serversopen

Only Port 22 (SSH) of App Servers open

All other traffic denied

Only Permit Web Servers access to App servers via HTTP/HTTPS

Only Permit Appservers access to DB servers

Tenant_A


Virtual NetworkManagement

Center(VNMC)

Introducing Cisco Virtual Security Gateway

VM context aware rulesContext aware SecurityContext aware Security

Establish zones of trustZone based ControlsZone based Controls

Policies follow vMotionDynamic, AgileDynamic, Agile

Efficient, Fast, Scale-out SWBest-in-class ArchitectureBest-in-class Architecture

Security team manages securityNon-Disruptive OperationsNon-Disruptive Operations

Central mgmt, scalable deployment, multi-tenancy

Policy Based AdministrationPolicy Based Administration

Virtual SecurityGateway

(VSG)

XML API, security profilesDesigned for AutomationDesigned for Automation


Virtual Security GatewayLogical deployment like physical appliances

Nexus 1000VDistributed Virtual Switch

VM VM VM

VM VM

VM

VM VM VM

VM

VM

VM VM VM

VM VM VMVM

VM

vPath

VNMC

Log/Audit

VSG

Secure Segmentation(VLAN agnostic)

Efficient Deployment(secure multiple hosts)

Transparent Insertion(topology agnostic) High Availability

Dynamic policy-based provisioning

Mobility aware(policies follow vMotion)


Virtual Security GatewayIntelligent Traffic Steering with vPath


VM VM VM

VM VM

VM

VM VM VM

VM

VM

VM VM VM

VM VM VMVM

VM

vPath

VNMC

Log/AuditInitial Packet Flow

VSG

11Flow Access Control(policy evaluation)

22

DecisionCaching 33

44


Virtual Security GatewayPerformance Acceleration with vPath


VM VM VM

VM VM

VM

VM VM VM

VM

VM

VM VM VM

VM VM VMVM

VM

vPath

Remaining packets from flow

ACL offloaded to Nexus 1000V

(policy enforcement)

VNMC

Log/Audit

VSG


Apply Security at Multiple Levels

Specify zoning policy with the appropriate granularity Tenant VDC vApp

Tenant A Tenant B

VDC vApp

vApp

vSphereNexus 1000VNexus 1000V

vPath


Virtual Network Management Center (VNMC)Seamless Policy-Based VSG Management

Nexus 1000V

vCenter

VNMC

Port Profile

VMContext

Security Profile

VM

VM

VM

VM

VM

VM

VM

VM

VM

ServerTeam

NetworkTeam

SecurityTeam

Management/Orchestration tools

• Centralized mgmt of VSG &security profiles• Security team manages security• Architected for multi-tenancy, RBAC• XML API for automated provisioning


Protect the Endpoint

Host Posture & Event Information

Host IPS

Network IPS

HIPSManagement Center

SDEEHost Posture &

Quarantine Events

VM Guest OS Protection

A host is quarantined manually by an administrator or rule-generated by global correlation

Quarantine events include the reason for the quarantine

the protocol associated with a rule violation (TCP, UDP, or ICMP), an indicator on whether a rule-based violation was associated with an established TCP connection or a UDP session

the IP address of the host to be quarantined.

Host IPS and Integration with Network IPS


Remember…

Security best practices still apply

Limit Data Flow to other servers and resources

Do not use non-persistent disks

Harden the Host OS, Hypervisor, & Guest OS

Use AV, maintain patches and updates

Consider using a HIPS solution


Takeaways

Device VirtualizationScale use of network and security components

Flexible integration options

Can get complicated…plan accordingly

Server VirtualizationSecure virtual machine environment

Use features to maintain visibility

Ensure Separation of Duties is maintained

Don’t do what you wouldn’t do on a physical machine


Additional Resources

Data Center Design Zonehttp://www.cisco.com/go/designzone

security and virtualization in the data center - cisco - · pdf filesecurity and...

Documents