OUCC 2015Inspiring Innovation

Carleton University: Our Experience Implementing the IPv6 Network Protocol

Mike Milne: mike_milne@carleton.ca

IPv4 Landscape

IPv4 in the Campus

IPv6 Landscape

IPv6 in the Campus

IPv4/IPv6 Landscape

IPv4/IPv6 in the Campus

Other IPv6 Solutions

• Tunnelling – Network to Network• IPv4 networks can be tunneled across an IPv6 backbone• IPv6 networks can be tunneled across an IPv4 backbone

• Tunnelling – Client to Network• ISATAP• Teredo

How did Carleton get Started?

• IANA controls the global allocation of IP addresses• ARIN is the regional registry serving North America• In the spring of 2011, IANA announced it had allocated the last remaining free

IPv4 address space to the regional registries

• In Spring of 2011, I attended the IPv6 Summit

Developing the Plan

• Pros• Greatly increased address space• Elimination of NAT• Easier management of IP address space• Future proofing

• Cons• Potential capital costs• Potential infrastructure incompatibility• Staff knowledge/skills

• Deciding to proceed…

PROS – Increased Address Space

• IPv6 uses 128 bit addresses• 340,282,366,920,938,463,463,374,607,431,768,211,456

• 296 times larger than IPv4 Internet• Carleton has acquired a /48 IPv6 prefix

• 280 IPv6 addresses available for campus use • 1,208,925,819,614,629,174,706,176 Unique Addresses• 248 times larger than the existing IPv4 Internet• Should accommodate growth for the next few years ;-)

PROS – Elimination of NAT

• The existing IPv4 Internet would not exist without NAT• Static one – one translation• Dynamic many – one translation

• NAT Issues• Breaks some applications• Obfuscation/Hiding• Performance hit on translating device

Easier IP Address Layout

• Shortage of IPv4 addresses required VLSM• All IPv6 subnets allocated as /64

• Carleton Prefix is 2620:22:4000::• Subnet allocation is 2620:22:4000:xxxx::• XXXX is the subnet #• Allows for 216 on-campus subnets• Each subnet allows 264 hosts!

PROS – Future Proofing

• WITHOUT QUESTION, IPv6 is the long term solution for IPv4 address space exhaustion• Several client organizations have expressed an interest in IPv6

connectivity• Start now, develop skills and expertise

CONS – Potential Costs

• From the starting point, the potential costs are unknown.• Every device that IPv6 traffic flows through must support IPv6

• Client workstations/servers• Hubs/Switches• Routers• Firewalls• DNS & DHCP services• Traffic shaping devices

• Costs unknown until infrastructure assessment

CONS – Infrastructure Compatibility

• Hubs – should be OK• Switches – Needs to be assessed• Routers – Needs to be assessed

• Support for IPv6 packet forwarding• Routing protocol support (IGP & EGP)• ACLs, static routing/redistribution, policy based routing

• Firewalls – Needs to be assessed• Support for IPv6 packet forwarding & routing• Support for IPv6 rule sets and objects

• DNS – OS support for IPv6 addressing & AAAA records• DHCP - Support for DHCP v6

CONS – Knowledge/Skills

• Staff must build skills to;• Create an IPv6 address plan• Design and implement IGP & EGP routing• Provision firewalls• Provision DNS services• Provision DHCPv6 services• Support ongoing operations

Technical Plan

• Determine scope of deployment• Acquire IPv6 address space from ARIN• Assess infrastructure readiness• Select IGP routing protocol• Design internal IPv6 address space• Select client IPv6 address assignment method• IPv6 enable DNS services• Establish Internet routing• Test with limited deployment• Expand deployment to wider audience

Scope

• The problem as a whole looks impossibly large• Implement client side IPv6 first• Implement IPv6 application services later• Deploy IPv6 only where benefits exist

• Exclusions:• VoIP• Security cameras & video recording, intrusion alarms• Building automation• Point of sale, credit/debit authorization• Stored value payment (Campus Card)

Acquiring IPv6 Public Addresses

• IANA controls IP addressing globally• Regional registry serving North America is ARIN• Carleton University acquired a /48 prefix:• 2620:22:4000::/48• Additional info:

• https://www.arin.net/knowledge/ipv6_info_center.html

Infrastructure Readiness

• 100 % good to go:• (2) Internet routers and (2) DMZ firewalls• (27/28) Campus building/core routers• (4/10) Residence building/core routers• (821/858) Layer 2 access switches• Internal and external DNS servers

Infrastructure Readiness

• Infoblox DHCP servers required OS Upgrade• To support DHCPv6

• In campus and residence buildings where routers did not support IPv6• Did not implement IPv6

• In campus buildings with non-compliant switches;• Did not implement IPv6 on those VLANs

• In residence buildings with non-compliant switches;• Did not implement IPv6 in those houses

Routing Protocols

• Valid IGP protocols supporting IPv6 routing;• RIPnG• EIGRP (Cisco Proprietary)• OSPFv3 (We chose this one)• IS-IS

• EGP Protocols• BGPv4

Design/Layout of IPv6 Addresses• 4350 of 16384 available subnets allocated• Summarization at OSPF area boundaries

OSPF Area Serves Summary First Subnet Last Subnet0 CORE-Core Backbone 2620:22:4000::/56 2620:22:4000::/64 2620:22:4000:ff::/641 Library - Southam Hall 2620:22:4000:100::/56 2620:22:4000:100::/64 2620:22:4000:1ff::/642 Tory Building 2620:22:4000:200::/56 2620:22:4000:200::/64 2620:22:4000:2ff::/643 Azrieli Pavillion - Dunton Tower 2620:22:4000:300::/56 2620:22:4000:300::/64 2620:22:4000:3ff::/644 Life Sciences - St. Patricks 2620:22:4000:400::/56 2620:22:4000:400::/64 2620:22:4000:4ff::/645 Loeb - Patterson 2620:22:4000:500::/56 2620:22:4000:500::/64 2620:22:4000:5ff::/646 HCI-VSIM 2620:22:4000:600::/56 2620:22:4000:600::/64 2620:22:4000:6ff::/647 Robertson - Nesbitt 2620:22:4000:700::/56 2620:22:4000:700::/64 2620:22:4000:7ff::/648 Architecture - Unicentre 2620:22:4000:800::/56 2620:22:4000:800::/64 2620:22:4000:800::/649 Herzberg - Steacie 2620:22:4000:900::/56 2620:22:4000:900::/64 2620:22:4000:9ff::/64

10 Minto - MacKenzie 2620:22:4000:a00::/56 2620:22:4000:a00::/64 2620:22:4000:aff::/6411 River - Canal 2620:22:4000:b00::/56 2620:22:4000:b00::/64 2620:22:4000:bff::/6412 Primary Datacentre 2620:22:4000:c00::/56 2620:22:4000:c00::/64 2620:22:4000:cff::/6413 Secondary Datacentre 2620:22:4000:d00::/56 2620:22:4000:d00::/64 2620:22:4000:d00::/6414 DMZ 2620:22:4000:e00::/56 2620:22:4000:e00::/64 2620:22:4000:eff::/6415 Athletics - CTTC 2620:22:4000:f00::/56 2620:22:4000:f00::/64 2620:22:4000:fff::/6416 Residence 2620:22:4000:1000::/56 2620:22:4000:1000::/64 2620:22:4000:10ff::/64

Assigning IPv6 Addresses

• Same general rules as IPv4• Static

• Generally used for servers• Client devices requiring fixed IP addresses

• Dynamic – Multiple methods exist;• SLAAC• Stateless DHCP• Stateful DHCP

Address Assignment - SLAAC

• IPv6 speaking machines have a new concept• Link Local Address• Valid ONLY on the local network

P:\>ipconfig

Windows IP Configuration

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . : ro-ccsnsg.carleton.ca

Link-local IPv6 Address . . . . . : fe80::75fe:fb38:d531:e70%12

IPv4 Address. . . . . . . . . . . : 134.117.107.134

Subnet Mask . . . . . . . . . . . : 255.255.255.224

Default Gateway . . . . . . . . . : 134.117.107.129

Address Assignment - SLAAC

• Using the link local address, the connected client issues a RS (Router Solicitation) message to IPv6 multicast address FF02::2• The connected router(s) reply with a RA (Router Advertisement)

message• The client learns:• The network prefix and length• The identity of one of more default routers

• Low order 64 bits are auto-generated by EUI-64

SLAAC Issues

• The client is ready to communicate except for;• SLAAC does not identify • DNS servers• Domain name

IPv6 DHCP Options

• Two types• Stateless

• Client learns network prefix & length from RA• Low order 64 bits auto-generated by EUI-64• Client gets “non-address” info from DHCPv6

• IPv6 address(es) of DNS server(s)• Domain name, NTP servers, others…..

• Stateful• All IPv6 parameters learned from DHCPv6

DHCP Options

• SLAAC not a full solution• Stateless DHCP• Client assigns low order 64 bits of host address • Does not allow tracking of client IPv6 address

• Stateful DHCP• Client is assigned full 128 bits of host address from DHCP scope • Allows tracking of client IPv6 address to MAC

Stateful DHCP

DHCP Forwarding

interface Vlan313

description CCS-TestLab

ipv6 address 2620:22:4000:705::/64 eui-64

ipv6 enable

ipv6 nd managed-config-flag

ipv6 dhcp relay destination 2620:22:4000:C90::10

ipv6 dhcp relay destination 2620:22:4000:D47::10

End

IPv6 Privacy Extensions

• EUI-64 Format makes the low order 64 bits of the IPv6 address the same on any network• Uniquely identifying the machine regardless of connection location• Bad for end user privacy

• Most Operating Systems use IPv6 privacy extensions as defined in RFC 4941. • Low order 64 bits of host address are randomized.

Issues with Privacy Extensions

• When using DHCP, we found each machine had THREE IPv6 addresses;• Link local• DHCP learned global unicast• SLAAC derived global unicast with privacy extensions• SLAAC derived address was used by default

• Privacy extensions can be disabled• Alternatively, suppress advertisement of the network prefix in the

RS/RA exchange ipv6 nd prefix 2620:22:4000:705::/64 no-advertise

IPv6 DNS Services

• Internal DNS servers• Should have IPv4 & IPv6 addresses• Have IPv4 & IPv6 Internet connectivity• Have AAAA records defined for all internal IPv6 hosts

• Clients will prefer IPv6 connections if DNS reply shows A and AAAA records for target

P:\>nslookup www.he.net

Non-authoritative answer:Name: he.netAddresses: 2001:470:0:76::2 216.218.186.2Aliases: www.he.net

IPv6 DNS Services

• External (Internet Facing DNS Servers)• Should have IPv4 & IPv6 addresses• Have IPv4 & IPv6 Internet connectivity• Have AAAA records defined for all internal IPv6 hosts

• Remember to change the root registration records

ISP Connectivity

• Single ISP connected organizations can use static routing• Multiple ISP connections require BGP• Two methods exist;• Exchange IPv4 & IPv6 routes over existing IPv4 BGP peering relationship• Two peerings, one for IPv4 one for IPv6• Consult your Internet providers

Validating the Design – GNS2

Execution – IPv6 in the Core• Enable IPv6 on each core-core link• Test using IPv6 ping

Execution: IPv6 Routing in the Core• Enable OSPF routing on all core-core links• Validate correct route propagation

Execution: IPv6 to non-Core Areas

Execution

Execution: IPv6 Internal DNS

• Assign IPv6 addresses to existing internal DNS servers• Enable IPv6 through firewall, open DNS ports• Add IPv6 AAAA records, test with nslookup or dig

Exceution: Client IPv6 Assignment

• Configure DHCPv6 packet forwarding to designated DHCPv6 servers• ipv6 dhcp relay destination <IPv6 Address>• ipv6 nd managed-config-flag• ipv6 nd prefix 2620:22:4000:XXXX::/64 no-advertise

• Create DHCPv6 service and scopes for all networks• Open DHCP ports on the firewall• Test client IPv6 address assignment with DHCPv6

Execution: IPv6 External DNS

• Assign IPv6 addresses to existing external DNS servers• Enable IPv6 through firewall, open DNS ports• Add IPv6 AAAA records, test with nslookup or dig• Change root registration records

Execution: IPv6 to the ISP

• ISP Discussions;• Can you / will you support IPv6 routing?• BGP peering over IPv4 or IPv6?• ISP provides IPv6 addressing for interconnect link

Testing

Execution: Testing

http://speedtest.comcast.net

Execution: Bigger Footprint

• Expand deployment to more internal VLANs• (Lather, Rinse Repeat)

Current Traffic Levels

Problem 1: Wireless

Problem 2: High Router CPU

• Cisco has a hardware acceleration mechanism to speedup routing table lookup• CEF (Cisco Express Forwarding)• With CEF disabled routing table lookups are processed in software vs

hardware• We had three routers that supported CEF for IPv6, but was not enabled by

default• Solved by enabling CEF “ipv6 cef”

Problem 3: High Router CPU

• High CPU utilization caused by IOS process “ipv6 dhcp relay”.• Misbehaving DHCPv6 clients causing DHCPv6 request “storms”

• Consumer grade HP printers• APC Uninterruptable Power Supplies

• Solved by disabling IPv6 on the misbehaving DHCP clients

Security Implications

• NAT is gone, all IPv6 addresses are public• Security provided by NAT must be replaced by other filters

• Look at what you don’t like about your IPv4 security policies and filtering rules• Consider “fixing” these issues with IPv6

Lessons Learned

• Split end-user and server implementations• Implement IPv6 in a limited manner, build your skills & confidence • Optimize your design for reliability and scalability• Carefully consider changes to security policy • There is no deadline, take your time

What’s Next for Carleton

• Enable IPv6 on remaining ISP links• Deployment on Wireless• Elimination of old hardware inhibiting IPv6• Services;• Application services behind the F5• Windows infrastructure services

OUCC 2015Inspiring Innovation

Questions & Answers