network field day 10 - juniper networks part 2: qfx10000 architecture
TRANSCRIPT
Copyright © 2015 Juniper Networks, Inc. 1
Juniper QFX10K Use Cases and Architectures Overview of of Juniper’s Data Center Strategy and Use Cases
Doug Hanks Director of Core Architecture, JDI
#NFD10
Copyright © 2015 Juniper Networks, Inc. 2
Blank Slide Title and Content 1-‐Line
Agenda
§ Overview of Juniper DC strategy § What use cases are we focused on § Overview of EVPN-VXLAN architecture § Juniper switching portfolio § Juniper QFX10000 overview
Copyright © 2015 Juniper Networks, Inc. 3
Blank Slide Title and Content 1-‐Line
Data Center Strategy
Open Simple Programmable
§ Multiple architectures § Open protocols § Install your own software
§ Plug and play § Auto Deployment § Analytics
§ Full APIs § Based on Linux § Structured data
Copyright © 2015 Juniper Networks, Inc. 4
Blank Slide Title and Content 1-‐Line
Common Building Blocks
S S
L L L
IP Fabric
L3
S S
L L L
EVPN-VXLAN Fabric
L2/L3
S S
L L L
Junos Fusion for DC
L2/L3
S S
L L L
MC-LAG
L2/L3
QFX10K + QFX5K Fabric Architectures
Traditional IT SaaS / Telco Cloud / Hosting Enterprise ITaaS
Copyright © 2015 Juniper Networks, Inc. 5
Blank Slide Title and Content 1-‐Line
Juniper Data Center Use Cases
IT-as-a-Service Hosting Software-as-a-Service
§ Deliver VM in seconds § Anywhere in the network § Simple network
management § Full integration:
§ VMware § Cloud management § Monitoring tools
§ Tenant separation § High logical scale § High physical scale § Ultra-resilient to failure § Multi-vendor friendly § Standards-based
§ IP Fabric § Rich BGP support § REST APIs § On-box Python § Structured data § Support multiple
programming languages
Copyright © 2015 Juniper Networks, Inc. 6
Blank Slide Title and Content 1-‐Line
IT-as-a-Service
Junos Fusion for Data Center IEEE 802.1BR Ethernet Fabric and JSON
Overlay Networking Juniper Contrail or VMware NSX
Cloud Management OpenStack, CloudStack, or VMware vCloud Suite
End Hosts Servers and Storage. 1/10/25/40/50GbE
QFX10K QFX10K
QFX5K QFX5K QFX5K
Junos Fusion for Data Center
Element Management / Monitoring Junos Network Director / Cloud Analytics Engine
Copyright © 2015 Juniper Networks, Inc. 7
Blank Slide Title and Content 1-‐Line
Hosting Overview
Junos IP Fabric
Controller Juniper Contrail or VMware NSX
Cloud Management OpenStack, CloudStack, or VMware
vCloud Suite
End Hosts Servers and Storage. 1/10/25/40/50GbE
Controller-less Network-Based Tunnels
Junos EVPN-VXLAN Fabric
Fabric Management Junos EVPN-API and OpenClos
QFX10K QFX10K
QFX5K QFX5K QFX5K
Junos IP Fabric or
Junos EVPN-VXLAN Fabric
Element Management / Monitoring Junos Network Director / Cloud Analytics Engine
Copyright © 2015 Juniper Networks, Inc. 8
Blank Slide Title and Content 1-‐Line
Hosting Overview Bare Metal Hosting
Internal Management System (IMS)
Self-Service Portal
VXLAN Fabric Network
BMS BMS BMS
EVPN-API VXLAN Fabric Network
BMS BMS BMS
VXLAN Fabric Network
BMS BMS BMS
VXLAN Fabric Network
BMS BMS BMS
Customer requests new server
IMS finds next server, network devices, IP addressing, ACLs, and other information.
Programmatically provisions server.
Configures VXLAN Fabric based off
IMS request.
Network elements are configured by EVPN-API. Server fully accessible
by customer.
Reports tenant and network statistics to
IMS via API.
Juniper Value
Copyright © 2015 Juniper Networks, Inc. 9
Blank Slide Title and Content 1-‐Line
Software-as-a-Service
Junos IP Fabric Junos MP-BGP: IPv4 and IPv6
Provisioning OpenClos: plug-and-play IP Fabric
Programmability REST APIs, Python, Linux, Chef, Puppet, NETCONF/YANG
End Hosts Servers and Storage. 1/10/25/40/50GbE
QFX10K QFX10K
QFX5K QFX5K QFX5K
Junos IP Fabric
Copyright © 2015 Juniper Networks, Inc. 10
Big Concept Slide
HOSTING AND DCI USE CASE ARCHITECTURE
EVPN-VXLAN
Copyright © 2015 Juniper Networks, Inc. 11
Title and Bullets
DCI Overview
DC2 DC3
DC1
Data Center Interconnect § Exchange data between data centers § Provide L2 and L3 transport § Data separation and isolation § Node and link redundancy
D1-R1
D1-R2
Copyright © 2015 Juniper Networks, Inc. 12
Title and Bullets
DCI Options
QFX10K DC 1
QFX10K DC 2
MX1 MX2
L3VPN-MPLS
EVPN-VXLAN
Option 1
QFX10K DC 1
QFX10K DC 2
MX1 MX2
EVPN-VXLAN
EVPN-VXLAN
Option 3 § Existing WAN § MX ASBR § OTT DCI (Internet) § Like Option B
QFX10K DC 1
QFX10K DC 2
MX1 MX2
EVPN-MPLS
EVPN-VXLAN
Option 2 § EVPN stitching § Requires planning § MPLS Option A
§ Existing MPLS § Easy implementation § OTT DCI (L3VPN)
QFX10K DC 1
QFX10K DC 2
EVPN-VXLAN
Option 4 § Direct connect § Easy implementation § No MPLS § OTT DCI (dark fiber)
Copyright © 2015 Juniper Networks, Inc. 13
Blank Slide Title and Content 1-‐Line
OpenClos EVPN-API
EVPN-VXLAN Fabric Overview
S S S S
BMS BMS BMS BMS
L L L L L L L L L L L L L L L L
BMS NAS
Switch
T1 – VRF1 T2 – VRF2
IRB
BD1 – VNI1 BD2 – VNI2 BD1 – VNI3 BD2 – VNI4
EVPN-VXLAN Fabric
Bare Metal Compute and Storage
Copyright © 2015 Juniper Networks, Inc. 14
Title and Bullets
Multi-Tenant DC Options
Spine 1
Spine 2
Leaf 2 Leaf 3 Leaf 4 Leaf 1
Option 1 § 3-stage Clos topology § Small to medium deployment § Generally one BGP design (iBGP)
Spine&1
Spine&2
Leaf&2 Leaf&3 Leaf&4Leaf&1
Spine&1
Spine&2
Leaf&2 Leaf&3 Leaf&4Leaf&1
Fabric1
Fabric2
Option 2 § 5-stage Clos topology § Medium to large deployment § Lots of BGP design options
Copyright © 2015 Juniper Networks, Inc. 15
Title and Bullets
Multi-Tenant DC Fundamentals
VRF 1 VRF 2
default.vs
VN
ID 1
VN
ID 2
VN
ID 3
VN
ID 4
IRB IFD
BD3 BD4 BD1 BD2
Spine Leaf
VRF_1_VS
BD1 BD2
VRF_2_VS
BD3 BD4
VTEP – bound to lo0.0
VTEP – bound to lo0.0
irb.1 irb.2 irb.3 irb.4
Copyright © 2015 Juniper Networks, Inc. 16
Title and Bullets
Load Balancing Hierarchy Spine1 Spine2
Leaf2 Leaf3 Leaf4 Leaf1
Host2
ESI
Host1
ESI
m n o p
Host2 ⇛ (o, p) VP-LAG (Spine1, Spine2) L3 ECMP (L1, L2) L2 ECMP
L3 LAGs L1, L2
§ VP-LAG § L3 ECMP § L2 ECMP
Copyright © 2015 Juniper Networks, Inc. 17
Blank Slide Title and Content 1-‐Line
The Problem
S S S S
L L L L L L L L L L L L L L L L
3-Stage Clos Spine and Leaf
L L L L L L L L
A A A A A A A A A A A A A A A A
S S S S
EVPN-VXLAN Fabric PODs
+ Interface Assignments § IP addressing § Loopback addressing § Subnet masks § PTP Links § Server VLAN § RVI assignment
Control Plane § BGP ASN assignments § BGP import policy § BGP export policy § BGP peer group design § BGP next-hop self
VXLAN Configuration § VTEPs § VNIDs § Routing instances § BUM traffic
How can you automate this? OpenClos
Copyright © 2015 Juniper Networks, Inc. 18
Blank Slide Title and Content 1-‐Line
OpenClos Project What is it?
S S S S
L L L L L L L L
S S S S
L L L L L L L L
Create Maintain Monitor
§ Build 3-stage Clos topologies § Build EVPN-VXLAN fabrics
§ Add/remove switches § Add/remove tenants § Tenant networks
§ Optics § BGP Sessions § EVPN Fabric § Tenant Stats
Copyright © 2015 Juniper Networks, Inc. 19
Blank Slide Title and Content 1-‐Line
OpenClos Project How Does it work?
§ Type of Clos network § Size of network § Type of switch used § IP address blocks § BGP ASN blocks
§ Full topology § IP/BGP/lo0 assignments § BGP policies § Junos configuration § ZTP files for bootstrap § Cabling plan § Topology diagram
Input Program/Filter Output
Copyright © 2015 Juniper Networks, Inc. 20
Blank Slide Title and Content 1-‐Line
OpenClos EVPN-VXLAN API
Spine1 Spine2
Leaf2 Leaf3 Leaf4 Leaf1
Host2
ESI
Host1
ESI
m n o p
L3 LAGs L1, L2
OpenC
los
NE
TCO
NF
AP
I
Centrally provision EVPN-VXLAN § Add/remove tenants § Add/remove networks
Aggregated tenant information § Traffic statistics § Host statistics
Integration with Network Director § Wizard for provisioning § Monitoring and information
Copyright © 2015 Juniper Networks, Inc. 21
Big Concept Slide
OVERVIEW
QFX10000
Copyright © 2015 Juniper Networks, Inc. 22
Blank Slide Title and Content 1-‐Line
Leaf Merchant Silicon
Spine Juniper Silicon
Juniper Switching Portfolio
QFX10002-36Q QFX10002-72Q QFX10008 QFX10016
QFX5100-24Q QFX5100-48T QFX5100-48S QFX5100-96S
Juniper + Merchant Silicon Switching Strategy
Copyright © 2015 Juniper Networks, Inc. 23
Blank Slide Title and Content 1-‐Line
Juniper QFX10000 Innovation
Features Scale Technologies
§ Full Ethernet switching § Full IP routing § Full MPLS § Full VXLAN routing
§ 512K LPM FIB § 1M LPM FIB Upgrade § 2M host routes § 100ms buffer per port
§ Tri-Speed Ports § 10GbE § 40GbE § 100GbE
§ Multiple Fabrics § Junos Fusion for DC § IP Fabric § EVPN-VXLAN Fabric § MC-LAG
§ Grandmaster PTP
Copyright © 2015 Juniper Networks, Inc. 24
Blank Slide Title and Content 1-‐Line
Collapsed Edge and Spine
VS
WAN WAN
§ Less devices § Less latency § Less links and optics § Simplified management
Traditional Collapsed
Copyright © 2015 Juniper Networks, Inc. 25
Blank Slide Title and Content 1-‐Line
Juniper QFX10000 Overview
36X40GE 12X100GE 144X10GE
72X40GE 24X100GE 288X10GE
288X40GE 240X100GE 1152X10GE
576X40GE 480X100GE 2304X10GE
QFX10002-36Q QFX10002-72Q QFX10008 QFX10016
MAC 256K 512K 1M 1M
FIB 512 LPM Routes (max 256K IPv4 and max 256K IPv6), XL License for QFX10002: 1M LPM Routes 2M Host Routes
ACL 30K
Latency 2.5us-5us
Delay bandwidth buffer/packet buffer Up to 100 ms/port Up to 100 ms/port Up to 100 ms/port Up to 100 ms/port
Copyright © 2015 Juniper Networks, Inc. 26
Blank Slide Title and Content 1-‐Line
Software Architecture
x86 CPU and Juniper PFE
VM VM
Guest OS
VM
Junos RE 0 Junos RE 1
Linux KVM
Gue
st A
pp
Pyth
on, T
hrift
PFE (Native Linux)
Platform (Native Linux)
Ana
lytic
s
Aut
omat
ion
CLL, XML, JSON, REST APIs, Apache Thrift, Cloud Management
Yocto Linux
Copyright © 2015 Juniper Networks, Inc. 27
Blank Slide Title and Content 1-‐Line
Juniper QFX10002
§ Fixed platform to support transition from 10GE to 100GE § Support for higher scale spines § 36X40GE system for smaller data centers or access applications
Copyright © 2015 Juniper Networks, Inc. 28
Blank Slide Title and Content 1-‐Line
Juniper QFX10002 Port Groups
0 2
1 3 5
4
36 38
37 39 41
40
6 8
7 9 11
10
42 44
43 45 47
46
12 14
13 15 17
16
48 50
49 51
52
18 20
19 21 23
22
54 56
55 57 59
58
24 26
25 27 29
28
60 62
61 63 65
64
30 32
31 33 35
34
66 68
67 69 71
70
53
§ Within a group of 3 ports , one of the port can be used as 100GE port (SR4, LR4) § The ports marked in red become disabled if 100GE is in use within a port group § All ports can be 40GE § All ports can be 4X10GE § Within same port groups different interface speeds cannot be mixed
Port Group Summary
Copyright © 2015 Juniper Networks, Inc. 29
Blank Slide Title and Content 1-‐Line
Juniper QFX10000 Midplane-Less
§ Power and cooling efficiency
§ Reliability
§ Future scale
Copyright © 2015 Juniper Networks, Inc. 30
Blank Slide Title and Content 1-‐Line
Juniper QFX10000 Line Cards
36X40GE with QSFP+ 12X100GE with QSFP28
144X10GE with 4X10GE Breakout
30X100GE with QSFP28 24X40GE with QSFP+, 6X100 GE with QSFP28
100GbE ports
Copyright © 2015 Juniper Networks, Inc. 31
Blank Slide Title and Content 1-‐Line
Juniper QFX10000 Line Cards
§ 60X10GbE + 6QSFP+ § Port group: 2 X100GbE QSFP28 OR 4 x 40GE OR 8 x 10GbE § 84X10GbE with breakout cables § 1GbE support on all SFP+ ports
Ports 0, 3..57 Ports 1, 4..58 Ports 2, 5..59
100 GE ports
Copyright © 2015 Juniper Networks, Inc. 32
Blank Slide Title and Content 1-‐Line
Product Scale Comparison
7308/7316X
Buffer
DC Edge Leaf
Features
Logical Scale
Throughput
Power Efficiency
Latency
MX Juniper Trio
QFX5100 BRCM Trident 2
Spine
QFX10000 Juniper Q5
Copyright © 2015 Juniper Networks, Inc. 33
Blank Slide Title and Content 1-‐Line
Juniper Q5 Chipset
Purposed built silicon for data center
§ 1Tbps chipset
§ 400GbE ready
§ Flexible pipeline for diverse applications
§ External memory for greater scale & delay bandwidth buffer
§ Multi vector scaling (I/O and logical scale)
Copyright © 2015 Juniper Networks, Inc. 34
Blank Slide Title and Content 1-‐Line
Juniper Q5 + Hybrid Memory Cube
§ High bandwidth, energy efficient, high density memory
§ Critical to building a high I/O system
§ 10.2x the bandwidth of DDR3 module
§ 8.5x the bandwidth of DDR4 module
Copyright © 2015 Juniper Networks, Inc. 35
Blank Slide Title and Content 1-‐Line
HMC vs DDR Memory
A"ribute DDR3/4 HMC
Number of Memory Devices 90 and Up 2
Total number of pins between ASIC and memory More than 2400 422
Power 61W 49W Memory surface area 12750mm2 or more 1922mm2
Copyright © 2015 Juniper Networks, Inc. 36
Blank Slide Title and Content 1-‐Line
QFX10000 Summary
§ IP, MPLS, and overlay networking
§ Based on Linux – full REST API for control and data plane
§ Network telemetry and analytics
§ High logical scale
§ Carrier grade reliability and high availability
Thank You!