next gen infrastructure core (ngic) & m-cord · intel labs next gen infrastructure core (ngic)...
TRANSCRIPT
Intel Labs
Next Gen Infrastructure Core (NGIC) & M-CORD
CORD Build Event - November 7-9th, 2017 – San Jose, CA
Christian Maciocco
Principal Engineer, Director of Telecom Systems Research – Intel Labs
Ashok Sunder Rajan, Kannan Babu Ramia, Sai Edupuganti, Jacob Cooper, Karla Saur – Intel Labs
2
Legal Disclaimer
Intel Confidential
• This presentation contains the general insights and opinions of Intel Corporation (“Intel”). The information in this presentation is provided for information only and is not to be relied upon for any other purpose than educational. Use at your own risk! Intel makes no representations or warranties regarding the accuracy or completeness of the information in this presentation. Intel accepts no duty to update this presentation based on more current information. Intel is not liable for any damages, direct or indirect, consequential or otherwise, that may arise, directly or indirectly, from the use or misuse of the information in this presentation.
• Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer.
• No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
• Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and other countries.
• *Other names and brands may be claimed as the property of others.
• © 2017 Intel Corporation.
Intel Labs3
Agenda
•vEPC Background
•Next Generation Infrastructure Core (NGIC)
•CORD High Level View
•NGIC and M-CORD
•Summary / Next Steps
Intel Labs4
SGWMME
vEPC Background (1/2)
S11
S1U
Control Data
PGW
• Operators’ real traffic (San Jose, Houston, Chicago, …)
• Identified system’s bottleneck
• “Understanding Bottlenecks in Virtualizing Cellular Core Network functions”, IEEE LANMAN ‘15
• No independent control or data plane scaling
User Data
”Other names and brands may be claimed as the property of others” User plane capacity reduction as control plane interference increases
Intel Labs5
vEPC Background (2/2)
”Other names and brands may be claimed as the property of others”
IVB
SKT0 HT=1 2x12=24 LCORES
MME HSS
IVB
SKT1 2x12=24 LCORES
PGWS1MME
Mngt IF
S1U SGi
NODE1
S1MME
S1U
1GbE
SW
IVB
SKT0 HT=1 2x12=24 LCORES
IVB
SKT1 2x12=24 LCORES
Mngt IF
S1U SGi
NODE2
10GbE
SW10GbE
SW
IVB
SKT0 HT=1 2x12=24 LCORES
IVB
SKT1 2x12=24 LCORES
Mngt IF
S1U SGi
NODEN
SGi
PktGen1
GTP-U
UDP/IP
PktGen4
GTP-U
UDP/IP
U/L
eNB
RAN Emulator
Signaling & U/L User Plane
Traffic generator
App
Server(Video,
Web, …)
App
Server(Video,
Web, …)
App
Server(Video,
Web, …)
App
Server(Video,
Web, …)
D/L Application Load
Generator
PktGen1
IP
PktGen4
IP
SGW
PGWSGW
PGWSGW
EPC Setup
User Plane Traffic Model
User plane and control plane queue flooding
Intel Labs6
Agenda
•vEPC Background
•Next Generation Infrastructure Core (NGIC)
•CORD High Level View
•NGIC and M-CORD
•Summary / Next Steps
Intel Labs7
What is it ? Why ?
• Research and collaborations to understand telecom workloads behavior on Intel Architecture
• Research to address software/hardware platform improvements
• Released learnings and optimized reference software into open source community (DPDK, CORD)
• S/P-GW, Cuckoo Hash for optimized lookup from collaborations with industry or academic partners
• MME, HSS, PCRF will be released later
• Investigate functionality for new usage models, e.g. Connectionless IOT, Multi-Radio Access Technologies, etc
• NGIC is not a product and will not be a product from Intel
Next Generation Infrastructure Core (NGIC)
https://gerrit.opencord.org/#/admin/projects/ngic
Intel Labs8
SGWMME
Next Generation Infrastructure Core (NGIC)
S11
S1U
Control Data
PGW
• Operators’ real traffic (San Jose, Houston, Chicago, …)
• Identified system’s bottleneck
• “Understanding Bottlenecks in Virtualizing Cellular Core Network functions”, IEEE LANMAN ‘15
• No independent control or data scaling
SW Defined Network (Separation of Control and
Data)
SGW
DataONLY
MME & S/PGW Control ONLY
Control Data
PGW
DataONLY
User Data
• SDN based architecture
• Match/Action semantic data plane
• Independent & scalable control & data
• Functional EPC per operator’s requirements
Traditional EPC Architecture Disaggregated Architecture
User Data
SDN Controller
”Other names and brands may be claimed as the property of others”
Intel Labs
Data Plane9
Functional EPC per Operator’s Requirements - NGIC
SGi
S1U
UL Uplink (UL)
Downlink (DL)
DL
Tables and Policies
Spirent
RAN Emulator
Spirent
Apps Server
GTP-U Decap
Rule Identif.
Bearer Mapping
SDF Metering
Charging IP Forward
IP ForwardGTP-U Encap
DL Data Indic.
Charging Metering(SDF, APN)
Bearer Mapping
Rule Identif.
Control Plane
SDN Controller
MME
Spirent
Sprint
Intel IETF DMM FPC (Forwarding & Policy Control)
ZeroMQ, OF 1.3 previously w/ Vendors Ext.
MMEHSS PCRF
”Other names and brands may be claimed as the property of others”
https://gerrit.opencord.org/#/admin/projects/ngic
SW to be released later
10©2016 Sprint. This information is subject to Sprint policies regarding use and is the property of Sprint and/or its relevant affiliates and may contain restricted, confidential or privileged materials intended for the sole use of the intended recipient. Any review, use, distribution or disclosure is prohibited without authorization.
11/9/2017 2:02 PMVersion 1.0
Linear Scaling Characteristics
0
1000000
2000000
3000000
4000000
5000000
6000000
7000000
8000000
9000000
1 2 3 4
PA
CK
ETS
PER
SEC
ON
D
NUMBER OF PACKET PROCESSINGCORES
Scaling with Number of Cores
50k Subs
250k Subs
500k Subs
• 1 packet processing core• 2.125 Million pps (50k subs)• 1.625 Million pps (500k subs)
• 4 packet processing cores• 8.2 Million pps (50k subs)• 6.5 Million pps (500k subs)
All packet processing cores implement all network functions (PGW, SGW, DPI, Child Protection,CG-NAT, static FW & SFC)
• Additional (overhead) cores for C3PO EPC Data Plane Node• Rx Core• Tx Core• Load-balancing Core • Master Core• Interface Core
• Packet processing cores are in addition to the overhead cores
From Sprint’s Presentation at NFV World Congress, May 2017
Intel Labs11
Agenda
•vEPC Background
•Next Generation Infrastructure Core (NGIC)
•CORD High Level View
•NGIC and M-CORD
•Summary / Next Steps
Enabling the Programmable 5G Edge Cloud
M-CORD
Source: Open Networking Foundation
M-CORD: Enabling the Programmable 5G Edge Cloud
Open Source
Disaggregation
Virtualization
Cloudification
Programmability
PROGRAMMABLE 5G EDGE CLOUD
Source: Open Networking Foundation
Edge Cloud
EPC
Wireless Access Edge Cloud Core
eNodeBRadio
After with SDN & NFV
Before with dedicated hardware
DISAGGREGATE RAN
VIRTUALIZE RAN components in the Cloud
SDN-ize it for PROGRAMMABILITY
Virtualized RAN
Control
DISAGGREGATE Core
VIRTUALIZE all components in the Cloud
SDN-ize it for PROGRAMMABILITY
Virtualized EPC
PROGRAMMATICALLY instantiate slices for different use cases
Source: Open Networking Foundation
M-CORD as the Edge Cloud
RRU + DU
CU-U
CU-c Low
ONOS xRAN
vMME
vHSS
vSGW-U
vSGW-C
vPGW-U
vPGW-C
OpenStack
vPGW-u
vPGW-c
HSS PCRF
vSGW-u
vSGW-c vMME
Core Cloud
PDN
M-CORD Edge Cloud
Source: Open Networking Foundation
XOS
ONOS
vPCRF
M-CORD Usage Models
Virtualized RAN
Control
Virtualized EPC
SD-RAN:
xRAN + ONOS
Commercial Small Cells
CBRS Spectrum
Multi-Radio (MRAT)
WiFi / LTE Steering
WiFi RAN Slicing
Open Source EPC
Infrastructure Acceleration
Multi-Access CORD
Diversification of Hardware Choices
End-to-End Network Slice Stitching
End-to-End Solution PoCs & Capabilities
Mobile Core
RAN and Access
Source: Open Networking Foundation
Intel Labs17
Agenda
•vEPC Background
•Next Generation Infrastructure Core (NGIC)
•CORD High Level View
•NGIC and M-CORD
•Summary / Next Steps
NGIC (vEPC) & M-CORD
https://gerrit.opencord.org/#/admin/projects/ngic
NGIC & M-CORD
Internet
• Millions of Any-G
Connections
• Full Mobility with Traffic
• Call Modeling
• Topology Emulation
• Full virtual
• ETSI Compliant
M-CORD
Compute Networking Storage
Infrastructure Virtualization (Openstack)
(OvS/DPDK based)
ONOS
XOS
S/P-GW-u
(VM)
S/P-GW-c
(VM)MME
(VM)
SGiS1-APS1-U
HSS PCRF
Source: Open Networking Foundation
vLSSpirent Landslide
Virtual
S/P-GW-c
(VM)
S/P-GW-u
(VM)
CP-DP_net
S11_net S1U_net SGI_net
MME
(VM)
Emulated
eNBApp Server
LOGICAL NETWORK CONNECTIVITY IN CORD
vSwitch (OVS / OVS-DPDK)
S/P-GW-uS/P-GW-c
A B C D E F G
Switch
vLS RAN +
MME + App server
Legend:A,D management MANAGEMENT
B,E CP-DP_net PRIVATE
C S11_net ACCESS_AGENT
F S1U_net ACCESS_AGENT
G SGI_net ACCESS_AGENT
PHYSICAL NETWORK CONNECTIVITY
Infrastructure Virtualization (Openstack)
(OvS/DPDK based)
ONOS
XOS
S/P-GW-u
S/P-GW-cMME
HSS PCRF
CORD InstallerCORD-1483
CORD-1484
CORD-1486
networking-onosCORD-1488
nova-computeCORD-1684
CORD-1485
CORD-1842
CORD-1483: Configure Nova with DPDK-enabled flavors
CORD-1484: Add nova config options to juju_config.yml
CORD-1485: Install OVS-DPDK through nova-compute charm
CORD-1486: Set kernel commandline parameters for compute nodes
CORD-1488: Add VIF details to handle VHOST_USER port type.
CORD-1684: Backport libvirt driver changes, to enable memAccess=shared tag required for correct functioning of VHOST_USER
CORD-1842: Replace default QEMU with patched QEMU 2.7.0 for correct functioning of VHOST_USER
M-CORD using OvS/DPDK and Enhanced Platform Awareness
M-CORD using OvS/DPDK and Enhanced Platform Awareness
- Features used to achieved high performance on CORD- OvS/DPDK
- Enhanced Platform Awareness:
- VM memory backed by Huge Pages
- CPU isolation
- Core pinning
- vHost user data path
L2FWD Throughput in CORD w/ OvS/DPDK vs. CORD w/ OvS(Higher is better)
L2 FWD Latency in CORD w/ OvS/DPDK vs. CORD w/ OvS(Lower is better)
Modules: vEPC & M-RAT (1/2)
InternetEmulated eNB
Compute Networking Storage
Infrastructure Virtualization (Openstack)
(OvS/DPDK based)
ONOS
XOS
S/P-GW-u
S/P-GW-cMME
HSS PCRF
M-RAT
client
LTE Wi-Fi
Ctrl
WiFi User Data
LTE User Data
Multi-RAT Data-Plane
Encapsulation &
Tunneling
Sequencing
& Splitting
Concatenation &
Fragmentation
Buffering &
Retransmissions
Multi-RAT Control-Plane
E2E Protocols
Measurements
management
Client / Session
Management
Data-Plane / Traffic
Managements
”Other names and brands may be claimed as the property of others”
Modules: vEPC & M-RAT (2/2)
Intel Labs26
Agenda
•vEPC Background
•Next Generation Infrastructure Core (NGIC)
•CORD High Level View
•NGIC and M-CORD
•Summary / Next Steps
Intel Labs27
Summary / Next Steps
•NGIC is a functional, DPDK based, EPC operating in bare-metal, VMs, or containers and orchestrated in M-CORD
•Additional EPC components, MME, HSS, PCRF, will be made available to CORD over time
Intel Labs28
NGIC Hands-on Tutorial Next
Prerequisites for Hands-on Tutorial
• Install Docker (1.13 or higher) and Docker Compose
• Docker images for NGIC control, data plane and traffic generationdocker pull ngiccorddemo/ngic-cp
docker pull ngiccorddemo/ngic-dp
docker pull ngiccorddemo/ngic-traffic
• Demo foldergit clone https://github.com/ngiccorddemo/cordbuild2017.git
Intel Labs29
Thank You
Intel Confidential – Internal Use Only
Intel Labs31
Backup
Intel Labs32
Match/Action : Optimized Table Lookup with Cuckoo Hashing[Pagh 01]
• “Scalable, High Performance Ethernet Forwarding with CuckooSwitch”, Dong Zhu, Bin Fan, Dave Anderson (CMU), Michael Kaminsky (Intel)
H1(x) H2(x)
X
1
H1(x) H2(x)
X
2
H1(x) H2(x)
X
Y
3
H1(x) H2(x)
X
Y
4
H1(x) H2(x)
X
Y
Z
5
H1(x) H2(x)
X
Y
Z
6
One Insert may move a lot of items especially at high table occupancy.Optimal multi-writer insertion using Intel® TSX
collision
Efficient MATCH/Action Semantic Data Plane (1/2)
Intel Labs33
Traditional hash
0
1
2
3
4
5
6
7
8
9
0% 10% 20% 30% 40% 50% 60% 70% 80%
Inse
rts
/ S
eco
nd
Mil
lio
ns
Table Load
Cuckoo Hash Insert Performance per CPU Core
32M Random
8M Random
2M Random
32M Sequential
15X
0.00
500.00
1,000.00
1,500.00
2,000.00
2,500.00
3,000.00
10 1M 4M
Memory bandwidth vs. # of entries
rte_hash
Cuckoo
Previous hash
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
1M 2M 4M 8M 16M 32M
Throughput vs. # of entries
rte_hash
Cuckoo
Previous hash
Efficient MATCH/Action Semantic Data Plane (2/2)
Modules: vEPC & M-RAT (2/2)