how to use machine learning for testing and …€¢ dynamic networking requires sophisticated...
TRANSCRIPT
1 © Nokia 2016 Public 2017
How to use Machine Learning for Testing and Implementing Optical Networks
Jesse Simsarian, Steve Plote, Marina Thottan, and Peter J. Winzer October 4, 2017
2 © Nokia 2016
1. Introduction • Machine learning applications and algorithms • Network operating system • Network abstractions
2. Sensors • OSNR • Fiber polarization sensing
3. Algorithms • QoT estimation optimization for margin reduction
4. Actions • Error-aware rerouting - SLA to set error tolerance
5. Challenges
Outline
Public 2017
3 © Nokia 2016
• Ubiquitous cloud services will drive bandwidth and dynamism • Computing resources instantiated in minutes • Comparable flexibility presently not possible in IP/Optical networks
• Today: Largely quasi-statically re-configurable components • Flexgrid ROADMs, flexible bitrate transponders, tunable lasers, FlexEthernet, …
• Dynamic networking requires sophisticated Network Operating System (OS) • Manage and allocate resources for optimal network capacity, security and reliability
• Need to be able to physically sense and control the optical network
• Advanced machine learning algorithms on top of the Network OS • Dynamic Networking = Sensors + Abstraction + Algorithms + Actuators
• Communications industry efforts: • Optical Internetworking Forum (OIF): SDN for transport networks
• Open Daylight controller (ODL)
• Open Network OS (ONOS): Open-source NetOS from ON.Lab, Linux Foundation
Introduction
Public 2017
4 © Nokia 2016
Foundations of Machine Learning Applications
Public
Data Center 1
Data Center 2
Flexible, tunable network
Programmable network operating system (netOS)
Machine learning applications IP/Optical grooming
Optical margin
reduction
Optical network defragmentation
sensors actuators
sensors + abstraction + algorithms + actuators algorithms
2017
OSNR Sensor
abstraction
Polarization Sensor
5 © Nokia 2016
T. Mitchell, Machine Learning, McGraw Hill (1997) “A computer program is said to learn
• from experience E • with respect to some class of tasks T • and performance measure P
if its performance at tasks in T, as measured by P, improves with experience E.”
Algorithms Machine Learning
Public
Artificial Neural Networks
Numerical Optimization
https://www.youtube.com/watch?v=qv6UVOQ0F44
2017
MarI/O: Artificial Neural Network
http://www.benfrederickson.com/numerical-optimization/
W2
W3
WN
W1
x > T ?
I1
I2
I3
IN
I1
I2
Ip
…
W11
Wp1
W12
Wp2
W1q
Wpq
…
O1
O2
Oq
…
Σ >
> Σ
Σ >
Inputs Outputs
O
x T
O
6 © Nokia 2016
Dynamic and Flexible Networks
Public 2017
Data Center 1
Data Center 2
Data Center 3
IP Layer
Optical Layer
Flex Ethernet client interface
Bandwidth variable transponders and channels
Flexgrid ROADMs
Disturbance /Impairment
7 © Nokia 2016
WDM Transmission Capacity and Flexible Optical Transponders
Get the Most Capacity out of Your WDM System
5
10
15
2 100 1,000 10,000
Transmission distance [km]
Spe
ctra
l effi
cien
cy [b
/s/H
z]
25
50
75
10
C-b
and
capa
city
[Tb/
s]
PSE
2S (3rd gen. coherent, 2016)
Public
Bell Labs 1 Tb/s Field Trial
400G
250G
200G
100G
2017
8 © Nokia 2016
NetOS: The Network Brain
The network brain is a cognitive global network control plane
Network state
Application needs Network resources capacity
latency
security
reliability
coordinate
schedule
control
2017
9 © Nokia 2016
NetUNIX NetOS Functions
Network Config.
Dynamic intents
Network Resource Manager
Network Events
Network Topology
Performance Data
Intent Framework
Prototype NetUNIX system designed and demonstrated
Continuous testing fram
ework
2017
10 © Nokia 2016
• Advanced algorithms on an vendor-independent, abstract network representation • What information is needed? How many data points are needed? • Joint IP/Optical network grooming and optimization - IP/Optical network topology is needed • Optical margin reduction - optical network OSNR data is needed
• The Internet Engineering Task Force (IETF) • YANG data modeling language to model configuration of network elements
• OpenROADM (AT&T, Ciena, Fujitsu, Nokia, SK Telecom…) • Defines interoperability specifications for ROADMs, transponders, pluggable optics • Includes YANG data models for devices, network, services
• OpenConfig (Google, AT&T, Microsoft, BT, Facebook…) • YANG data models for optical transport: terminal optics, wavelength router, optical amplifier
• OpenFlow 1.4 (Open Networking Foundation) • Common southbound interface definition for multi-vendor network element control
• NetGraph - mathematically-rigorous, expresses multilayer network complexity • Does not replace other efforts, adds explicit layering and mathematical rigor to network model
Public
Data Models, Network Abstraction
2017
11 © Nokia 2016 Public
NetGraph Primitives Underlying Mathematical Model
+
2
Southbound(discover, monitor, program)
Distributed Core(network state management)
Northbound(service intent framework)
Application(e.g., multi-layer network optimization)
a) b)
Multi-Vendor IP/MPLS/Optical Network Elements
primitiveslink
(e.g. fiber)adapter
layer 1
layer 2
1 3
connect
paths
bind
s t s ts t
link
NetGraph
S. Fortune, “Equivalence and generalization in a layered network model”, J. Comput. Syst. Sci., vol. 81, no, 8, pp. 1698–1714, 2015.
2017
Primitives Link
(e.g. fiber) Adapter
Layer 1
Layer 2
Connect
Path
Bind Link
12 © Nokia 2016
NetGraph abstract network representation maps to network resources
Implementation: Adaptation to Network Elements
Public
PXCsOF Switches
DeviceMgr
LinkMgr
FlowMgr
REST/GRPC
1830PSSs
Telnet/SSH
WiFi APs
NokiaNE Proxy
REST
OpenFlow
Device/Link/Host/Flow Providers
HostMgr
NetGraphMgr
Intent Mgr
ResourceMgr
DeviceDrivers
Network-‐level granularity
Device-‐level granularity
Device-‐specificadaptors
Protocols
Event bus
SAM-‐ SOAP
2017
Populate network elements into NetUNIX Gather statistics
Control network elements TCP
REST
Optical Transport
13 © Nokia 2016
1. Introduction • Machine learning
• Network operating system
• Network abstractions
2. Sensors • OSNR
• Fiber polarization sensing 3. Algorithms
• QoT estimation optimization for margin reduction 4. Actions
• Error-aware rerouting - SLA to set error tolerance
5. Challenges
Outline
Public 2017
14 © Nokia 2016
• Single photodiode eye diagram machine learning analysis: • J. Thrane, J. Wass, M. Piels, J. C. M. Diniz, R. Jones, and D. Zibar, “Machine learning techniques for optical performance monitoring from directly
detected PDM-QAM signals,” Journal of Lightwave Technol., vol. 35, no. 4, 2017.
OSNR Sensing with an Artificial Neural Network
Public
Artificial neural network used to estimate OSNR from eye diagram
2017
Measurable Input Output Unknown transformation
Target OSNR Function of eye diagram variance and weights
15 © Nokia 2016
Fiber as a Sensor
Optical fiber network becomes a massively-distributed motion detector
2017 Public
16 © Nokia 2016
OSC for Polarization Sensing – Span Monitoring
OSC SFP
OSC SFP
Direction 1
Direction 2
OA
OA
PBS
Fiber
Fiber
D1
D2
Amplifier Cards
OF
OF
OSC SFP
OF
OF
+-
Processing
+-
OSC data channel
Polarization monitor signal
2017 Public
OSC OSC OSC
… Rx
OA OA
PBS
PDTIA
1 mm
PD
TIA
V
H
17 © Nokia 2016
Experiment
Pol.Analyzer
CW Laser
D1
D2PBS
Scope
Hammer
Test Fiber7 cm
Pol.Synth.
Adaptif A3300
Ch. 1
Ch. 2
Trig.
2017 Public
( )( ) scopeArmS SR
VVVVPR ⎟
⎟⎠
⎞⎜⎜⎝
⎛
+
−Δ= −
21
2112, 5.0sin2
3 Results for Comparison
( ) analyzerfullA SRSSSPR 23
22
21
1, 5.0sin2 Δ+Δ+Δ= −
( )( ) analyzerArmA SR
PPPPPR ⎟
⎟⎠
⎞⎜⎜⎝
⎛
+
−Δ= −
21
2112, 5.0sin2
Full analyzer measurement
“2-Arm” analyzer calculation
2-Arm scope measurement
18 © Nokia 2016
Results Summary
2017 Public
58 Fiber Impacts
Figure of Merit
22.058.0
22.062.0
)max()max(
2,
2,
,
2
±=
±=
=
ArmS
ArmA
fullA
Arm
F
F
PRPRF
90% > 0.25
J. E. Simsarian and P. J. Winzer, “Shake Before Break: Per-Span Fiber Sensing with In-Line Polarization Monitoring,” in Proc. OFC 2017, paper M2E.6, 2017.
19 © Nokia 2016
F. Boitier, V. Lemaire, J. Pesic, L. Chavarría, P. Layec, S. Bigo, E. Dutisseuil, “Proactive fiber damage detection in real-time coherent receiver,” ECOC 2017.
Coherent Transponder Polarization Sensing – Path Monitoring
Public
Machine Learning Event Classification
2017
controller
Event classification: Y = f(X)
20 © Nokia 2016
1. Introduction • Machine learning
• Network operating system
• Network abstractions
2. Sensors • OSNR, BER, NGMI
• Fiber polarization sensing 3. Algorithms
• QoT estimation optimization for margin reduction 4. Actions
• Error-aware rerouting - SLA to set error tolerance
5. Challenges
Outline
Public 2017
21 © Nokia 2016
Why and When do We Need to “Learn from Experience” (As opposed to “knowing what we are doing”)
• Good models of the underlying physics are generally best • Subsystem: Chromatic dispersion compensation in a coherent DSP • System: Predicting physical-layer performance through adequate models • Network: Rule-based (x-layer) routing and wavelength assignment
• But: How good / comprehensive is the respective model ? • Hybrid approach: physical model + parameter tuning with machine learning
- A dB in system planning accuracy is as valuable as a dB in coding gain!
FEC threshold
Q fa
ctor
Predicted performance
Wavelength
Actual performance
Margin
Baseline planning tool
FEC threshold
Q fa
ctor
Predicted performance
Actual performance
Margin
Wavelength
Improved planning tool
FEC threshold
Q fa
ctor
Actual performance
Margin
Wavelength
“Supervised learning” (TRX feedback)
2017
22 © Nokia 2016
Optical System Margin Reduction
Public
E. Seve, J. Pesic, C. Delezoide and Y. Pointurier, “Learning process for reducing uncertainties on network parameters and design margins,” in Proc. OFC 2017, paper W4F.6.
2017
Learning process to reduce uncertainties Lifecycle
Re-training
Learning Algorithm
SDN data base
23 © Nokia 2016
1. Introduction • Machine learning
• Network operating system
• Network abstractions
2. Sensors • OSNR, BER, NGMI
• Fiber polarization sensing 3. Algorithms
• QoT estimation optimization for margin reduction 4. Actions
• Error-aware rerouting - SLA to set error tolerance
5. Challenges
Outline
Public 2017
24 © Nokia 2016
SDN Testbed at Bell Labs
Public
• Optical switching inside data center
• PXC MEMS switch
Photonic Crossconnect (PXC)
TOR Switches & Servers
AirFrame POD AirFrame Rack
Executive Briefing Center
IP Transformation Center (IPTC)
• Error-aware networking
• Optical Transport
• Service Routers
• SDN Switches
Service Routers
Optical Transport NetUNIX
Fiber Connectivity
2017
25 © Nokia 2016
Error-Aware Rerouting
Public
Host 1Host 2
Switch 1
Switch 2
Switch 3
Transport 2
Transport 3
Host 3Host 4
Transport 1
Host 1Host 2
Switch 1
Switch 2
Switch 3
Transport 2
Transport 3
Host 3Host 4
Transport 1
Optical Transport
SDN Switch
2017
Regular intent Error-aware intent
26 © Nokia 2016
• Stability and reliability • Event classification probability
• How much data is “enough” as a basis for machine learning? • What data is needed at which layer of the network? (abstraction)
• Machine learning algorithms local to network element for sensor data
• Reliability and operational implications • Human operator confirmation of network operations
• Business structures - IP and optical businesses
• What are operational barriers to more efficient and dynamic network operation?
• Outages and accountability – applications on top of NetOS
Challenges and Discussion Points for Network Operators
Public 2017
*J. Thrane, J. Wass, M. Piels, J. C. M. Diniz, R. Jones, and D. Zibar, “Machine learning techniques for optical performance monitoring from directly detected PDM-QAM signals,” Journal of Lightwave Technol., vol. 35, no. 4, 2017.
Classification of Modulation format
27 © Nokia 2016 Public 2017
• Cloud services as drivers of bandwidth and network dynamism • Dynamic networks = Sensors + Abstraction + Algorithms + Actuators • Machine learning algorithms to identify network events, reduce optical system margins, and optimize
the network • Built on programmable NetOS and flexible network infrastructure
• Different machine learning methods ranging from “black box” artificial neural networks, to event classification, to hybrid methods using a physical model of the system and machine learning
• NetUNIX prototype NetOS • Improve sensing of the physical layer to detect impairment or possible disruption
• Abstract representations of the network layers with NetGraph
• Applications • Impairment aware IP/optical re-routing
• Optical channel monitoring
Conclusions