garr-x progress: a new step forward on garr optical network€¦ · · 2014-09-17garr-x progress:...

Garr-X Progress: a new step forward on

GARR optical network

8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th

[email protected]

Gloria Vuagnin Paolo Bolletta

Massimo Carboni (GARR)

OUTLINE

GARR the dream (i.e. the overall design) 2012: GARR-X - the basic national optical network

footprint Transport technology Lesson learned

2014: GARR-X Progress - extension in South Italy Main objectives Footprints Transport technology Timing

Integration and future what next ?

8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th #2


Design optical fibre footprint

Blue color: implemented at first phase (2012)

Gray color: not implemented at first phase

Italy Shape does not help to have uniform coverage

Network topology mesh Good/acceptable in Nord-

West and Central

Poor in Nord-East, South, Insular



GARR-X Network (phase1-2012)

Backbone Fibres

Some integration with leased lines

Access 10G (co-located)

10M->1G Leverage on MANs

Leased fibres

Leased lines


1

1

2

) )

#4


Total number of ports: Network boards: 155 ND2 (2x10G) + 10 NS3 (1x40G)

Client boards: 41 TOA (8x multirate; 328 ports but 242 sfp) + 63 TQX (4x 10G; 253 ports but 232 xfp)

GARR-X Transmission Equipment

Total number of nodes: 81 32 x ROADM; 23 x OTN 49 x Amplifier 1R (OLA);

Average span length: 71 Km Longest span ~ 130 km (BoFi)

BER target value: 10-16 post-eFEC Pre-FEC BER @ 10G < 10-15 Pre-FEC BER @ 40G < 10-9

No-Coherent 10G/40G DCM Upgrade 100G Coherent

ROADM OSN8800 T32 Subrack

OLA OSN6800

3 6-degree node

3 5-degree node

5 3-degree node

15 2-degree node

4 1-degree node



Huawei solution: overall good, however…

Robust and flexible technology, however: To grow in speed is a cost

Substantial overhead to 100Gb/s evolution Lack to understand our point of view

Operations Time Consuming Non-optimal optical power adjustment Restoration implies constant tuning activity

WSON: works perfectly when check on rerouting path is done periodically

Raman pumps: to handle with care Fibres cuts, Reflection Sensitivity, ….

Too Long Delivery Time Low level skills in Local support team Professional skills for complex task ONLY on Chinese experts

Fibre footprint …. Limited effect coming from fibre mixture G.652/G.655

Guard band (10/40/100)


Learned Lesson

#6


1

1

2

) )

1

1

2

) )

GARR-X & GARR-X Progress (2015)


GARR-X Progress is the extension of GARR-X in the South of Italy

GRANT fund from Italian R&E Ministry to remove the digital divide in the South

Network infrastructure

ITC (end user’s services)

Training

Constrain: expenditure before March 2015

#7


GARR-X Progress

A shared High Speed Network for the community of: Schools

Universities

Research

Applications to delivery widespread IT services to the R&E Community


Project Requirements

Physical Layer

Circuit Layer

Packet Layer

Users

e-Infrastructure

Service Community

#8


GARR-X Progress Pillars

Fibre for everyone Backbone

Access

Schools

High Speed Transport Network DWDM Coherent vs Terabit Network Solution

Superchannel P-t-P with services at: 10G/40G/100G

OTN Restoration

A2B: Traffic Matrix @ 10G/40G/100G

A2A: Traffic Matrix @ 10G/100G

Access on fibre Schools 100M on 1G link

Access 1G over 10G link

High End 10G over low expensive DWDM solution

ICT (end user’s services) Distributed Data Center 5 PoPs

High level Service: IaaS / PaaS / SaaS

End User Service: GARRBox (on



Planned Initial Capacity

Banda Ag Ba1 Br Cs Ct1 Cz Fg Le Maz Me Na1 Na2 Na6 Pa1 Rc Rg Sr Ta Tp A->B Ag 0 0 0 10 0 0 0 0 0 0 0 10 0 0 0 0 0 20

Ba1 20 0 280 10 10 20 0 20 360 40 0 180 0 0 0 10 0 950 Br 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 40 Cs 40 0 0 0 0 0 40 0 0 0 0 0 0 0 0 80 Ct1 0 0 0 10 0 320 0 0 180 0 10 30 0 10 890 Cz 0 0 0 0 10 0 0 0 0 0 0 0 0 20 Fg 0 0 0 10 0 0 0 0 0 0 0 0 20 Le 0 0 20 0 0 0 0 0 0 0 0 40

Maz 0 0 0 0 10 0 0 0 0 0 20 Me 0 0 0 20 0 0 0 0 0 40 Na1 40 20 0 10 0 0 10 0 860 Na2 20 0 0 0 0 0 0 100 Na6 0 40 Pa1 10 10 10 0 10 440 Rc 0 0 0 0 20 Rg 0 0 0 20 Sr 0 0 40 Ta 0 20 Tp 20 3680



Na1 Na2

Na6

Fg1

Ba1

Ta1

Cz

Rc

Me

Pa1

Tp

Maz

Ct1

Sr

Rg Ag

Cs1 Cs

Br

Le

Fg

The Progress Transmission Network

8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th [email protected] #11


• Tributary Modules (xTM) support 10 x TIM slots

• TIM modules for client interfaces: Sub 10 G, 10G, 40G & 100G

• <10G service through sub 10G TIM and aggregation shelf

• PIC-based 500Gb/s DWDM line modules (AOLx-500)

• 5 x 100Gb/s with FlexCoherentmodulation (QPSK, BPSK, eBPSK)

• ULH reach up to 2800km+

• Up to 5Tb/s switching per bay

• Multiple switching options:

• OTN muxing/switching (OXM)

• Hybrid packet/OTN (HXM)

• Ten (10) universal I/O Slots

• Mix & Match Trib / Line Modules

Universal Slots

Universal Slots

XTC Chassis

Tributary Modules

DWDM Line Modules

Switch Modules

5 x 10G

1 x 40G

1 x 100G

Switch Slots

16 x 2.5G

Swit

ch S

lots

Universal Slots

Universal Slots

Infinera Solution


16-OCG @500Gbps - 8Tbps max capacity - 100Gb/s client

Capacity model: pay as you go Line 500Gb/s p-2-p Client 10G/40G/100G

#12


The IP/MPLS (2012)

8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th [email protected]

SS CA1

VE-2

MI2

BO1

RM2

TO1

FRA

PEERING NATIONAL

PEERING RESEARCH

GLOBAL INTERNET

MI1

PD2

PD1 MI2 MI1

MI3 TN TS1 PV

PI1

GE

RM1

RM4

PG

AQ

BO3

BO1

FE

UR

AN

FI1

RM2

MI1 MI2

BO1 RM2

PA1 CT1 NA1

ME1 SA CS BR LE PZ MT

BA1

T

x100GE x40GE x10GE

Access Core

#13


The IP/MPLS evolution

8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th [email protected]

CT1

CS

PA1

SA BA1

BR LE

FG

NA2

NA1

ME

TA1

NA6

CZ

PA2

ME1

RC DC

DC

PA1

DC DC

T2

T2

T2

NA1 BA1

CT1

x100GE x40GE x10GE

Access Core

#14

Mx960 with @ 100G/40G/10G Mx480 with @ 40G/10G/1G


GARR-X Progress vs GARR-X


Numbers comparison

GARR-X GARR-X Progress

Backbone 6500 km + 3000 km

Access 1500 km + 2000 km

Schools 0 km + 1000 km

WDM Nodes 32 + 21

IP/MPLS nodes 38 + 8

Client Capacity 1 Tbps + 5 Tbps

vCPU (E5-2697v2) 0 +8800

Storage (PB) 0 +10.3

#15


Where we are now

GARR-X Progress Procurement Procedure: [1301] DF 07/13

[1401] DWDM 02/14

[1402] IP 03/14

[1403] ICT 04/14

GARR-X Progress is now on delivery phase: Fibres: delivery at 80%

Routers: before end of November

WDM: end of year

ITC: within this year


… everything in parallel

#17

Massimo Carboni (GARR) 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th [email protected]

The Whole

IP-Network

100G Evolution

‘14 Upgrade

T

x100GE x40GE x10GE

Access Core

120 GB 60 GB

60 GB

60 GB

#18


Integration and Future evolution


from South North

1

1

2 ) )

Mi2 Mi1

CERN

RENATER

ACONET

ARNES

SWITCH

Malta

ALBANIA

GREECE

#19

Massimo Carboni (GARR) 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th

100G Evolution

Full Upgrade

T

x100GE x40GE x10GE

Access Core

[email protected]

The Whole

IP-Network

#20

To do What?


Fibre to the Schools

• Give schools a tool for teaching

• Represent an alternative network (compared with the Operator approach)

• Only High Schools (colocated with the GARR POPs)

• GARR Network Model

• Acquisition of Dark Fibre (IRU 15y)

GARR and Schools

• Only IP

• Demarcation Point Punti di demarcazione del servizio In fase di valutazione

Management

• Symmetric connection @ 100M (1G ready)

• Public IPv4 and IPv6 address space

• No filtering and limitation

Services

Riunione CCR - Roma, Presidenza INFN - Roma - 8/settembre/2014 #22


Schools Aggregation Model


100 Mbps Fibre

100 Mbps Radio

1 Gbps Fibre

10 Gbps Fibre

CPE

Access Node

Aggregation Node

Service Node

Service Edge

100 Mbps

1 Gbps

10 Gbps

Aggregation

Aggregation

Massimo Carboni (GARR) 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th

Cloud Services

KVM Translator VM

VM Repository

Hardware Level

Virtual Disk System

Virtualisation Level

#24

ICT Logical Infrastrutture


ICT Components & Basic Service


Item Quantity Technology Comments

Computing

system

8800

vCPU

Blade systems with

L2/L3 and FC

11 Blades 384 GB Memory x 2 Processors x 12 Cores

x 2 HyperThreading = 512 Virtual CPUs +

FiberChannel and 40 GB Ethernet Switches.

Disk

system

10 PB RAID6 disks in FC 10 PB storage with redundant FC links (4 controllers

x 2 8Gbit/s links each) and servers + FC Switches.

Servizi Base su Cloud Volume

BigDataStorage-Small 100GB

BigDataStorage -Medium 500GB

BigDataStorage -Large 1TB

GARRbox-Small 10GB

GARRbox -Medium 20GB

GARRbox -Large 200GB

Virtual Server vCPU vRAM

(GB)

vStorage

(GB)

vServer-Small 1 2 100

vServer-Medium 2 4 250

vServer-Large 4 8 500

vServer-ExtraLarge 8 16 500

vServer-ExtraLarge-32GB 8 32 500

#25

11 Racks with Power Consumption >15kWh/Rack


App1

CLOUD MONITORING

1

2

3

3

App2

Site A Site B Site C

Cloud Controller


METROLOGY APPLICATION

From Davide Calonico, PhD @ (INRIM)

Optical Division

Atomic clocks accuracy in the last 50 years

First Cs beam clock (1955)

First Cs fountain (1996)

Cs fountain clocks (today)

Best optical clocks (today)

10-8

10-10

10-12

10-14

10-16

10-18

10-20

Satellite techniques (1 day averaging)

Optical links (1 day averaging)

Towards an Italian Optical Fiber Network for high accuracy Time&Frequency Metrology

TORINO, INRIM

MILANO, IFN-CNR

BOLOGNA, INAF

FIRENZE, UNIFI

LENS, INO-CNR

First Step:

INRIM (TORINO) to LENS, (FIRENZE),

650 km optical fiber link,

Operational from late 2013

INAF – MEDICINA RADIOTELESCOPE

INAF – NOTO RADIOTELESCOPE

INAF – SARDINIA RADIOTELESCOPE

MALTA – PRIMARY METROLOGY

ROME-FUCINO, METROLOGY,

SPACE, GALILEO

CNR-LENS-UNIFI ATOMIC PHYSICS

MATERA, SPACE GEODESY

LIFT impact on research

Frejus

Bologna

Radio-astronomy Very Long Baseline Interferometry

Relativistic Geodesy

Atomic and molecular Physics Florence

Matera

Noto

Space Geodesy Center Bepi Colombo

Cagliari

EMRP Project: NEAT-FT (2012-2015)

2014-2015

2015-2016

2014-2015


Conclusions

Managing extra fund in a short time frame is not simple The experience of GARR-X help us to adopt some shortcuts: administrative and technical

From the competition comes new opportunity at reasonable price

Fibre is the solution adopted for the integration To support new applications and new users community

The evolution of the network continue. The next step is fibre for everyone on the same duct Backbone, Access and other users like metrology

In case of schools we need to adopt a planB, the system is not ready

Two Optical layer will be used Access Optical Layer @ low speed with Lambda @ 10G, 40G

Services @ 1G/10G/40G

Backbone Optical Layer @ High Speed with SuperChannel @ 500G, 1T or more Services @ 40G/100G/400G (~ 2017)

The Computing Model of LHC experiments could consider the WAN as sort of distributed LAN (or a multidomain LAN) Esnet show us the requirement of the HEP community Science DMZ

8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th 33

BACKUP SLIDE


Network Comunication


App1

MONITORING

App2

MONITORING

Network Bridge Network Layer Network Layer

Network Virtualization Infrastruttura Hardware (CPU+Storage) Infrastruttura Hardware (CPU+Storage)

Storage Syncronization Storage Layer Storage Layer

garr-x progress: a new step forward on garr optical network€¦ · · 2014-09-17garr-x progress:...

Documents