study for fibre-br backbone network architecture

FIBRE-BR Camp, 28-29 April 2012Ouro Preto (MG), Brazil

Study for FIBRE-BR Backbone Network Architecture

Alex S. Moura

1

• Agreement with local telco Oi, brokered by regulatory agency, Anatel

• 3Gbps and 10 Gbps capacities in 24 of 27 state capitals

• Currently no fibre to last 3 capitals. (Maybe in 2013?)

•North: 2 terrestrial links and 1 satellite link

RNP Backbone Physical Topology

22

RNP Backbone Logical Topology

RNP Panorama: http://www.rnp.br/en/traffic/weathermap.php33

https://www.rnp.br/en/traffic/weathermap.php

https://www.rnp.br/en/traffic/weathermap.php

RNP Backbone Core Equipments

Backbone Core Equipments

Juniper MX-480 (25)4 to 15x 10GigE 20x 1GigE

Juniper M10iM10iNx 1GbENx E1

Juniper M7iNx 1GbENx E1Nx STM-1

PoPs: Distribution and access equipments

Brocade Netiron MLX-4

Extreme Networks X-450a

Extreme BlackDiamond 8810

Cisco 7206VXR

Cisco 6509

44

FIBRE Testbed Facility

FIBRE-BR Testbed Service Premises

• Service should be simple to deploy and operate in the environment of PoP operators, and easy to use by project researchers and end users

• Preferably the complexity should be left on the FIBRE-BR side of the island of the demarcation point

• Implementation in RNP backbone PoPs should not require tools or processes that demand significant amount of investment in human resources for its operation

55

FIBRE Testbed Facility

FIBRE-BR Testbed Service Premises (cont.)

• Service will must have maximum bandwidth enforced in RNP Backbone, PoPs networks and in client organizations infrastructures in order to not cause negative impact in production traffic

• Parts of the FIBRE network can be shutdown by RNP NOC operators if needed

• The service may not have infrastructure redundancy (high availability)

• Users must comply with service premises and restrictions

66

RNP Backbone: Infrastructure for FIBRE-BR

Lack of diversity of fibers, lambdas and equipments lead to design of an logical overlay infrastructure

• Allows deployment of DCN service and testbeds

“Virtual backbone” using same equipments of physical backbone that runs the IP network by use of virtualization technology

• Approach used for offer dedicated infrastructure for RNP’s DCN service

• Logical topologies over physical topology

Deployment of virtual backbone made by RNP’s engineering and operations team

77


DF

BA SE

AL

PE

PB-CG

RN

PB-JP

CE

MA PA

PIMS

MT

TO

GO

RO

AC

RR

AM

SP

SCRS

PR

MG

AP

ESRJ

Physical connections and rings

Fiber “virtual” path

•No mul(ple lambdas available

• Transparency to codifica(on and bandwidth

88


DF

BA SE

AL

PE

PB-CG

RN

PB-JP

CE

MA PA

PIMS

MT

TO

GO

RO

AC

RR

AM

SP

SCRS

PR

MG

AP

ESRJλ Routers Layer 3 (IP)Logical Systems

(virtual routers)Layer 2 (circuits)

•No mul(ple lambdas available

• Transparency to codifica(on and bandwidth

9

Physical connections and rings

Fiber “virtual” path

9

FIBRE-BR Topology: Proposal #1

1010


PoP DistributionRouter / Switch



Client Client Client Client Client Client

PoP A PoP B PoP C

Router Router Router

10GE 10GE

GbE GbE GbE

Backbone VLAN Backbone VLANBackbone VLAN

1GE1GEAccessAccessAccessAccess

PoP Access PoP Access PoP Access

1111

12

Proposal for FIBRE-BR Architecture

A 10,000 Feet View

12

FIBRE-BR Experiments requeriments

The FIBRE-BR should allow

• Perform networking experiments in parallel (how many?)

− (There cannot be any interference or conflicts between experiments)

• Each experiment should be allowed to choose resources: servers, interfaces etc.

• Each experiment should be allowed to define the network topology (?)*

• Acess to control of selected resources

• Allow acess to measurements and statistics from the experiment

• Some facility to reproduce each experiment

13

13

FIBRE-BR Backbone requeriments and some questions

• Should allow communication in Layer 2 between 10 islands of resources in geographically dispersed end sites in Brazil

• Some questions:

• How important is the High Availability (HA) for this infrastructure?

• Reminder: backbone links failures do happen.

1414

FIBRE-BR Backbone requeriments and some questions (cont.)

• Some questions (cont.):

• Does the FIBRE-BR should allow to each experiment to “build” its own topology “mapped over” (strict) the physical RNP backbone paths?

• In other words, is there a need to allow each experiment to define it’s own logical network topology “in sync” with the physical backbone topology?

1515

Questions About Experiments Topologies

• Actual CMFs do not allow automatic control to create arbitrary network topologies

• Physical topology (physical backbone) x Logical topology (logical FIBRE backbone)

• Knowledge of the geographic locations of each island wouldn’t be enough for each experiment (at least in the initial project phase)?

• Does the presence of OpenFlow switches in RNP PoPs of the "physical backbone" which does not have connected islands can change any requerement of the physical or logical topology of FIBRE experiments?

16

16

Questions About Experiments Topologies (cont.)

Some facts:

• The offering of automatic definition and configuration of a arbitrary topology for each experiment will demand efforts in development - make changes - in some actual CMF

• Up to where was possible to verify, this capacity is not offered in any testbeds environment in production nowadays (this affirmation can require a confirmation from experts)

1717

Questions About Experiments Topologies (cont.)

Complexity x Pragmatism

• Is it really necessary for the FIBRE to offer automatic arbitrary topology configuration geographically in sync with the physical backbone topology for the experiments since day one?

− What kind of experiments would require this capability?

− To obtain what kind of results?

1818

Proposal for FIBRE-BR Network Architecture

Proposal: Deploy the FIBRE Backbone in phases

• Phase 1 - Deploy a "full mesh" topology using VPLS interconecting all PoPs that connect FIBRE-BR islands

• Phase 2 - Extend Phase 1 topology to an “hybrid topology”, with some circuits with explicitly defined paths over the physical RNP backbone, allowing some “determinism” in part of the FIBRE-BR topology

• Phase ...

• Phase “N” - Deploy a CMF with capability to control all FIBRE and experiments topologies automatically.

1919

Proposal for the FIBRE-BR Backbone

Proposal for Phase 1

Deploy a "full mesh" topology using VPLS to

interconect all FIBRE-BR islands

2020

FIBRE Testbed Facility (cont.)

Option 1

• 1 or 2 dedicated physical 1GigE downlink interfaces from Juniper’s routers to dedicated box (router or switch) in the PoPs that connect FIBRE islands

• Interconnection of all interfaces in 1 or 2 L2 networks using VPLS (MPLS Virtual Private LAN Service)

• The VPLS can be configured in two fashions:

− “Loose”: the interconnections of end nodes using MPLS not necessarily will go through specific paths over the physical layer backbone and can be rerouted dinamically in case of a physical layer topology failure (H.A.)

− “Strict”: the interconnections of end nodes using MPLS will go through explicitly defined paths over the physical layer backbone and can be rerouted dinamically in case of a physical layer topology failure (H.A.)

Option 2

• Use of Juniper JUNOS Logical Systems feature

• 1 or 2 dedicated VPLS Layer 2 networks between logical systems’ interfaces2121


22

Regular routeror switch



Common Island Use Case Common Island Use Case Common Island Use Case

PoP A PoP B PoP C


10GE 10GE

1GE 1GE 1GE

FIBRE MPLSCLOUD

FIBRE MPLSCLOUD

1GE1GE1GE1GE1GE1GE

Experiment VPLS Experiment VPLS Experiment VPLSDedicated physical routerinterface

ISLAND

FIBR

E BA

CKBO

NE


22


23

OpenFlow BoxLegacy FlowRouteFlow




Logical System


10GE 10GE

1GE 1GE 1GE

Logical System Logical SystemFIBRE VMAN FIBRE VMAN

1GE1GE1GE1GE1GE1GE

Experiment VLANs Experiment VLANs Experiment VLANsDedicated physical routerinterface

ISLAND

FIBR

E BA

CKBO

NE

PoP A PoP B PoP C


23


24

CommonRouter or Switch







Logical System


10GE 10GE

1GE 1GE 1GE

Logical System Logical System


FIBRE VMAN FIBRE VMAN

1GE1GE1GE1GE1GE1GE

Experiment VLANs Experiment VLANs Experiment VLANsDedicated physical routerinterface

ISLAND

FIBR

E BA

CKBO

NE

PoP A PoP B PoP C

24

Thank youObrigadoAlex S. Moura

[email protected]

25

mailto:[email protected]

mailto:[email protected]

study for fibre-br backbone network architecture

Technology