Shared Service Delivery in the Shared Service Delivery in the Windy CityWindy City

Patrick ChristianPatrick Christian

University of WisconsinUniversity of Wisconsin

July 17, 2006July 17, 2006

OutlineOutline

What’s CIC?What’s CIC?

Why Chicago?Why Chicago?

CIC Chicago Fiber CollaborationCIC Chicago Fiber Collaboration

CIC OmniPoP CollaborationCIC OmniPoP Collaboration

QuestionsQuestions

What’s CIC?What’s CIC?CIC = Committee on Institutional CooperationCIC = Committee on Institutional CooperationFounded in 1958Founded in 1958CIC membersCIC members– University of ChicagoUniversity of Chicago– University of Illinois (UIUC & UIC)University of Illinois (UIUC & UIC)– Indiana UniversityIndiana University– University of IowaUniversity of Iowa– University of MichiganUniversity of Michigan– Michigan State UniversityMichigan State University– University of MinnesotaUniversity of Minnesota– Northwestern UniversityNorthwestern University– Ohio State UniversityOhio State University– Pennsylvania State UniversityPennsylvania State University– Purdue UniversityPurdue University– University. of Wisconsin - MadisonUniversity. of Wisconsin - Madison

Governed by Provosts (Chief Academic Officers) of member institutionsGoverned by Provosts (Chief Academic Officers) of member institutionshttp://www.cic.uiuc.eduhttp://www.cic.uiuc.edu

Why Chicago?Why Chicago?

Shared Service Delivery in the Shared Service Delivery in the Windy CityWindy City

Why Chicago?Why Chicago?

BOREAS-Net

BOREAS-Net

WiscWaves & BOREAS-Net

MiLR

I-WIRE

& ICCNI-Light

TFN

PSU-Net

Why Chicago?Why Chicago?

Chicago…Chicago…– is approximately the geographic center of all CIC is approximately the geographic center of all CIC

(Big10 + UIC + UC) institutions(Big10 + UIC + UC) institutions– has STARLIGHT (international network exchange)has STARLIGHT (international network exchange)– has both Abilene and NLR nodeshas both Abilene and NLR nodes– has carrier hotels & other buildings where many ISPs has carrier hotels & other buildings where many ISPs

and carriers are accessible & have space for leaseand carriers are accessible & have space for lease– has many long-haul dark fiber providershas many long-haul dark fiber providers– has several local dark fiber providershas several local dark fiber providers– has pretty good beer (next to Wisconsin!)has pretty good beer (next to Wisconsin!)

CIC Fiber CollaborationCIC Fiber Collaboration

Shared Service Delivery in the Shared Service Delivery in the Windy CityWindy City

CIC Fiber CollaborationCIC Fiber CollaborationChicago fiberChicago fiber

Very expensive local loops in downtown ChicagoVery expensive local loops in downtown Chicago– Local loop often a significant fraction (1/4 – Local loop often a significant fraction (1/4 –

1/3) of long-haul cost1/3) of long-haul cost– e.g. Merit was paying $10k/mo for CHI local e.g. Merit was paying $10k/mo for CHI local

loops vs. $35k for a 20yr IRUloops vs. $35k for a 20yr IRU

Multiple places of interestMultiple places of interest

– NLR & NewNet at 111 N. CanalNLR & NewNet at 111 N. Canal

– Other research-flavored stuff at 710 N. Other research-flavored stuff at 710 N. Lakeshore (aka StarLight)Lakeshore (aka StarLight)

– Commodity Internet carriers at Equinix…Commodity Internet carriers at Equinix…

CIC Fiber CollaborationCIC Fiber Collaboration

CIC Fiber CollaborationCIC Fiber Collaboration

Chicago Fiber

CIC Fiber CollaborationCIC Fiber CollaborationFinal project summaryFinal project summary

12 of 13 CIC institutions participated in this 12 of 13 CIC institutions participated in this project + Notre Dame + Wayne State + Meritproject + Notre Dame + Wayne State + Merit104 strands of fiber purchased (52 from Level3 104 strands of fiber purchased (52 from Level3 and 52 from LGN)and 52 from LGN)$30,668 average price per pair of fiber for 20yr $30,668 average price per pair of fiber for 20yr indefeasible right-of-use (IRU) & O&Mindefeasible right-of-use (IRU) & O&MEstimated joint savings of ~$13.2 million vs. Estimated joint savings of ~$13.2 million vs. individual institutional purchases through volume individual institutional purchases through volume pricing, avoidance of legal fee duplication and pricing, avoidance of legal fee duplication and sharing the fiber construction cost for multiple sharing the fiber construction cost for multiple laterals into the same buildinglaterals into the same building

OmniPoP CollaborationOmniPoP Collaboration

Shared Service Delivery in the Shared Service Delivery in the Windy CityWindy City

OmniPoP CollaborationOmniPoP Collaboration

RegionalOptical

Networks

NetworkOperations

Center

24/7/365Production

Support

CommodityInternetService

DirectConnection

Between CICMembers

Internet2Connection

For CICMembers

Colocation Space

For NetworkEquipment

NationalLambda

Rail

CIC FiberIn

Chicago

OmniPoPDrivers

What is an OmniPoP?What is an OmniPoP?

Omni-Omni-– A prefix meaning “all”A prefix meaning “all”

PoPPoP– Three letter acronym Three letter acronym

popular in network popular in network engineering meaning engineering meaning “Point of Presence”“Point of Presence”

– A place where an A place where an organization’s (or organization’s (or several) networking several) networking equipment is presentequipment is present

OmniPoPOmniPoP– A place where all A place where all

types of networking types of networking equipment is presentequipment is present

– A place where all A place where all members are welcome members are welcome to place equipmentto place equipment

– A place to develop a A place to develop a rich mesh of all types rich mesh of all types of services between of services between membersmembers

OmniPoP CollaborationOmniPoP Collaboration

OmniPoP

Colocation Space Network Hardware NOCTechnical Supportand Engineering

Services

Governance

Multiple fiber-rich, friendly locations

Optical switchesEthernet switchesRouters, and more

7/24/365Hands and Eyes

Production operations

OmniPoP ServicesOmniPoP Services

Shared research networking access (NLR, Shared research networking access (NLR, Internet2, Starlight, etc) in ChicagoInternet2, Starlight, etc) in Chicago

Commercial peering (non-transit/transit)Commercial peering (non-transit/transit)

Possible Internet2, NLR links and Possible Internet2, NLR links and collocation space in additional cities collocation space in additional cities

L2/L3 infrastructureL2/L3 infrastructure

Possible future L1Possible future L1

NOC/technical supportNOC/technical support

OmniPoP: Phase IOmniPoP: Phase I

Mod

em

MREN

Participant Network #1

Participant Network #N1G Backup

NLR L3 Service

710 N Lakeshore Dr

International R&E Peers

National Lambda Rail

10 G NLR L3 Service

Fiber provided by CIC Ring participants

1G NLR L2 Natonal Peering Fabric

MgmtSwitch

ConsoleServer

Participant Network #2

National R &E Peers

OmniPoPSwitch

Abilene

StarLight

OmniPoP: Phase IIOmniPoP: Phase II

Mod

em

MREN

Participant Network #1

StarLight

Participant Network #N1G Backup

NLR L3 Service

710 N Lakeshore DrAbilene

Futu

re

Transit

Peering

International R&E Peers

National Lambda Rail

10 G NLR L3 Service

Fiber provided by CIC Ring participants

CommercialInternet

Futur

e Peerin

g

Transit

Peering

1G NLR L2 Natonal Peering Fabric

MgmtSwitch

ConsoleServer

Participant Network #2

National R &E Peers

if possible

OmniPoPSwitch

OmniPoPRouter

CIC OmniPoP: Current ActivitiesCIC OmniPoP: Current Activities

Execution of MOU among CIC institutions for Execution of MOU among CIC institutions for governance/cost sharing/high-level operationsgovernance/cost sharing/high-level operationsSelection of GRNOC as OmniPoP operator Selection of GRNOC as OmniPoP operator Acquisition of a F10 e1200 switchAcquisition of a F10 e1200 switchSept ’06 go liveSept ’06 go live

Questions/Thank YouQuestions/Thank You

GovernanceGovernanceGoverning BoardGoverning Board– One representative from each Partner (initially One representative from each Partner (initially

the participating CIC schools)the participating CIC schools)– Responsible forResponsible for

PoliciesPolicies

Finances, including major expendituresFinances, including major expenditures

Approving new Partners (not limited to CIC schools)Approving new Partners (not limited to CIC schools)

Selecting the OmniPoP operatorSelecting the OmniPoP operator

Requires two-thirds vote to approve all actionsRequires two-thirds vote to approve all actions

– Meets at least annuallyMeets at least annually

GovernanceGovernanceTechnical Advisory Council (TAC)Technical Advisory Council (TAC)– Up to two representatives from each Partner plus Up to two representatives from each Partner plus

representation from the OmniPoP operator & CICrepresentation from the OmniPoP operator & CIC– Responsible forResponsible for

Preparing an annual strategic technical planPreparing an annual strategic technical plan

Recommending an annual budgetRecommending an annual budget

Recommending changes to services offeredRecommending changes to services offered

Approving routine purchases and service contractsApproving routine purchases and service contracts

– May appoint a subset Tactical Technical TeamMay appoint a subset Tactical Technical TeamNot more than six membersNot more than six members

To work with the OmniPoP operator and CIC on day-To work with the OmniPoP operator and CIC on day-to-day technical coordination of operational issuesto-day technical coordination of operational issues