BPSDC(Data Center Networks & Cloud

Computing Security)

Lecture 3

Data Center Standars

2

Service Level Agreement

3

Building a Data Center is just a Start …What is Service Level Agreement (SLA)?

An official commitment between the service provider and a clientCan be a legally binding formal or an informal "contract" Originally used by fixed line telco operators from 1980sCommonly includes several components, from a definition of the services to the termination of the agreement

Definition of type of service to be providedThe service's desired performance level (+ reliability and responsiveness)Monitoring process and service level reportingSteps for reporting issues with the serviceResponse and issue resolution time-frameRepercussions for service provider not meeting commitment, especially financial

4

Where and how does SLA apply?

Where are we able to find SLA?Backbone Internet ProvidersWeb services

e.g. the availability of REST API to customersData Centers (both shared, on-premise and outsourced)Cloud computing shared resources SLA

Example SLA (one of the Czech/Itallian Providers)100% uptime for power and cooling99,95% Internet connectivity99,95% physical node availability for Virt. infrastructure Servers99,8% access to provided physical nodes

5

What does SLA not cover?“Higher power” aka “act of God” aka “Force Majeure”

wars, terrorism, strikes, traffic accidents, sometimes also natural disasters (see previous lecture)

Extraordinary interventions to be carried out urgentlyto avoid hazards to safety/stability/confidentiality/integritytypically announced in advance to customers (e.g. 48h before execution when possible or ASAP)

Unavailability or blocking of the infrastructure due toCustomer actions (shutdown of servers, abuse, misconf.)3rd party OS or applications usednon-fulfillment or breach of Contract by customerInternet or connectivity problems caused by customer or 3rd parties

Planned maintenance (normal amount)

6

Data Center Standards

7

A Data Center must follow some …

Guidelines/Best practicesANSI/BICSI 002, Data Center Design and Implementation Best Practices (USA → International)

StandardsTIA 942 (USA)ISO/IEC 24764 → ISO/IEC 11801-5 (Worldwide)EN 50600 series (WiP) + EN 50173-5 (EU)…

Certification requirementsUptime Institute Tier certification (Worldwide)

8

Building Industry Consulting Service International 002 DC Design and Implementation Best Practices (1)

Site selection – hazards, environments, access, regulationsSpace planning – capacity, power, cooling, supporting spaces, IT Equipment placement, network Architectural – design concepts, access paths, planning details, construction componentsStructural – general, specificMechanical – classes, cooling conditions, thermal, mech., …

Electrical systems – utility serv., distribution, mechanical, UPS, standby and Emergency, Automation & Control, Lighting, Protection, …Fire Protection – walls, floors, ceilings, aisle containment, extinguishers, protection, detection, …DC Management and Building Systems – building automation systems, electronic safety and security systems

9

Building Industry Consulting Service International 002 DC Design and Implementation Best Practices (2)

Security – physical s. plan, risks & threats, regulatory & insurance, DC security plan, crime prevention, access control, alarms, barriers, lighting, surveillance, guards, disaster recovery, building site considerations, building shell, DC securityTelecommunications, Cabling, Infrastructure, Pathways, Spaces

C0-C4 Cabling class, topologies, spaces, pathways, access providers

Backbone & horizontal c.Installation, testing, racks

Information Technology – disaster recovery, c. room layout, communication, operations center, network infrastructure reliability, securityCommissioning (+testing) Maintenance (of all systems)Annexes (informative)

Design Process Reliability & AvailabilityAlignment, OutsourcingMulti-DC arch., energy efficiency

10

BICSI 002 – Annex B – Operational Requirements

Operational Level

Annual Planned

DowntimeDescription

0 > 400 hOperational less than 24 hours a day & less than 7 days a week. Scheduled maintenance “down” time available during working hours and off hours.

1 100 – 400 h As above.

2 50 – 99 hOperational up to 24 hours a day, up to 7 days a week, and up to 50 weeks per year. Scheduled maintenance “down” time as above.

3 0 – 49 hFunctions are operational 24 hours a day, 7 days a week for 50 weeks or more. No sch. maintenance “down” time is available during working hours

4 0 hFunctions are operational 24 hours a day, 7 days a week for 52 weeks each year. No scheduled maintenance “down” time is available

11

BICSI 002 – Annex B – Downtime Impact

DescriptionClassification

(Impact)

Local in scope, affecting only a single function or operation, resulting in a minor disruption or delay in achieving non critical ‐organizational objectives

Isolated(Sub-Local)

Local in scope, affecting only a single site, or resulting in a minor disruption or delay in achieving key organizational objectives

Minor(Local)

Regional in scope, affecting a portion of the enterprise or resulting in a moderate disruption or delay in achieving key organizational objectives

Major(Regional)

Multiregional in scope, affecting a major portion of the enterprise or resulting in a major disruption or delay in achieving key organizational objectives

Severe(Multiregional)

Affecting the quality of service delivery across the entire enterprise, or resulting in a significant disruption or delay in achieving key organizational objectives

Catastrophic(Enterprise)

12

BICSI 002 – Annex B – Data Centre Class

Facility Availability ClassesF0/F1 – Single path (maps to T-1, R-1, AC-1)F2 – Redundant components (maps to T-2, R-2, AC-2)F3 – Concurrent Maintainability (maps to T-3, R-3, AC-3)F4 – Fault Tolerant (maps to T-4, R-4, AC-4)

13

BICSI 002 – Annex B – Availability Requirements

Allowable Annual Downtime (minutes)

Allowable Availability (Uptime 9s next lecture)

> 5000 > 99%

500 – 5000 99% 99.9%‐

50 – 500 99.9% 99.99%‐

5 – 50 99.99% 99.999%‐

0 – 5 99.999% 99.9999%‐

14

TIA-942 – Telecommunications Infrastructure Standard for Data Centers (1)

Specifications for DC telecommunications pathways & spacesRecommendations on media & distance restrictions for structured cabling system and applications over it (2005)

Telecommunication spaces and topologiesCabling, pathways, redundancy, Informative annexes: Design, administration, access provider information, equipment plans, dataspace considerations, site selection, tiers, examples, references

Components known from TIA-568Addendum 1 (2008) – usage of 75 Ω coaxial cableAddendum 2 (2010) – additional guidelines for DCs – lighting in 3 tiers, recommendation from CAT-6/6A to CAT-6A only (minimum required category is Cat-6)

15

TIA-942 – Telecommunications Infrastructure Standard for Data Centers (2)

TIA-942-A (2012)harmonization with TIA-568Cleft some limitations to other standards (removed from here)removed 100m limitation for optical fibersmulti-mode cable possible for horizontal & backbone cablinguse of LC & MPO connectors for optical fibersIntroduced Intermediate Distribution Area (IDA) Zone Distribution Area (ZD) can contain only passive componentsenergy efficiency recommendations, harmonized with IEC 24764

TIA-942-A Addendum 1 (2013) – mainly data center fabric topologies examples, new switch topologies

Fat tree, full mesh, inter-connected meshes, Centralized switch, virtual switch

16

TIA-942 – Telecommunications Infrastructure Standard for Data Centers (3)

TIA-942 Revision B (2017)Added Cat-8 cabling, recommended cabling Cat-6A or higherMaximum EDA cable length 10 → 7mat least 1200mm deep cabinets, considerations for 24”+ (600mm+) cabinets, pre-terminated cabling, labeling, cable routing, adding/removing cords, … MPO-16 and MPO-32 connectors for 200G and 400GWideband multimode fiber (WBMMF) cable addedANSI/TIA-568-C.4 coaxial cables and F connectors may be usedNormative references to other standards, including revised references to temperature and humidity guidelinesModifications for use outside of US, optical cable quality req.

17

TIA-942 – Ratings of Data Centres (1)

Rated-1: Basic Site InfrastructureSingle capacity components and a single, non-redundant distribution path serving the computer equipment.

Limited protection against physical events

May not even have a raised floorSusceptible to disruption from planned & unplanned activities

28.8 hours of annual downtime permissible

1 entrance pathway from access provider to facility, single pathway for all cabling

18

TIA-942 – Ratings of Data Centres (2)

Rated-2: Redundant Capacity Component Site InfrastructureRedundant capacity components and a single, non-redundant distribution path serving the computer equipment.

Improved protection against physical events

Does have to use a raised floorSlightly less susceptible to disruptions

22.0 hours of annual downtime permissible

Requirements of Rated-2 must be observed, also 2 entrance pathways from access provider to facility existRouters & switches have redundant power supplies & processors

Vulnerability of service entering building is addressedN+1 redundant UPS modules, single generator

19

TIA-942 – Ratings of Data Centres (3)Rated-3: Concurrently Maintainable Site Infrastructure

Redundant capacity components and multiple independent distribution paths serving the computer equipment (power, data, cooling). N+1 rule for everything.Typically, one single distribution path serves the computer equipment at any time.

Protection against most physical events

The site is concurrently maintainable – each & every capacity component incl. elements which are part of the distribution path, can be removed/replaced/serviced on a planned basis without disrupting the ICT capabilities to the End-User.

1.6 hours of annual downtime

Requirements of Rated-2 must be observed, also requires at least 2 access providers + a secondary entrance roombackbone pathways have to be redundantmultiple routers and switches must be included for redundancy

Vulnerability of a single access provider is addressed

20

TIA-942 – Ratings of Data Centres (4)Rated-4: Fault Tolerant Site Infrastructure

Redundant capacity components and multiple independent distribution paths serving the computer equipment. All redundant capacity components and independent distribution paths are active at the same time. 2(N+1) for all components

Protection against almost all physical events.

The data center allows concurrent maintainability and one fault anywhere in the installation without causing downtime. All computer hardware must have dual power inputsCan sustain at least one worst-case, unplanned failure or event with no critical load impact

0.4 hours (18 minutes) of annual downtime

Requirements of Rated-3 must be observed, alsorequires redundant backbone cabling, which should be in conduit or have interlocking armor, optional secondary distribution area optionally, horizontal cabling is also redundant

Addresses any vulnerability of the cabling infrastructure

21

ISO/IEC 11801-5 – Generic Cabling for Customer Premises – Part 5: Data centers (1)

Latest revision ISO/IEC 11801-5:2017Balanced & optical fibre cabling specifications, normative parts:

Structure of the generic cabling systemChannel performance requirementsLink performance requirements Reference implementationsCable requirementsConnecting hardware requirementsRequirements for cords and jumpersAnnex A - Combination of balanced cabling links

22

ISO/IEC 11801-5 – Generic Cabling for Customer Premises – Part 5: Data centers (2)

Informative Annexes (optional): Usage of high density connecting hardware within optical fibre cablingExamples of structures in accordance with ISO/IEC 11801-5

Data center minimum configurationEnd of Row conceptMiddle of Row conceptTop of Rack conceptEnd of Row and Middle of Row concept with redundancyTop of Rack concept with redundancyEnd of Row and Middle of Row concept with full redundancyTop of Rack concept with (full) redundancy

Examples of networking fabric architectures: fat-tree, full-mesh, interconnected meshes, centralized switch, virtual switch

23

ISO/IEC 11801-5 – Cabling

Cable classes Twisted pair (100 Ω impedance)

Class EA: link/channel up to 500 MHz Cat-6A cable/connectors

Class F: link/channel up to 600 MHz using Cat-7 cable/connectorsClass F

A: link/channel up to 1000 MHz using Category 7A

Class I/II: link/channel up between to 1600 and 2000 MHz using Category 8.1/8.2 cable/connectors2-4 mated connectors per copper channel, RJ-45 or TERA connector

Optical fiber interconnect using multi-mode fibreOM3: Multimode fiber 50µm, min. modal bw of 2000 MHz*km at 850 nmOM4: Multimode fiber 50µm, min. modal bw of 4700 MHz*km at 850 nmOS1/OS2: Single-mode fiber type 1 dB/km / 0.4 dB/km attenuationduplex LC (2 fibers) or MPO (3+ fibers) connector

Channel length is determined by media choice

24

ISO/IEC 11801-5 – Data Centre Topologies

Fat tree without port extenders

Standard 3-tiered architecture

Interconnected meshesFull meshPort extenders

25

EN 50173-5 – IT – Generic cabling systemsPart 5: Data centres

Structure of the generic cabling system in data centres Channel performance in data centres Reference implementations in data centresCable requirements in data centresConnecting hardware requirements in data centres Requirements for cords and jumpers in data centres

26

EN 50600 series – IT – Data centre facilities and infrastructures

EN 50600-1 – General conceptsEN 50600-2-1 – Building constructionEN 50600-2-2 – Power distributionEN 50600-2-3 – Environmental controlEN 50600-2-4 – Telecommunications cabling infrastructureEN 50600-2-5 – Security systemsEN 50600-3-1 – Management and operational informationEN 50600-4-1 – Overview of and general requirements for key performance indicatorsEN 50600-4-2 – Power Usage EffectivenessEN 50600-4-3 – Renewable Energy Factor

27

EN 50600-2-5 Security Systems

Physical security – general, risk assessment Designation of data centre spaces - Protection Classes

Protection Class against unauthorized access Protection Class against fire events igniting within data centre spaces Protection Class against environmental events (other than fire) within data centre spaces Protection Class against environmental events outside the data centre spaces

Systems to prevent unauthorized access Informative Annex – Pressure relief: Additional information

28

EN 50600 – Availability classes

29

EN 50600 – Protection classes