schoolcraft college data center
DESCRIPTION
Schoolcraft College Data Center. Tier 3+ . Title: Design of a Shared Tier III+ Data Center: A Case Study with Design Alternatives and Selection Criteria Abstract: - PowerPoint PPT PresentationTRANSCRIPT
1
Schoolcraft College Data CenterTier 3+
2
Title:Design of a Shared Tier III+ Data Center: A Case Study with Design Alternatives and Selection Criteria
Abstract:Schoolcraft College is constructing a High Availability (HA) Data Center that is targeted at an Uptime Institute Tier Rating of III+. The new center will provide colocation hosting services to the education, municipal, and commercial communities. The design criteria of this facility include substantial redundancies in the power, cooling, network, and security areas. For example, the power infrastructure redundancy includes a continuous-duty natural gas generator that will supplement the DTE public power grid, such that campus power capacity should remain flat even when this 150+ rack facility is at capacity. The data center is also being designed to give students first-hand exposure to the skills needed to design and operate such a high-performance facility, without compromising security or uptime for municipal, commercial, and educational institution customers. The design criteria will be presented, alternative designs discussed, and final selections presented and rationalized.
3
Data Center Colocation Philosophy Space is money Reputation is
success or failure Customer service
must be paramount
PUE drives profitability/cost containment
Design flexibility
4
Concurrently maintainable Redundant Capacity components Multiple independent distribution paths One distribution path required to serve
computer equipment at any time Dual powered IT equipment Twelve hours of on-site engine generator fuel
storage for N capacity
Understanding Uptime Institute Definition of a Tier 3+Data Center
Additional Tier 4 Features• Fault tolerant• N capacity power/cooling available after any infrastructure
failure• Multiple independent active distribution paths
5
100% Uptime Tier 3+ rated data center PUE ↠ 1.5 Minimum of N+1
redundancy in all critical systems
Private power generation 2N back-up power Carrier neutral with multiple
carriers
Design GoalsBasic Co-Location
Space
Power/Cooling
Access to Carriers
Security
And Beyond…
SystemDesign
LeaseEquipment
DisasterRecovery Maintenance
RemoteHands DBMS
OSMgmt.
6
High density power – average 5kw/rack 24/7 monitoring 24/7 client access Latest high efficiency HVAC system Inert gas primary fire suppression system
Design Goals - Continued
Dual authentication physical security with biometrics
7
Power – Design Alternatives CHP Generator vs. Fuel cell Single vs. dual utility feeds # of failover protections Single vs. dual backup
power UPS Battery vs. Flywheel UPS Tier III vs. IV (A+B) Busway vs. conduit/wire Branch circuit monitoring
◦ Panel vs. Busway vs. PDU Transfer switch vs. PLC
failover management On site load bank vs.
Maintenance service
8
Data Center Power Plant
9
Power DeploymentA
BABABABABABABABAB
Power Strip (a.k.a. PDU or
Power Distribution Unit )
APower Busway
Power Busway BusPlug
AB
Single Corded Device
Solid State Transfer Switch
Dual Corded Server(Two Power Supplies)
A B
Back of Rack
1.) From the Row…
2.) To the Rack…
3.) To inside the Rack…
10
Network – Design Alternatives
Entry pathways owned vs. Carrier owned
Single vs. Dual core carrier routers to MDF
Single vs. Dual edge switches (HSRP)
Cisco 6500 series vs. Nexus with SDN
Carrier Battery plant vs. Operator A-B UPS
Single vs. Multiple carriers Single vs. Multiple entry points (Diverse Entry) Single vs. Multiple carrier paths (Diverse Path) Single vs. Dual lateral connection per carrier
11
Data Center Networking Redundancy
12
SCDC and MERIT Network Relationship
Merit will re-sell SCDC colocation services
Merit delivers a 150Mbps into the
Applied Sciences via an AT&T EVC
Merit Networks services will come into the SCDC via AT&T or Level3 “last
mile” with whom they already have a
relationship.Merit has approximately
3000 miles of fiber network in Michigan.
Merit has relationships and fiber access to area
public school districts and universities as
potential SCDC clients
13
Cooling Considerations Cooling 101 – Get air to
front of device to allow device fans to pull air to the back
Cooling is largest “non-IT” power usage
Leakage – Open spaces create air mixing, turbulence, loss of efficiency
Design layout CFD failure mode verified
14
Cooling – Design Alternatives Area/perimeter (blowing up the
balloon) Ducted supply and/or ducted
return Raised floor, In row,
Economizers, etc. Hot aisle vs. Cold aisle
containment DX (Air), Glycol, Chilled Water DX pumped refrigerant with
free cooling below 55 F Heat exchangers
15
N+1 cooling capacity Liebert DSe pumped
refrigerant with free cooling
CFD Validation Highest efficiency with hot
aisle containment Typical PUE = 2.0
Cooling
Hot exhaust air
Cold Supply Air
PUE = Power to RacksTotal D. C.
Power
16
Security 24/7 Alarm active An MCOLES certified PA330 Police
Authority, co-located in the Data Center building
Dual authentication with Prox card and Biometrics
24/7 monitored (…by third party) CCTV mega pixel security cameras
with remote viewing Motion activated video record with
90 day retention minimum Non-Clients/Vendors 100% escorted
17
EPO - Emergency Power Off Building vs. Room UPS vs. Feeder General vs. First responder
activation Fire Suppression Activation Code requirements…
Equipment servicing room CRAC’s and IT equipment vs.
CRAC’s only EPO
◦ First responder only◦ Equipment servicing room –
CRAC’s – Agent effectiveness◦ IT Equipment power – optional◦ Power for lighting & utility outlets
Disconnection Means (Emergency Power Off) Section 645.10 of the 2008 NEC requires that there be disconnecting means for each zone in the IT room. Section 645.10 of the 2011 NEC has two alternatives for the disconnecting means, (A) covers remote disconnect controls with requirements the same as the 2008 NEC and (B) covers critical operations data systems. Critical operations data systems (defined in 645.2) are permitted to have alternate disconnecting means provided that five additional conditions are met: (1) An approved shut down procedure has
been established (2) Qualified personnel are continuously
available 24/7 (3) Smoke sensors are in place. (4) A fire suppression system is in place.(5) Plenum cables are used for signaling
NEC Article 645 - B
18
Fire Suppression New evaporative
particulate Inert gas FM200/ECARO –
dual detector 165 Dry pipe – Dual action -
185◦ 2 detector active to charge
lines◦ Pellet melt water zone
First Responder Training
19
Monitoring Preventative vs. Reactive How much – Granular view vs. Sensory Overload Methods & Protocols
◦ SNMP ◦ BACnet◦ Mod bus◦ Dry Contact
Alerting◦ email ◦ text ◦ phone call◦ audible alarms
Response Policy Infrastructure HW vs. Network
20
Monitoring Considerations Critical to Uptime and PUE Hand in Hand with Redundancy
SNMP BACnet Modules BMS DCIM
Transfer Switch
Facility Power Meter
PUE = Power to RacksTotal D. C.
Power
21
Management Control Points Policy Compliance – SSAE16 SOC2,
HIPAA ~100 Control policies with Quality
Control Repository Operations guide Risk Analysis & Mitigation Plan xx
points Disaster Recovery Plan – First
Responder Guide Employee handbook DCIM & Asset management Incident management & Ticketing
System times
22
Preventative Service-effecting or Non-service-effecting Notification of Clients (2-3 weeks in advance) Network and compute redundancy and DR CRAC’s & Condensers Primary Transformer Generator – Switchgear UPS – Wrap around maintenance bypass Breakers (ARC Flash) & Coordination Fire Suppression & EPO Transfer switches & Control logic
Maintenance
23
Data Center Footprint
24
Academic Program focus:◦ Data Center Design ◦ Operation Management
Continuing Education Seminars Teaching lab in data center for hands-on
learning Lab sponsorships being sought from EMC,
CISCO, HP, Dell, etc. Focus on latest offerings/technologies
Data Center Academic Programs
25
A SSAE16 SOC2, HIPAA, PCI compliant facility
Superior Infrastructure Superior Redundancy Superior Power Security Expertise in Commercial Data Center Design
and Management
Data Center Design Benefits
26
Data Center Networking Redundancy
MDF = Main Distribution FacilityLGX = Light Guide Cross-connect – CarriersBGPHSRP
Add Vblock schematic here
27
28
DTE Substation Supplying Data Center
29
Data Center
30
Caging Purpose vs. Necessity
◦ Access control – individually locked racks◦ Layers of security
Styles◦ None ◦ traditional ◦ wrap ◦ Individually locked racks◦ impact on floor plan
31
Rack Management AUP – Acceptable Use Policy
◦ Velcro vs Zip tie◦ Cable labeling◦ Blanking plates◦ Fan direction
32
Network & Cloud A virtualized MDF
Infrastructure◦ Carrier Neutral◦ BGP Carrier Redundancy◦ Edge Router Redundancy◦ Multi-tenant Cloud
Services◦ Multiple Carriers Services
Delivery Modes◦ Ready for BUaaS, BCaaS,
IaaS, etc.Client RackClient
Rack
Client RackClient
RackClient Rack
LGXCarrierRacks
CarrierFiberPlant
Data Center MDF
Edge and Carrier Protected
Carrier Protected
ASR/ISR
Nexus 5K/7k Nexus 5K/7k
UCSBlade
Chassis CLOUD
UCSBladeSAN
Converged Network and Compute
ASR/ISR
Edge #1 Edge #2
Fiber Intercon. Fiber Intercon.
1 2 3 4
Chassis CLOUD
Carrier Direct
33
Colocation services focused on power, space, bandwidth, and physical security
Flexibility – Independently variable levels of space, power, bandwidth, cooling, etc.
Environmental controls Security Carrier neutral/multiple
carriers
Data Center Provides