advanced cooling optimising for immersion… · advanced cooling optimising for immersion rolf...
TRANSCRIPT
![Page 1: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/1.jpg)
![Page 2: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/2.jpg)
Advanced Coo l ing
Opt imis ing fo r immers ion
Rolf Brink/CEO/[email protected]
HPC/Advanced Cooling Track
![Page 3: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/3.jpg)
Copyright © 2018 by Asperitas
+31 88 96 000 00
www.asperitas.com
![Page 4: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/4.jpg)
Copyright © 2018 by Asperitas
IMMERSED COMPUTING®: AIC24
▪ 100% Removal of heat from the IT
▪ Tier 3-4 focus
▪ 22/36 kW or 31-50 kW/m2 (free cooling/chiller)
▪ No airflow, Passive liquid circulation
▪ Intelligence system
▪ Management control and insight
▪ Automatic hydraulic optimisation
▪ Optimised for high density cloud/HPC nodes
▪ Varying servers
▪ Flexible IT hardware
▪ Feed: 12-40°C, upto 55°C / ΔT 2-16°C
![Page 5: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/5.jpg)
Copyright © 2018 by Asperitas
ENCLOSED IMMERSION TECHNOLOGY
▪ Self Sustained
▪ Driven by gravity
▪ Self regulating
▪ Reliable
▪ No moving parts
▪ No oxygen
▪ High heat capacity
▪ Reduced thermal shock
▪ Efficient
▪ IT energy reduction
▪ No chiller requirements
▪ Specialised servers
▪ Liquid certified
▪ Optimised for liquid
▪ SMC Supported
![Page 6: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/6.jpg)
Copyright © 2018 by Asperitas
SENSORS
POWERInput totalper outlet
WATERTemperature in/out
Flow & Pressure
OILTemperature
Level
AIC24 INTEGRATED SAFETY
ASPERITAS INTELLIGENCE OVERVIEW
FAILSAFECloud/HPC
On/off
Open/close
CONTROL
OnOff
Electric Valves
![Page 7: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/7.jpg)
Copyright © 2018 by Asperitas
IT MANAGEMENT INTEGRATION
▪ Server mainboards
▪ Built-in Temperature sensors
▪ Mapped X/Y coordinates
▪ Temperature map template
▪ Data extraction with IPMI (management port)
▪ Management integration
▪ Cassette location awareness
▪ Z-coordinate
▪ 3D location of sensors
Y
X
![Page 8: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/8.jpg)
Copyright © 2018 by Asperitas
REAL-TIME 3D THERMAL ANALYSIS
▪ 1000+ sensors
▪ Integrated temperature
▪ IT temperature readings
▪ Temperature logging
▪ Trend analysis
▪ Fault analysis
▪ Optimisation water circuit
▪ Real-time IT health
▪ Water input control
![Page 9: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/9.jpg)
![Page 10: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/10.jpg)
Copyright © 2018 by Asperitas
SERVER DESIGN
AIR VS LIQUID DESIGN ASPECTS
THERMAL DESIGN GUIDELINES
MATERIAL COMPATIBILITY
CHIP DESIGN ASPECTS
ASPERITAS CERTIFICATION
![Page 11: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/11.jpg)
Copyright © 2018 by Asperitas
AIR VS LIQUID DESIGN
▪ Air based Open Cloud servers
▪ 1 kW/U
▪ Liquid optimised servers (passive)
▪ 1,5 kW/U
![Page 12: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/12.jpg)
Copyright © 2018 by Asperitas
AIR DESIGN VS LIQUID DESIGN
▪ Chassis design for low resistance and open structure
▪ Design for natural flow (most effective for any circulation)
▪ Low flowrate, minimal pressure drop(high flow may damage components)
▪ Using thermal shadow
▪ Thermal shock dampened by liquid
▪ Design for thermal buffer
▪ Temp sensors for thermal environment
▪ Chassis design for stability, strength and closed structure
▪ Design for forced airflow
▪ High flowrate, large pressure drop
▪ Preventing thermal shadow
▪ Thermal shock managed by fans
▪ Reliance on continuous cooling
▪ Temp sensors for component health
![Page 13: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/13.jpg)
Copyright © 2018 by Asperitas
CASSETTE CONSIDERATIONS
▪ Vertical orientation!!!
▪ Fixation in rack (prevent movement/floatation)
▪ Gravity
▪ Serviceability, dry interfaces
▪ Extraction
▪ Chassis dimentions
▪ 19” sub-optimal (fluid dynamics/density combo)
▪ Narrow(er) form factor for dual-board
▪ Unobstructed liquid flow
▪ Sideways outflow on surface
▪ Cooled liquid distribution/levelling on bottom
![Page 14: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/14.jpg)
Copyright © 2018 by Asperitas
THERMAL DESIGN GUIDELINES
▪ Cooling input at bottom (coolest liquid)
▪ Cooling output on top (hottest liquid)
▪ Component placement considerations
1. Thermal tolerance
2. Thermal load (rate of heating)
3. Thermal buffering
4. Fluid dynamics
![Page 15: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/15.jpg)
Copyright © 2018 by Asperitas
SANDWICHED CHIPS (NON-OCP, PATENT PENDING)
▪ GPU sandwiches
▪ Optimised for Asperitas technology
▪ “Thermal pump” within Asperitas Immersed Computing®
▪ Compatibility with GPU and CPU
▪ Geared towards OCP double densities
![Page 16: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/16.jpg)
Copyright © 2018 by Asperitas
EXTREME DENSITY GPU (SANDWICHED THERMAL CASCADE)
▪ Record GPU density▪ 288 GPU
▪ 1 rack space
▪ ½ rack height
▪ More impact▪ More IT power
▪ Less space
▪ Thermal reuse
▪ Datacenter in a box ▪ Deployed in different locations.
▪ Quick, clean & green.
▪ On-going development▪ 5000-10000 to be produced
![Page 17: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/17.jpg)
Copyright © 2018 by Asperitas
LIQUID COMPATIBILITY ASPECTS
▪ Labels, stickers and ink unsuitable
▪ Plasticizers (in cables) are risk factor
▪ Thermal compounds may not dissolve (Indium foil!)
▪ Most hydrocarbons (oil) react with EPDM rubbers
▪ Optical risk
▪ Material discoloration
▪ Liquid interference with non-sealed transceivers
![Page 18: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/18.jpg)
Copyright © 2018 by Asperitas
Chip
TIM Tcase, +/- 80°C
Chip interior 120-145°C
Chip insulation (ceramic)
CHIP DESIGN FOR LIQUID
▪ Current chip designs
▪ 2-dimensional problem approach
▪ Chip packaging poor thermal properties
▪ Electrical insulation=thermal insulation
▪ Liquid optimisation
▪ Could dielectric liquid insulate internal electric circuits?
▪ Cooling rear-side of chips?
▪ Cooling interior of chips directly?
▪ 3D “open” chip design?
▪ Result: Higher (valuable!) temperatures accessible => Reusable heat
Chip
Heatsink
![Page 19: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/19.jpg)
Copyright © 2018 by Asperitas
ASPERITAS CERTIFICATION
▪ Asperitas is NOT manufacturer/(re)seller
▪ Focused on oil immersion (limited chemical)
▪ Asperitas OPEN certification process
▪ Focus on material compatibility R&D
▪ Driven by CFD analysis
▪ Aimed at long term risk assessment
▪ Potentially destructive
▪ 3 certification levels
▪ Level 1, feasibility
▪ Level 2, prototyping & benchmark test
▪ Level 3, 10-week duration and operation test
▪ Level 4, 24 systems production for 3 months
![Page 20: Advanced Cooling Optimising for immersion… · Advanced Cooling Optimising for immersion Rolf Brink/CEO/Asperitas Rolf.Brink@ocproject.net HPC/Advanced Cooling Track](https://reader036.vdocuments.site/reader036/viewer/2022081515/5e89796e029c4c3f8f1184d9/html5/thumbnails/20.jpg)