big disasters

Download Big disasters

Post on 26-Jan-2015

104 views

Category:

Technology

1 download

Embed Size (px)

DESCRIPTION

Learn how to fail from the experts.

TRANSCRIPT

  • 1. Big DisastersLearn how to Fail from Worlds Top ExpertsGwen Shapira, Senior Consultant

2. Lessons from Really BigDisasters. 2012 Pythian 3. Swiss Cheese ModelFor a catastrophic error to occur, the holesneed to align for each step in the process.Sometimes this is the right models.Other times the causes are fairly simple.Sometimes there is a Swiss CheeseIllusion Many theories about cause makeit seem like there are many causes. 2012 Pythian 4. 1986 was aBad Year 2012 Pythian 5. Chernobyl 2012 Pythian 6. Sequence of Events1. Critical test planned for day shift2. Gradual shutdown initiated before day shift3. Test delayed and re-starts at 11PM4. Night shift took over, with no time to prepare5. Due to mistake, power dropped too low for the test6. Attempt to restore power7. Unstable core temperature and coolant flow8. Lots of alarms and emergency signals9. No control rods, coolant close to boiling.10. and the test began! 2012 Pythian 7. More events1. Turbines shut down and Diesel engines started2. Decreased water flow, increased vapors3. Which causes a positive feedback loop in this reactor4. More steam -> more power -> more heat -> more steam5. Automatic system inserting control rods6. Emergency shutdown initiated7. All rods inserted. Displacing some fluid8. Massive power spike and first explosion 2012 Pythian 8. And there is more!1. Some rods broke and blocked.2. Rise in power, increased temperature, steam buildup3. Last reading on control panel 30GW output4. Probably steam explosion5. Destroying reactor casing and 2000 ton upper plate6. Total water loss caused even higher power output7. Another explosion8. Dispersing radioactive material.9. Graphite fire burning by now10. Inaccurate dosimeters indicate reactor is still working 2012 Pythian 9. Causes Bypass of many procedures Operator errors Operator lack of training Operator lack of experience Non-intuitive reactor design Dangerous reactor design Non-compliance with standards Total belief in in-accurate monitors Disabled safety features 2012 Pythian 10. Challenger 2012 Pythian 11. Sequence of Events Destroyed on minute two of tenth mission Flame leaked from SRB to external fuel tank Damage to tank caused released of hydrogen Pushing hydrogen tank into liquid oxygen tank Resulting in massive explosion Caused by O-Ring Failure Due to unusually low temperatures during lift-off 2012 Pythian 12. Causes NASA organizational culture and decision making are keycause Problem with O-Ring was known Disregarded warnings from engineers O-Ring not certified for low temperatures No test data for these conditions Customer intimidation Lack of clarity in information presentation 2012 Pythian 13. K219 2012 Pythian 14. Sequence of Events K219 was patrolling near Bermuda Seal in missile hatch failed and water went in Causing poison gas, explosion, fire and war-head ejection One missile hatch was already disabled Vessel surfaced. Nuclear reactors shut down. One seaman died while securing reactor Towing attempts unsuccessful Poison gas leaks Captain evacuates ship against orders Submarine sunk. Maybe on purpose. 2012 Pythian 15. Admiral Nakhimov 2012 Pythian 16. Sequence of Events Passenger ship Minutes into voyage, pilot noticed collision course withbulk carrier Radioed warning. Answer: Dont worry. We will take care of everything. Carrier didnt take care of anything Kept radioing the carrier Eventually both carrier and Admiral Nakhimov changedcourse. Hard. Too late. Unofficial root cause: Both captains were drunk. 2012 Pythian 17. Mikhail Lermontov 2012 Pythian 18. Sequence of Events Left Picton, Australia toward Marlborough Sounds Experienced Australian Captain Who believed Cape Jackson was twice its real width And that there are no dangerous rocks And that he doesnt need a chart So he made last minute decision to go through thepassage Despite advice from officers Hit rocks, water poured in. Ship was beached and eventually sank from damage One crew member died. Passengers rescued. 2012 Pythian 19. Ufa 2012 Pythian 20. Sequence of Events Engineers noticed drop of pressure in gas pipeline To solve the problem, pressure was increased No additional checks or analysis was done Leaked gas formed a flammable cloud Ignited by two passenger trains passing through Estimated explosion of 200 to 10,000 tons of TNT 575 dead, 800 injured Monitoring by Robot Pigs was added after the disaster todetect leaks. 2012 Pythian 21. Bhopal 2012 Pythian 22. Sequence of Events History of leaks in plant since 1979. Many events 1982-1984. Warning by engineers never reached management Safety systems not functioning Tank contained more MIC than regulation allowed During night, water entered the tank Exothermic reaction. Pressure was vented Releasing poison gas No consensus on how water entered the tank 2012 Pythian 23. Top Tips to Avoid Disasters1. Avoid being the USSR2. Communicate. Over-communicate.3. If your engineers say there is a problem There is a problem.4. Fix all issues ASAP5. Never ignore almost accidents6. Never ignore monitors7. Always troubleshoot8. Follow processes and procedures9. Escalate to the most qualified employees ASAP10. Have a DR plan. Many of them. 2012 Pythian 24. Thank you and Q&ATo contact ussales@pythian.com1-866-PYTHIANTo follow ushttp://www.pythian.com/news/http://www.facebook.com/pages/The-Pythian-Group/http://twitter.com/pythianhttp://www.linkedin.com/company/pythian 2012 Pythian