Ethics and EngineeringWELCOME!

71ST CONFERENCE FOR PROTECTIVE RELAY ENGINEERSDR. B. DON RUSSELL

MONDAY, MARCH 26, 2018

Four concepts –four signs of an unhealthy corporate culture1) Normalization of Deviance

2) Proper Management Decision

vs.Proper Engineering Decision

3) Groupthink

4) Whistleblowing

Concept #1:Normalization of Deviance

The tendency to come to consider behavior that varies from either design or good engineering practice as acceptable

What is an everyday example of normalization of deviance?

Concept #2:Proper Engineering Decision orProper Management Decision PED…involves technical matters that

require engineering expertise or judgment or involve ethical standards in engineering codes, especially w.r.t. protecting the public. (AA 191, 1979.)

PMD…involves factors affecting the well-being of the organization anddoes not force engineers to compromise their own technical or ethical standards.

Concept #3:Groupthink “situations in which groups come to agreement

at the expense of critical thinking.” Illusion of invulnerability to failure Us-versus-them attitude Exclusion of external contrary views Illusion of morality Shifting blame to outsiders Self-censorship Illusion of unanimity Pressure to silence dissenters

Concept #4:Whistleblowing Going outside of normal channels,

outside the chain of command, usually to address an immoral, illegal, or dangerous situation

Internal or external Must always be justified. Why? What

steps must be taken beforehand?

Case Study 1: The loss of the Challenger

28 Jan. 198673 sec after liftoff altitude>35,000 ftairspeed>Mach 1.5

Technical Cause: Joint Failure in the Solid Rocket Booster

• Right SRB O-rings failed to seat properly at the joints

• Hot gases leaked through the joint during propellant burn of the rocket motor

• Hot gases burn through the external fuel tank

SRB Seal: Engineering Design & Operation

http://shippai.jst.go.jp/en/

Two Rubber O-rings in ea. joint

1. Growth of SRB segment diameter

2. Clearance between O-ring and upstream channel wall

3. Putty: heat-resistant barrier was added between O-rings and combustion gases

4. Temperature-dependent

SRM cross section

SRB Seal: Effect of Internal pressure on joint rotation

http://shippai.jst.go.jp/en/

Previous warnings

• 1977: Morton-Thiokol discovers joint rotation issue

• 1981: O-ring erosion was found in the 2nd flight

• 1985: Worst O-ring blowby of shuttle flight (Jan.)

• 1985: Thiockol’s Engineer Roger Boisjoly wrote to his VP Bob Lund (July)

“…take immediate action and dedicate a team to solve the problem..” (no mention of temperatures)

Previous warnings (cont’d)• Aug. 1985: Briefings on RBS issues to NASA

Mgmt., who responded:

“safe to continue flying existing design”

“..continue at an accelerated pace to eliminate SRM seal erosion”

• Jan. 27. 1986: Night teleconference: discuss flight readiness review

• Jan. 28, 1986: Challenger exploded!!!

How the flawed launch decision was reached at teleconf. Jan. 27, 1986?

• Roger Boisjoly & his manager, Bob Lund, objected to launch due to concerns of O-rings’ performance at low launch temperature. Lund was told :

“…take off your engineering hat and put on your management hat.”

• Rockwell Engineers: “no launch” due to visible ice buildup in RSB

• The launch review meeting was to ask all teams if their systems were ready for flight. The purpose was reversed when Boisjoly was asked to prove the O-ring seals were not ready for flight

Contributing Causes

(from the Rogers Commission Report)

• “..If the decision makers had known all of the facts, it is highly unlikely that they would have decided to launch…”

• “..propensity of management at Marshall to contain potentially serious problems and to attempt to resolve them internally…”

Normalization of Deviance in the Challenger

“…the tendency to come to consider anomalous behavior to be normal, even though the system was clearly designed not to behave in this way”

1977– 86:• Joint openings becoming “norm” “accepted”• Sealing putty fix in lieu of re-design• Leakage becoming expected, accepted• No engineering data relevant to the flying

conditions

Challenger: Proper Engineering Decision and Proper Management Decision?

• PED: involves technical matters that require engineering expertise or judgment or involve ethical standards in engineering codes, especially w.r.t. protecting public

• PMD: involves factors affecting the well-being of the organization and does not force engineers to compromise their own technical or ethical standards

Challenger: Groupthink

• The dynamics of teamwork and meetings can lead individuals to tend to conform to what they believe is a consensus.

Four Concepts : Signs of an Unhealthy Corporate Culture

1) Normalization of Deviance

2) Proper Management Decision

vs.Engineering Management Decision

3) Groupthink

4) Whistleblowing

Whistleblowing: The harm-preventing view

DeGeorge’s Criteria (p. 159) (1) A practice or product does or will cause

serious harm to individuals or society at large. (2) The charge of wrongdoing has been brought

to the attention of immediate superiors. (3) No appropriate action has been taken to

remedy the wrongdoing. (4) There is documentation of the potentially

harmful practice or defect. (5) There is good reason to believe public

disclosure will avoid the present or prevent similar future wrongdoing.

19

Whistleblowing: The complicity-avoiding view Davis’ Criteria (pp. 160-161) (l) What you will reveal derives from your work for an

organization. (2) You are a voluntary member of that organization. (3) You believe that the organization, though

legitimate, is engaged in serious moral wrongdoing. (4) You believe that your work for that organization

will contribute (more or less directly) to the wrong if (but not only if) you do not publicly reveal what you know.

(5) You are justified in beliefs (3) and (4). (6) Beliefs (3) and (4) are true.

20

Whistleblowing: The autonomy-promoting view A modification of Davis’ criteria (l) What you will reveal derives from your work for an

organization. (2) You are a voluntary member of that organization. (3) You believe that the organization, though

legitimate, is engaged in serious moral wrongdoing. (4) You believe that if you do not publicly reveal

what you know, then this will diminish people’s autonomy by e.g. not making it possible to make informed decisions about what risks to accept.

(5) You are justified in beliefs (3) and (4). (6) Beliefs (3) and (4) are true.

21

Case Study 2: Loss of the Columbia

Columbia, 1 Feb 2003airspeed about Mach 19 altitude about 200,000 ft

Cause: Failure of Thermal Protection SystemBased on CAIB findings

• Insulated foam - from the left bipod ramp -separated from ET 16 days earlier

1. Superheated air to penetrate leading edge insulation

2. Aluminum structure of left wing was progressively melting at re-entry (5000 F jet of plasma)

3. Loss of control & wing from other aerodynamics forces

Orbiter destroyed

Previous warnings

Before 2003

• “Foam Shedding”: reported from previous flights

• Flawed Analysis: staff knew tile monitoring was inadequate, could/did not challenge managers

• “Foam shedding” Reports : not a concern for NASA managers

Missed Opportunities(16-day into flight up to Feb 1)

• Launch video: insulating foam coming off the external liquid propellant tank (81. 7 s into the flight)

• Engineering studies of incident by NASA & contractors

• NASA Engineers’ requests ignored by mgmt.

1. Photographs from CIA, DoD, other agencies2. Spacewalk to examine and photograph damage3. NASA Engineer, Rodney Rocha, led calls for photos4. Requests ignored by mgmt. –no follow ups!

What could have been done if the damage had been known?

• Rescue mission using shuttle Atlantis• Extension of Columbia mission to 30 days• Atlantis : 41 days from scheduled launch• Atlantis : earlier launch to mid Feb. • Transfer Columbia crew to Atlantis via rope from

cargo bay-to-cargo bay• Fly back to earth in Atlantis• Park damaged Columbia in orbit or attempt

unmanned reentry

Normalization of Deviancein the Columbia

“The acceptance of events that are not supposed to happen has been described by sociologist Diane Vaughan as the ‘normalization of deviance.’ ” (CAIB)

CAIB Report-Highlights

• While many changes were made after the loss of the Challenger, the org. factors in the Columbia’s loss were similar, in many ways

• Inadequate deviations from expected performance• Silent Safety Program • Schedule pressure vs. safety • Poor operational and communication processes

• Organizational Cultures are hard to change• Org. culture Strongly influences people actions

Similarities in the Challenger and Columbia cases

Columbia 1.Why did NASA continue to

fly with known foam shedding problems in the years before the accident?

2. Why did the managers conclude the observed foam strike 81.9 seconds into flight was not a threat to the mission, despite concerns of engineers?

Challenger 1.Why did NASA continue to

fly with known O-ring erosion problems in the years before the accident?

2. Why did NASA managers on the eve of the launch decide that launching the mission in such cold temperatures was an acceptable risk, despite the concerns of their engineers?

Similarities: Challenger and Columbia cases

• Roger Boisjoly felt he had to prove it was not safe to launch, rather than only to object because it could not be proven safe.

• “Analysts on the Debris Assessment Team were in the unenviable position of wanting images to more accurately assess damage while simultaneously needing to prove to managers, as a result of their assessment, that there was a need for images in the first place.” (CAIB)

Similarities: the Challenger and Columbia cases

Boisjoly: “I left the room feeling badly defeated, but I felt I really did all I could to stop the launch. I felt personally that management was under a lot of pressure to launch and that they made a very tough decision, but I didn't agree with it.” (Rogers)

Columbia engineers indicated that concerns : “...they would be singled out for possible ridicule by their peers and managers.” (CAIB)

Lessons Learned to Engineers

• Be aware of increasing risks by normalization of deviance

• Learn to differentiate between Proper Engineering Decisions (PED) & Proper Management Decisions (PMD)

• Engineering processes need continuous review

• Be watchful for indications of “Group-Think”

• Recognize external pressures or conflicting interests (profits, prestige,..) causing deviations from usual eng. processes