iaea technical meeting on the optimization of non ... · what has been done? individual - what...

NDE RELIABILITY AND THE INFLUENCE OF

HUMAN FACTORS

Dr. phil. Marija Bertovic

BAM Federal Institute for Materials Research and Testing

Dept. Non-Destructive Testing

Reliability of Non-Destructive Evaluation Systems - Human factors

IAEA Technical Meeting on the Optimization of Non-Destructive

Examination and In-Service Inspection to Improve Nuclear Power

Plant Performance

Outline

Why is there a need to address human factors in NDT?

Literature review: Factors affecting human performance in NDT

Reliability discussions

Conclusions

2 M. Bertovic │ NDE Reliability and the Influence of Human Factors

1

2

3

4

M. Bertovic │ NDE Reliability and the Influence of Human Factors 3

Why is there a need to address human

factors in NDT?

1 2 3 4

WHY IS THERE A NEED TO ADDRESS HUMAN FACTORS?

Avoidance of accidents, incidents and events

Examples of accidents and events, causes of which were assigned to human factors limitations:

⇨ North Anna power plant event

WHAT HAPPENED: Five Primary Water Stress Corrosion Cracking (PWSCC) indications in the steam generator (SG) safe-end weld NOT DETECTED (2009). The two 100% through-wall and three partial through-wall indications exceeding the acceptance criteria were detected in a subsequent in-service inspection (ISI) in 2012.

POST-EVENT EVALUATION:

It was very likely that the indications existed in 2009

The indications were within the inspectors’ “ability to foresee and correct”

The NDT system did not provide assurance that the component remains capable of performing its intended safety function

CONCLUSION OF THE ROOT CAUSE ANALYSIS (p. 10):

The site NDT organization did not adequately implement their responsibility to ensure effective application of the examination procedure by supplemental examination personnel

The on-site briefing conducted with NDE technicians was not adequate to ensure successful execution of the examination

* U.S. NRC. (2012). North Anna Power Station - NRC integrated inspection report 05000338/2012003, and 05000339/2012003. U.S. Nuclear Regulatory Commission.



Human factors affect the reliability of NDT

Non-destructive testing (NDT) needs to be RELIABLE

Reliability of NDT depends not only on the intrinsic capability of the measuring systems and on the application factors (the conditions under which NDT is applied), but also—and to a great extent—on human factors and the organisational context under which each inspection is carried out

Human performance affects the inspection results (can affect reliability, safety, efficiency, and financial cost)

Inspectors vary in their capability to detect and characterise material defects


INFLUENCES ON RELIABILITY OF NDT*

*Figure adapted from "Paradigm Shift in the Holistic Evaluation of the Reliability of NDE Systems”, by C. Müller et al., 2013, Materials Testing, 55(4), p. 264 and reprinted with permission of the Hanser Verlag in “Human Factors in Non-Destructive Testing (NDT): Risks and Challenges of Mechanised NDT”, by M. Bertovic, 2015, doctoral dissertation, Technische Universität Berlin.


What are human factors?


“

False True

Human factors = human error Human factors ≠ human error

Human factors refer only to people, their experience, training, personality and the conditions under which they work

“Human factors refer to environmental, organizational and job factors, and human and individual characteristics, which influence behavior at work in a way which can affect health and safety“ (HSE, 1999)

The influences of human factors on the performance need to be reduced, eliminated, etc.

The influences of human factors on the performance need to be understood, so that the negative adverse effects could be reduced or eliminated.


Individual

Team

Technology Organisatio

n

Extra-organisational environment


What are human factors?


Human error as a symptom

Latent conditions Active failure

Barriers

BOOOOM

*Figure adaptation: B. Fahlbruch Reason, J. (1997), Managing the Risks of Organizational Accidents, Farnham, Surrey: Ashgate.

Swiss Cheese Model

(Reason, 1997)



Literature review: Factors affecting

human performance in NDT

1 2 3 4

EPRI

WHAT HAS BEEN DONE?

HF in NDT Timeline (according to the year of publications)

1985 1995 2005 2015 1990 2000 2010

PNNL, Batelle

EPRI

PISC 3 (JRC & NEA)

PANI 3 (HSE)

BAM

U.S. NRC

SKI

STUK

FAA RSSB

Nuclear

Railway Axle inspection*

Aviation*

*no detailed literature study


WHAT HAS BEEN DONE?

Individual - What makes a good inspector?


⇨ Development of cognitive

strategies, decision aids and

tools for personnel selection

Knowledge and understanding of

NDT theory

Abilities (Mechanical

comprehension)

Personality (Conscientiousness, Cautiousness, Original thinking)

Adherence to the inspection procedure

Attention Motivation

Self-confidence Self-efficacy (belief in own

abilities) Task orientation

Consistency Tolerance of

environmental conditions

Experience

WHAT HAS BEEN DONE?

Working conditions

Environmental conditions

Temperature, humidity, noise level – not experimentally confirmed (PISC 3)

Organisational working conditions

Shift length, shift times, breaks, number of working days, number of rest days

Time pressure & noise – a certain amount may foster performance (ENKVIST ET AL., 2001)


Yerkes & Dodson law

* CC0 - Yerkes and Dodson, 1908 - in Diamond DM, et al. (2007). The Temporal Dynamics

Model of Emotional Memory Processing: A Synthesis on the Neurobiological Basis of Stress-

Induced Amnesia, Flashbulb and Traumatic Memories, and the Yerkes-Dodson Law. Neural

Plasticity, 33. doi:10.1155/2007/60803.

WHAT HAS BEEN DONE?

The effect of time pressure on the inspection

precision of manual UT performance in NPPs


* From Bertovic, M., Gaal, M., Müller, C., & Fahlbruch, B. (2011). Investigating human factors in manual

ultrasonic testing: testing the human factors model. Insight, 53(12), 673–676. doi:10.1784/insi.2011.53.12.673

A No temporal demand

B Medium temporal demand

C High temporal demand

Organisational climate

Faith and trust placed in individuals and teams by the employer or

contracting organisations

Extent and quality of supervision

Management Feedback Organisational context (internal and external)

Preparation for the inspection

Suitable documentation Adequate time for the

inspection

WHAT HAS BEEN DONE?

Organisation

Major contributors to NDT reliability


WHAT HAS BEEN DONE?

Organisation (SPANNER, 1986)


* From “Human Reliability Impact on In-Service Inspection”, by J. C.

Spanner Sr., 1986, in D. Stahl, ed. Proceedings of the 8th International

Conference on NDE in the Nuclear Industry. Orlando, FL: American Society

for Metals, vol. 21(19), p. 91.

Model of UT/ISI man-machine system*

- performance is seen as a result of interaction between the inspector and the technology in the context influenced by the management, the procedures, and the immediate working environment


*From “Investigating human factors in manual ultrasonic testing: testing the human factors model”

by M. Bertovic, M. Gaal, C. Müller and B. Fahlbruch, 2011, Insight, 53(12), p. 673.

WHAT HAS BEEN DONE?

Organisation (BERTOVIC, ET AL. 2011)


Organizational Context of NDT

Internal External

WHAT HAS BEEN DONE?

Organisation (HOLSTEIN, 2015)

*From “How NDT is Organized - Influence on the Reliability of NDE”, by R. Holstein and C.

Müller (2015) Paper presented at the 42nd QNDE, 2015, Minneapolis, USA.

WHAT HAS BEEN DONE?

Group/Team

Human redundancy seen as beneficial in error recovery (HARRIS & CHANEY, 1969)

Benefits can be expected when the redundant inspections are carried out independently of each other and when all inspectors are identifiable (BERTOVIC, 2015)

Group decisions can fail due to groupthink (BERTOVIC, 2017)


Image c

redit:

researc

htr

ends.c

om

http://images.google.de/imgres?imgurl=https://www.researchtrends.com/wp-content/uploads/2011/11/Tug-of-war.jpg&imgrefurl=https://www.researchtrends.com/issue25-november-2011/two%E2%80%99s-company-how-scale-affects-research-g

WHAT HAS BEEN DONE?

Group/Team

Human redundancy seen as beneficial in error recovery (HARRIS & CHANEY, 1969)

Benefits can be expected when the redundant inspections are carried out independently of each other and when all inspectors are identifiable (BERTOVIC, 2015)

Group decisions can fail due to groupthink (BERTOVIC, 2017)


Image c

redit:

phillipcart

er

http://i114.photobucket.com/albums/n261/philipcarter/humour/groupthink.gif

WHAT HAS BEEN DONE?

Inspection procedure

Procedural deficiencies

- associated with higher event and incident rate in nuclear power plants (BENTO, 2002)

Improvements in the procedure

- expected by using a user-centred design and the application of human factors principles (MCGRATH, 2008)

Development of the NDT instruction using a user-centred approach (BERTOVIC & RONNETEG, 2014)

- Usability study, Eye tracking


WHAT HAS BEEN DONE?

Technology


NDT community perspective

Manual NDT: associated with errors

Mechanised NDT: beneficial over manual NDT

Human factors perspective

Benefits, yes, but new risks can arise

WHAT HAS BEEN DONE?

Technology

NDT community perspective

Manual NDT: associated with errors

Mechanised NDT: beneficial over manual NDT

Human factors perspective

Benefits, yes, but new risks can arise


Failure modes Risks in data acquisition (68) and data evaluation processes (90)

Causes Individual (unintentional, rule violations) Technology Organisation

Consequences Missing indications (defects) Wrong acceptance or rejection of the component

Existing preventive measures

Inspection procedure Training Skills & Experience

Potential preventive measures

Automation Hardware and software solutions Improvement of the inspection procedures Organisation Human redundancy Training Decision aids

Failure Modes and Effects Analysis, FMEA (BERTOVIC, PhD dissertation, 2015)

WHAT HAS BEEN DONE?

Technology

Overtrust in automation – belief in the high reliability of automation

-Automation misuse – reliance on automation even when it is wrong (PARASURAMAN & RILEY, 1997)

Automation reliance in NDT

- Participants agreed with 50% of implemented errors (BERTOVIC, 2015)


Image c

redit:

htt

p:/

/ele

ctr

ek.c

o/

Im

age c

redit:

Huffpost

http://electrek.co/

http://i.huffpost.com/gadgets/slideshows/303506/slide_303506_2579315_free.jpg

CONTEMPORARY RESEARCH

The USA re-awakens…

U.S. NRC & EPRI

Comparing in-field performance versus laboratory environment during qualification (i.e. EPRI’s Performance Demonstration Initiative (PDI))


Literature study

Field study Reporting

EPRI STUDY

Initial Conclusions and Recommendations (SELBY, 2016, WCNDT)

26

Literature review: challenges and potential opportunities for improvement Problems - Physical access to welds - Component shapes that are unique - Research cannot replicate the actual

conditions due to hazards and cost Helpful tools - Pre-briefing with drawings, photos,

video, and/or simulation can help prepare individuals or duos for the mental workload and stresses of the task

- Debriefing

Longer term

‒ Simulations can complement training and pre-briefing

‒ Equipment simplification can provide next generation manual UT devices that are more usable by a wider range or people in a wider range of conditions

‒ Procedure simplification can aid in this effort as well

M. Bertovic │ NDE Reliability and the Influence of Human Factors


Open Space Technology discussions

2013-2016

5th EAW, Berlin, 2013 6th EAW, Minneapolis, 2015 WCNDT, Reliability session, Munich, 2016

1 2 3 4

WHAT DOES THE INDUSTRY NEED?

„Open Space Technology“ discussions

Gap in knowledge

- Still plenty of unknowns

Gap in communication

- Between the customer and the NDT vendor

- Between psychology and engineering (working conditions, fatigue)

- Between HF and NDT science

Gap in implementation

- Results are scarcely implemented in the practice


Fatigue, Vigilance

Working conditions

Inspection procedure

Organisation Motivation Qualification

Redundancy Safety culture Responsibility


Conclusions

1 2 3 4

EFFORT VS. DEMANDS


DISCREPANCY

Conclusions


Whose responsibility is it?

WHAT TO DO?

Bridge the gap

Image c

redit:

beeja

lparm

ar.

com

http://www.beejalparmar.com/bridge-the-gap/bridging-the-gap-jpg/

ERROR MANAGEMENT

Traditional approach to dealing with errors

Active error

Illustration by FAHLBRUCH, 2009

32 *FReason, J. (1997), Managing the Risks of Organizational Accidents, Farnham, Surrey: Ashgate.

ERROR MANAGEMENT

Contemporary approach to dealing with errors

Active error

Illustration by FAHLBRUCH, 2009

33 *FReason, J. (1997), Managing the Risks of Organizational Accidents, Farnham, Surrey: Ashgate.

Thank you for

your attention!

ACKNOWLEDGMENTS

Dr. Christina Zanotelli (born Mueller)†, BAM, Germany

Dr. Babette Fahlbruch, TÜV Nord EnSys, Germany

Jorma Pitkänen†, Posiva Oy, Finland

Ulf Ronneteg, SKB, Sweden

Dr. Ralf Holstein, DGZfP Education and Training Ltd. ,

Germany

34

CONTACT

[email protected]

M. Bertovic │ NDE Reliability and the Influence of Human Factors

References


Bento, J. (2002). Procedures as a Contributing Factor to Events in the Swedish Nuclear

Power Plants [SKI Report 02:63]. Nyköping, Sweden: Swedish Nuclear Power

Inspectorate (SKI). Retrieved from

http://www.iaea.org/inis/collection/NCLCollectionStore/_Public/42/022/42022489.pdf

Bertovic, M. (2015). Human Factors in Non-Destructive Testing (NDT): Risks and

Challenges of Mechanised NDT. Doctoral dissertation. Technische Universität Berlin.

Retrieved from https://opus4.kobv.de/opus4-

tuberlin/frontdoor/index/index/docId/7129

Bertovic, M., Calmon, P., Carter, L., Fischer, J., Forsyth, D., Holstein, R., … Selby, G.

(2014). Summary of the “Open Space Technology Discussions” on Reliability of NDE.

Materials Testing, 56(7-8), 602–606. doi:10.3139/120.110604

Bertovic, M., Gaal, M., Müller, C., & Fahlbruch, B. (2011). Investigating human factors

in manual ultrasonic testing: testing the human factors model. Insight, 53(12), 673–

676. doi:10.1784/insi.2011.53.12.673

Bertovic, M., & Ronneteg, U. (2014). User-centred approach to the development of

NDT instructions [SKB Report R-14-06]. Oskarshamn, Sweden: Svensk

Kärnbränslehantering AB. Retrieved from www.skb.se/publication/2718046/R-14-06

.pdf

Enkvist, J., Edland, A., & Svenson, O. (2001). Effects of time pressure and noise on

non-destructive testing [SKI report 01:48]. Stockholm: Swedish Nuclear Power

Inspectorate (SKI). Retrieved from

http://www.stralsakerhetsmyndigheten.se/Global/Publikationer/SKI_import/020227/4a

29504cde409b07dda1c235bc958dbc/01-48.pdf

Harris, D., & Chaney, F. (1969). Human factors in quality assurance. New York, NY:

John Wiley & Sons, Inc.

Holstein, R. & Müller, C. (2015) How NDT is Organized - Influence on the Reliability of

NDE. Paper presented at the 42nd QNDE, 2015, Minneapolis, USA.

McGrath, B. (2008). Programme for the Assessment of NDT in Industry, PANI 3

[Report No. RR617]. Health and Safety Executive. Retrieved from

http://www.hse.gov.uk/research/rrpdf/rr617.pdf

Mueller, C., Bertovic, M., Kanzler, D., & Ronneteg, U. (2016). Conclusions of the 6th

European American Workshop on Reliability of NDE. 42st Annual Review of Progress in

Quantitative Nondestructive Evaluation. AIP Conference Proceedings, 1706, 020006–

1—020006–9. doi:10.1063/1.4940452

Murgatroyd, R., Chapman, R., Crutzen, S., Seed, H., Willets, A., & Worrall, G. (1994).

Human Reliability in Inspection, Final Report on Action 7 in the PISC III Programme

[PISC III Report n° 31]. Nuclear Energy Agency. JRC & NEA.

Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse,

abuse. Human Factors: The Journal of the Human Factors and Ergonomics Society,

39(2), 230–253. doi:10.1518/001872097778543886

Spanner Sr., J. C. (1986). Human Reliability Impact on In-Service Inspection. In D.

Stahl (Ed.), Proceedings of the 8th International Conference on NDE in the Nuclear

Industry, 17-20 November 1986, Kissimmee, Florida, USA (pp. 89–95). Orlando, FL:

American Society for Metals.

iaea technical meeting on the optimization of non ... · what has been done? individual - what...

Documents