IAEA International Atomic Energy Agency

INTEGRATION OF SAFETY CULTURE INTO REGULATORY

PRACTICES AND THE REGULATORY DECISION MAKING

PROCESS

Technical Meeting

IAEA Guidance on Safety Culture Self-Assessment and

Independent Assessment Abida Khatoon

Regulatory Activities Section, Division of Nuclear Installation Safety

Monica Haage

Operational Safety Section, Division of Nuclear Installation Safety

6-8 October 2014,

Vienna, Austria

IAEA

Contents

• Training Material on Safety Culture Self-

Assessment

• Independent Assessment (IRRS/OSART)

• Summary

IAEA

Basis for Training Material on Safety Culture

Self-Assessment

Requirement 13: Continuous improvement of safety culture

“All individuals in the organization, from senior management

down, shall demonstrate leadership by promoting safety. A

normative framework based on international best practices

shall be described in the management system to promote and

support a strong safety culture”.

Requirement 14: Assessment of leadership and safety

culture.

“Senior management shall regularly commission

independent assessments and provide for self-

assessments of safety culture and leadership”.

IAEA

• Culture is seen as something we can influence,

rather than something we can control

• Culture work needs to encompass the whole

organization – not only as a top-down process

• Edgar Schein’s well established iceberg metaphor

helps to understand and how to continuously

improve safety culture

• observe visible aspects (artefacts and

behaviour, “above surface”) and;

• interpret this information to reveal the cultural

reasons behind (found in attitudes, values and

basic assumptions, “below surface”)

• For safety culture improvement, the IAEA

emphasises human interactions (shared space)

including trust, mindful communication, learning

attitude, inquiring attitude, self-accountability,

diversity, self-reflection etc.

IAEAs Approach to

Safety Culture Continuous Improvement

Artefacts,

Behaviour

Values

Attitudes

Basic

Assumptions

IAEA

Why assess RB safety culture?

• Regulators play an integral role in shaping safety culture within a

nuclear power program, including the culture of Licensees, through

their regulatory strategy, organizational culture, communication, and

style of engaging/influencing

• Regulatory bodies and Licensees require a common language and

understanding related to safety culture

• Regulatory Bodies must lead by example in terms of exploring and

understanding their own Safety Culture in order to influence

licensees in positive ways

• Regulatory Bodies need to stay at the forefront of understanding

Safety Culture and related human and organizational factors if they

are to assess and positively influence Licensees

IAEA

• IAEA Safety Standards and IAEA Safety Culture publications

• Social, Organization and Behaviour sciences

• Past experiences- IAEA and external SC expert knowledge and

experience of NPP Safety Culture Self Assessment training

approaches and materials

Course material is based on

IAEA

Methodology for Training

Each module is consist of:

• Core set of lectures based on the IAEA safety standards;

• Further illustration by country specific example(s) through

Application set presentations,

• Practical examples from experts and Case Studies and

interactive exercises to ensure that the training is based on

the ‘learning by doing’ approach..

• Group discussions for further clarification

• Outcome and Future initiatives

IAEA

Types of safety culture assessments

• Self-assessment

• Independent - internal / external assessment

• Mix of independent and internal assessment

• Meta analysis

IAEA

1. Ambassador Workshop (2 Weeks)

The course duration is 10 days, divided into two one week

blocks. In between the two training blocks a book about

culture is compulsory to read.

2. Senior Management Workshop (3 Days)

3. Train the Trainer (1 Week)

Training Course for Safety Culture Self Assessment

IAEA Training Course Package:

IAEA

To enhance the understanding of:

• Safety culture and the value of safety culture assessments;

• How improvement in safety culture can enhance safety

performance; and

• Everyone’s roles and responsibilities in cultural work.

To provide knowledge and practical skills in SCSA methods,

based on current international research and practices in

nuclear industries.

10 INLEP Module 1 19

June

2013

Objectives of the Training Course

IAEA

SCSA 1 To describe the concepts of culture and safety culture and explain their

relationship to safety performance

SCSA 2 To understand the role and impact of the Regulatory Body’s strategy

and safety culture on the National Nuclear Power Programme

SCSA 3 To explain the relationship between SCSA and an ‘inquiring attitude’

SCSA 4 To describe the methodology and process for SCSA, including roles

and responsibilities and critical success factors

SCSA 5 To describe and be able to use practical methods to identify safety

culture strengths and weaknesses

SCSA 6 To describe the various stages involved in SCSA, including

descriptive and normative analyses

SCSA 7 To understand the variables that impact the effective collection of data

and communication of findings in a SCSA

SCSA 8 To understand the basis for transformational change of culture

Learning objectives

IAEA International Atomic Energy Agency

IAEA Safety Culture

Assessment Methods

IAEA

Prerequisite for Assessment of safety culture

GS-G-3.5

Safety culture self-assessement should:

• Include the entire organization

• Several different self-assessment tools should be used (e.g. interviews,

focus groups, questionnaires, observations and document reviews)

• A designated team representing all organizational levels and functions at

the installation should carry out the self-assessment

• A specialist in safety culture should be included in the team

• The self-assessment team should receive training

• The self-assessment team should summarize the results and identify areas

for improvement and may suggest actions to be taken

• The results should be reported to the management at an appropriate level

• A follow-up assessment should be performed

The independent assessment of safety culture should follow a similar approach

IAEA

The independent assessment of safety culture should follow a similar

approach as self-assessment

• The independence and qualification of the members of the

assessment team should be considered crucial for the success of the

assessment

• The team should be staffed with sufficient diversity of experience and

should include specialists in behavioural science, with knowledge of

statistical methods of analysis

• The independent assessment team should aim at identifying

strengths and areas for improvement

Prerequisite for Assessment of safety culture

GS-G-3.5

IAEA

Safety culture – multi-disciplinary expertise

• Anthropology

• Organizational theory

• Social Psychology

• Sociology

• Leadership and management theory

• Cognitive science

• Psychology

• Human Factor Engineering

• Resilience Engineering

• Organizational Factors

• ITO (interaction between Individuals, Technology and Organizations)

• Basic knowledge; Nuclear technology, nuclear organizations, regulatory

framework

Prerequisite for Assessment of safety culture

GS-G-3.5

IAEA

SCSA Process

IAEA

SCSA Process Step 1

• Scope

• Identify team

• Commit time and resources

• Strategy to address the results

• Communication strategy to inform staff

Step 2

• Train team members

• Clarify roles/ responsibilities

• Assessment strategy/methods

• Establish records management

Step 3

• Scope -cover activities from the

pre-launch to the communication

and follow-up of the results

• Minimize organizational disruption

• Adapt methods

• Plan to apply the methods in

parallel /independently

• Conduct a pre-job brief for the

team

• Step 4

• Orient the organization.

• Senior managers commitment

• Assessment is not an audit

• Learning about its existing

attitudes and behaviours and their

influence on safety

Step 5

• Capture information without interpretation or

judgment

• Maintain records

• Avoid personal biases

• Ensure access to information on progress of the

assessment, and respond to any questions

Step 6

• Analyse the data sources

• Draw conclusions according

to their logic

• Develop overarching themes

• Compare to the IAEA SCSA

framework

• Identify strengths and areas

for improvements

Step 7

• Develop key messages

• Develop communication packages

that suit the various target

audiences

• Team Lead to prepare a report with

input from team members

Step 8

• to management

• open dialogue regarding the

potential impact of the results

• Communicate with the organization

through various channels such as

discussion forums with groups,

cross-group training.

Step 9

• Engage the organization in action plan

finalization

• Depending on the approach preferred by

management, communication activities in Step 8

may include preliminary actions developed as a

consequence of the assessment

IAEA

• Using several assessment methods

The IAEA Safety Culture Assessment Methodology

Interviews

IAEA

Concept of Culture and Safety Culture

• The concept of culture is descriptive

• The concept of safety culture is normative

IAEA

Descriptive/Normative

Descriptive

‘is’

Based on data and a theory

of culture

Normative

‘should’

Based on data, a theory

of culture and a norm

Descriptive must come before the Normative!

• It sets standards for behaviour and

values

• It is related to practices known to lead

to safety

• It focuses on certain limited aspects of

the organization

• Being evaluative in the descriptive stage

can lead to exclusions

• Being descriptive helps to pick up weak

signals

• Being descriptive means having a broad

agenda for dialogue

IAEA

Descriptive analysis: Images of culture

Descriptive analysis

IAEA

Normative analysis

Interview data

Cultural facts

Cultural themes

Survey data

Cultural facts

Cultural Themes

Focus group

data

Cultural facts

Cultural Themes

Document

data

Cultural facts

Cultural Themes

Observation

data

Cultural facts

Cultural Themes

Descriptive Analysis: Images of culture

Normative,

evaluative analysis

Safety culture is not a culture; it is a normative interpretation of a culture

IAEA

Implementation of SCSA

• IAEA SCSA methodology was first implemented by

Pakistan Nuclear Regulatory Authority (PNRA)

• IAEA Training was provided to PNRA in 2013

• Ambassador Workshop (2 Weeks)

• Senior Management Workshop (3 Days)

• Train the Trainer (1 Week)

• Regional Workshop on Safety Culture Self

Assessment for Senior Management was conducted

in Slovenia - November 2013

IAEA

Future Work for the Review of Safety Culture by the IAEA

Peer Review Services (IRRS)

• Question Sets for Independent Assessment of Safety Culture

for Regulatory Bodies (eg. IRRS Mission)

Independent Assessment

IAEA International Atomic Energy Agency

IAEAs Approach to

ISCA

Independent Safety Culture Assessment

within Operational Safety Review Team (OSART)

IAEA

Safety Culture Independent Assessment should: The independent assessment of safety culture should follow a similar approach as

self-assessment

• The independence and qualification of the members of the assessment team

should be considered crucial for the success of the assessment

• The team should be staffed with sufficient diversity of experience and should

include specialists in behavioural science, with knowledge of statistical

methods of analysis

• The independent assessment team should aim at identifying strengths and

areas for improvement

Safety Standard GS-G-3.5:

Assessment of Safety Culture

IAEA

The ISCA Teams Area of Expertise

• Psychology

• Cognitive science

• Sociology

• Social Psychology

• Organizational theory

• Cultural theory

• Leadership and management theory

• Human Factor / Human Factor Engineering

• Organizational Factors

• Resilience Engineering

• ITO (interaction between Individuals, Technology and

Organizations)

Basic knowledge; Nuclear technology, nuclear organizations,

regulatory framework

IAEA

OSART Findings and Safety Culture

IAEA

Sources of information:

The IAEA ISCA Assessment Methods

Interviews

Team Findings

IAEA

Core of IAEA Safety Culture Analysis Process

e. g. independent assessment within OSART

Interview

data

Cultural facts

Cultural

themes

Survey

data

Cultural facts

Cultural

themes

Focus

group data

Cultural facts

Cultural

themes

Document

data

Cultural facts

Cultural

themes

Observation

data

Cultural facts

Cultural

themes

Overarching themes: Image(s) of culture

Final Issues;

Normative,

evaluative analysis

Team

findings

data

Cultural facts

Cultural

themes

IAEA

Example of amount of safety culture facts

• Team findings: Circa 220 facts

• 25 interviews: Circa 150 facts

• 7 observations: Circa 30 facts

• 7 focus groups: Circa 80 facts

• Survey 389 participated 97 questions

In total about 480 fact excluding the survey

material

IAEA

Reporting of the results

• Recommendations for areas in need of

attention

• Three senior managment workshops to

reach a shared understanding

• Exit meeting

• Report

IAEA

Summary

• The SCSA process involves 9 steps from

preparing the team and organization, through data

gathering and analysis, to reporting of results and

development of action plans

• Communication and engagement are critical along

the entire process

• Diversity of the team members is an asset that can

be used to avoid biases and enhance the rigour of

thinking through reflection and conversation

…Thank you for your attention