1 advanced engineering risk audit instruments syras & qmas bob bea national science foundation...
TRANSCRIPT
1
Advanced Engineering Risk Audit Instruments
SYRAS & QMAS
Bob Bea
National Science Foundation RESIN PROJECT
2
Probabilities & Consequences of Failure As IsM itig a tio n M ea su res
Probabilities & Consequences of Failure As Could BeIm p le m en ta tio n P ro gra m s
SYRASN o rm a l ra te s o f m a lfu nc tio ns , P erfo rm a n ce S ha p ing F ac to rs
P ro b ab ilit ie s o f p ro p e r D e te c tio n , A n a lys is , C o rre c tionP ro b ab ilit ie s o f F a ilu re fo r a ll S ys tem C om p o ne n ts , C o rre la tio ns
QM ASA u d it tea m , A u d it tra in ing , A u d it s ch e du ling
S c re en in g a ssessm e n ts, F a cto rs o f C on ce rnG ra d in g s , C o nse q ue n ce s o f 'F a ilu re ', R e m e d ia tio ns
D e fine & C h a ra cte rizeSystem, Perform ance Attributes & Failures
O p e ra to rs , O rg a n iza tion s , E n v iro n m e n tsH a rd w a re , S truc tu res , P ro ce du re s , In te rfa ces
3
Managing Pfi
C onc eptD efin ition
Des ign
C ons truc tion M aintenanc e
Dec ommis s ioning
P resent Future
Reduce L ikelihoods Reduce Eff ec ts Inc rease Detec tion
& Correc tion
Inc reas e
Margins of Quality
P fi
4
Managing Cfi
R es c ue R epa ir
R ec overy R ehabilita tion
Immediate Delayed
Direc t Indirec t On-S ite Off -S ite
C fi
5
Resilience - time & cost to regain Functionality
Time
Fu
nct
ion
alit
y
100 %
rescue
repairrecovery
rehabilitation
Good resililence
Poor resililence
6
Sustainability - Utility during life cycle
Time
Uti
lity
Benefits / Costs
Initiation Improvement
Operation Low Sustainability
High Sustainability
Decay
7
PROACTIVE to prepare
INTERACTIVE to respond
REACTIVE to recover
reduce consequences
increase detection &correction reduce likelihoods
8
An
alyt
ical
met
hod
s
QUALITATIVE'soft' - linguistic variables
'subjective' integration
QUANTITATIVE'hard' - numerical variables
analytical integration
MIXED QUALITATIVE & QUANTITATIVElinguistic variables translated to numerical variables
analytical integration
Low Moderate High
0.00 5.00 10.00
Low
0 5.0 10
Moderate High
2.5 7.5
9Sys
tem
an
alys
is a
pp
roac
h
Step #1: PERFORM A SYSTEM ANALYSIS
define system hardware, software, environments,organizations and individuals
Step #2: PERFORM A PROCESS ANALYSIS
define how things are done: how the system hardware,software, environments, organizations, and individuals
work
Step #3: PERFORM ANALYSES OF THE SYSTEM &PROCESS AS PRESENTLY CONFIGURED
define the likelihood and "quality" achieved
Step #4: PERFORM ANALYSES OF THERECONFIGURED SYSTEM & PROCESS
define the likelihood and additional"quality" that could be achieved
10
Screening analysis strategy
Develop CoarseQualitative Model
of Existing"System"
Identify High LikelihoodSystem Problem and
Consequence Componentsand Potential Mitigation Measures
Develop DetailedQuantitative Models
of High Risk Components
EvaluateHigh Risk
Components
Revise Models andEvaluate Risk Reductions
of Mitigation Measures
Evaluate Costs andBenefits of Mitigation
Measures and Impliment"Most Effective"
11
Objectives of analyses
• NOT prediction
• NOT numbers or analytical elegance
• Detect potential hazards and flaws
• Evaluate effects of mitigations
• Disciplined framework for thinking
• Empower those who have daily responsibilities for quality and reliability
14
Pro
acti
ve -
Rob
ust
nes
s =
dam
age
tole
ran
ce
Magnitude (Y) of Type (X) of Human Error
Pfs|e1.0
0.0
error tolerant
error intolerant
Intensity of damage or defects
defect tolerant
defect in tolerant
15
Durability
QA / QC LifecycleTask Organization
QA / QC Task Organization
InfluencingFactors
Base Rates
Task 1
InfluencingFactors
Base Rates
Task 2
InfluencingFactors
Base Rates
Task 3
DesignPhase
ConstructionPhase
OperationsPhase
QA / QC Task Organization
InfluencingFactors
Base Rates
Task 1
InfluencingFactors
Base Rates
Task 2
InfluencingFactors
Base Rates
Task 3
MaintanencePhase
Safety Compatibility Serviceability
QualityAttributesSYRAS©
16
Failures of QA/QC
Independent&
DependentErrors
Errors
•lack of expertise•resource deficiencies•excessive authority gradients•communication malfunctions•management breakdowns•rejection of information•violations
Correction
Detection
Analysis
17
P[F] = P[FI|E] P[E]+P[FI|E]P[E]+P[FE|E]P[E]
due to intrinsic causes with no error
due to intrinsic causes with error
due to extrinsic causes with error
221 E-6
1 E-5
1 E-4
1 E-3
1 E-2
1 E-1
1unfamilar taskperformed withspeed
simple taskperformd withspeed
change systemwith procedureswith checking
routine taskstrained, motivated
respond to system commandswith supervisory system
change system statewithout procedureswithout checking
routine taskperformed withspeed or divertedattention
23
1 1E11E-11E-21E-3 1E2 1E3SYRAS PSF
Extr
em
e ne
gati
ve
influen
ce
Mode
rate
neg
ative
influen
ce
Min
or
neg
ativ
e in
fluenc
e
Neglig
ible
influenc
e
Extr
eme p
ositiv
e in
fluenc
e
Moder
ate p
osit
ive
influe
nce
Min
or p
osi
tive
influen
ce
SMAS Grade 1 2 3 4 5 6 7
Outs
tandin
g
excee
ds
all s
tand
ard
s an
d r
equirem
ent
s
Goo
d, av
era
ge
exc
eed
s a
ll st
and
ards
and
req
uire
ment
s
Very
poor
doe
s not
mee
t an
y s
tand
ard
s or re
quirem
ent
s
Exc
elle
nt
Ver
y g
ood
Bel
ow
avera
ge
Poo
r
25
SMAS Main Menu
This is the starting point of the SMAS Data Management Tool.
Pre-Assessment Data - This way to Pre-Assessment Data Input Menu
Phase 1 Data - This way to Phase 1 analysis Data Input Menu
Phase 2 Data - This way to Phase 2 analysis Data Input Menu
FOCÕs / Scenarios - This way to Factors of Concern / ScenarioÕs Menu
Reports - This way to Reports
Utilities - This way to Import, Export, Delete Utilities
System Exit - This way to Exit SMAS and close MS Access.
26
QMAS components
OperatingTeamsFactors
OrganizationalFactors
ProceduralFactors
HardwareFactors
StructuralFactors
InterfacesFactors
EnvironmentalFactors
27
QMAS components, factors, attributes
COMPONENTS
1 - operators, 2 - organizations, 3 - procedures,4 - equipment, 5 - structure, 6 - environments,
7 - interfaces
FACTORS - graded
1.0 - operators1.1 - communications, 1.2 - process auditing, 1.3 - safety culture,
1.4 - risk perception 1.5 - emergency preparedness, 1.6 - command & controls,1.7 - training, 1.8 - resources, 1.9 - requesite variety
ATTRIBUTES - reasons for grades
1.1 - communications1.1.1 - same language, 1.1.2 - same vocabulary, 1.1.3 - established forms,
1.1.4 - clarity, 1.1.5 - concise, 1.1.6 - timely, 1.1.7 - appropriate amount1.1.8 - feedback, 1.1.9 - no significant barriers
29
QM
AS
gra
din
gs
1
2
3
4
5
6
7Very poor, does notmeet any standards
or requirements
Outstanding,exceeds all standards
and requirements
Good, average,meets most standards
and requirements
Poor
Excellent
Very good
Below averagehigh bound
mostprobable
low bound
31
QMAS mean results
Operators
Organizations
Procedures
Equipment
Structure
Environments
Interfaces
0 1 2 3 4 5 6 7
Co
mp
on
ents
Grades
Highest mitigation priorities
32
QMAS protocol
Stage #1ONSHORE
Stage #2OFFSHORE
Stage #3ONSHORE
general informationsystem specificinterviews - operating, management
walk downsoperations observationsmaintenance observationsemergency drillsinterviewsanonymous discussions
additional informationadditional interviews
33QM
AS
- c
oars
e q
ual
itat
ive
.
Step #1Select System
for Assessment
history of LOClikelihood of LOCconsequences of LOCschedule for assessment
Step #2Identify
AssessmentTeam
Designated InspectionRepresentatives (DIR's)
Step #3Coarse Qualitative
Assessment
general historysystem detailsoperations observationsprevious hazard studiesmanagement interviewsoperator interviews
Factorsof Concern
(FOC)
standard reports(SEMP, PFEER, ISM, user)rationalle for FOCmitigation measures
Proceed ?implement mitigationsrecord resultsschedule next assessment
no
yes
new
FO
C
34QM
AS
det
aile
d q
ual
itat
ive
.
Step #5Detailed
QualitativeAssessment of
FOC
general historysystem detailsoperations observationsprevious hazard studiesmanagement interviewsoperator interviews
SuggestedMitigationMeasures
mitigation measuresrationalle for suggestionsprojected effects
Proceed ?implement mitigationsrecord resultsschedule next assessment
no
proceed tocoarse quantitative
analyses of FOC
yes
yes
Step #4Develop Scenarios
for each FOC
new
FO
C
37
QM
AS
ass
esso
rs• QMAS counselor
• QMAS assessors– Facility operators
– Facility management
• QMAS training program– HOF background
– QMAS operations
– QMAS applications
38
QMAS assessors: THE MOST IMPORTANT PART!
• Experienced– Facility– Process auditing
• Respected
• Integrity
• Motivated
• Observant
• Thoughtful, insightful