-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
1/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
2/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
3/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
4/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
5/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
6/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
7/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
8/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
9/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
10/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
11/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
12/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
13/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
14/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
15/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
16/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
17/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
18/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
19/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
20/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
21/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
22/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
23/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
24/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
25/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
26/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
27/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
28/41
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
29/41
1-x O
o
to o
U 3 O
in O
JL
tn O
cr or-i
3" CD n CD
CM O
co co
H- . L U p a >-
>- >-
y H 11 CC < c = : _J ZD >- p i L U u _ u _ < t 3 H _ J h < c ^ - >- L U < c _j L U jy _J L L . ^ _ _
O o o
Q_ 1 _J
C O
0 0
O 3 C O 0
L L . o a i ; =D L U % t_ > O O O ^ L U
cc O S L U C O 5
1 C Q - O c e 0 Oh C O O S O : C u f c _ > LU C O * - Oa s
OO r H N - \ H
FIGURE 3
V *^-
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
30/41
J UJ
L7 J oo
UJ
X
UJ QQ
CD
3LT
CQ
CO CO o
C3
on GO 00 oa.
OQ
> < CO
UJ _J oa c / -> CO oQ_
O
UJ
>-
a s
CQ a
S
_CO
CO C O oQ. a? :
CO
CO
> ~
c _ > s .
>_j _< c
O uD r^\ LTl
O O
F rarer; n
3.33
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
31/41
CO
o t
- e ? J
o r r ^ CO o a . X L
QC UJ > - 0 CO
O
O S LU a .
* * I
C O 2=
0 0
o o c : OUJ
UJ = > C 4 ^ a :
z : o
> - >
UJ LU 3 c
> - _j
f UJ U_
s >OS UJ >
C D UD h O C M < O
FIGURE 5
-i.3H
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
32/41
o
r Z>
>
J z
x :
o< - > = C O CO
O
CO
o r
o o
CO
a:
Do r
a : Ql o
a :
UJ a e
aUJ ce
O
a : oo
CO
C _ >
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
33/41
V
I f a c o r r e c t i o n is r e q u i r e d (70-160) o n e m i g h t opt for a r o u t i n e type TCTO;
t h e r e is n o g r e a t u rg e n c y but t h e s i t u a t i o n s h o u l d b e c o r r e c t e d . f t h e
r i sk s c o r e i s 20 to 70, a t t en t i on is needed , b ut t h a t m i g h t b e th e type of
ac t ion w h e r e o n l y a Note or a Caut ion i n t h e d a s h o n e i s w a r r a n t e d to
c o r r e c t th e ac t ion .
Fur ther, t h e s e two g e n t l e m e n proposed a m e t h o d to e v a l u a t e the cos t
e f f e c t i v e n e s s o f any co r r ec t i ve ac t i on t h a t m i g h t b e p r o p o s e d , t h i s
ana lys i s , th e r i sk r e d u c t i o n e f f e c t i v e n e s s is th e p r o d u c t o f a r i sk
r e d u c t i o n m u l t i p l i e r an d r i s k score p rev io us l y d e v e l o p e d , div ided by a cos t
div i sor, e r i sk r e d u c t i o n m u l t i p l i e r is th e p e r c e n t a g e of th e r e d u c t i o n of th e hazard . or example , s o m e t h i n g tha t w o u l d r e d u c e th e h a z a r d p o t e n t i a l by 6 0* is a s s i g n e d a r i s k r e d u c t i o n m u l t i p l i e r o f . 6 ; i f i t is
100* r i sk reduc t ion , then th e r e d u c t i o n v a l u e is 1 .0 . ) h e cos t div isor is
e m p i r i c a l l y d e r i v e d cos t f rom (Cd = itJir . pec i f ica l ly, i t i s the cube root of a total c o s t d i v i d e d by 1 0 0 . h e cos t e f f e c t i v e n e s s is d e t e r m i n e d by th e e q u a t i o n : o s t e f f e c t i v e n e s s = **** t e o m - n o r i u LTi P L . i r . f i X TU - t s f " Th e d e g r e e of e f f e c t i v e n e s s o f var ious s y s t e m s can be e v a l u a t e d ' f f t h i s manner. See f i g u r e 1 7 ) . f f e c t i v e n e s s s c o r e s g r e a t e r than 20 i n d i c a t e tha t th e c o r r e c t i v e d e v i c e i s very w o r t h whi le ; scores l e s s than C i n d i c a t e
the f i x i s o f d o u b t f u l mer i t .
Th e m a t h e m a t i c s used i n s y s t e m s safety, a r e n o t h i n g m o r e than
p r o b a b i l i t i e s an d s t a t i s t i c s . s h o r t r e f r e s h e r i n p r o b a b i l i t i e s i s in
order. o n s i d e r the p r o b a b i l i t y t h a t d u r i n g th e f l i p of a co in i t w i l l be a head. h is is o b v i o u s l y 50-50 , a s s u m i n g i t c a n n o t l and on th e edge . e
Z. ZL
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
34/41
U3
II I
B
or
d 2
e r
II
ro
or
8
l
id
8
FIGURE 7 < 7
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
35/41
r e a c h t h a t conc lus ion , a p r i o r i t y s i m p l y d ed uc i n g the p o s s i b l e ways i t can
l and e a c h f l ip d i v i d e d b y th e to ta l n u m b e r of ways i t can land. i m i l a r l y th e p r o b a b i l i t y o f an y g i v e n n u m b e r on a d ie w i l l appear i s 1 / 6 , o n l y on e
number can c o m e up a l t h o u g h t h e r e are s i x p o s s i b i l i t i e s .
C o n s i d e r th e p r o b a b i l i t y of b e i n g a b l e to ro l l f ive 6s i n on e rol l and
win a free d r i n k a t th e bar. h a t is t h e c h a n c e of th e f i r s t d i c e b e i n g a six, th e second on e b e i n g a six, t h e t h i r d on e b e i n g a s ix and so on . h e to ta l p r o b a b i l i t y t h a t you can ro l l f i v e 6s i n on e r o l l w i l l of c o u r s e b e
1 / 6 t i m e s i t se l f f ive t imes . P r o b a b i l i t y = (1/6)5). h a t m e a n s the f i r s t
d i c e i s a s i x and th e s e c o n d on e is a s i x and th e t h i r d on e i s a six, etc .
W h e n you use th e verb "and" you ar e mul t i p ly in g ind iv idua l p r o b a b i l i t i e s .
Consider a n o t h e r poss ib i l i ty : h a t is th e p r o b a b i l i t y of f ive s ixes i n on e ro l l or f ive aces i n on e ro l l , in w h i c h case the b ar w i l l b uy you an d
your f r i ends a dr ink . h e p r o b a b i l i t y t h a t you may ro l l f ive s i x e s i n on e rol l or f ive o n e s i n on e ro l l are s i m p l y t h e two separa te c a s e s a d d e d
toge ther. ote t h e verb "or" m e a n s to add. h a t ' s th e p r o b a b i l i t y of f i v e s i x e s i n on e plus th e p r o b a b i l i t y o f f ive ones i n one h o s e are ery impor tan t i n u s i n g t h e " a n d " and "or" g a t e s i n th e fau l t t ree a n a l y s i s w h i c h
was d i s c u s s e d ear l i e r. h en you r e a c h an "and" ga te , th e two p r o b a b i l i t i e s are m u l t i p l i e d toge ther ; w h e n l t ' 3 an "or" ga te i t ' s th e s um of al l t h e
p r o b a b i l i t i e s g o i n g in to tha t "or" gate.
Cons ider for a m o m e n t th e p o p u l a r Sovie t p a r t y game ca l led R u s s i a n
Roule t te . t f i r s t g l a n c e you w i l l r e c o g n i z e the p r o b a b i l i t y o f b e c o m i n g a
fa ta l i ty i n a game o f R u s s i a n Roule t te , i f you were to play o n l y on e t ime,
3 . 3?
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
36/41
i s obvious one-sixth: ne c h a n c e ou t of six. ut w h a i . are the odds that the bullet under the chamber i s a dud or what i s th e probabi l i ty the f i r ing
pi n might fai l? ons ider ing the state of m i n d of on e wh o w o u l d play Russ ian
Roulet te , what are the chances tha t he ac tua l ly would hit h i m s e l f when he
pulled th e t r igger? ou can see tha t ac tua l ly the o d d s of dea th would be
the produc t of all: 1/ 6 an d p r o b a b i l i t y of n o d ud an d probabi l ty of n o
f a i l u r e an d probabi l i ty, of n ot m i s s i n g . )
H ow migh t you de te rmine th e var ious r e l i a b i l i t y ra tes? e te rmin ing th e
bul le t r e l i a b i l i t i e s are p r o b a b l y ra ther eas i ly obta ined . ou can loca te th e manufac ture r wh o has a guaran teed a spec i f ied rate of r e l i a b i l i t y on his
mullets , say n o more than f ive out of a thousand would fa i l . h e
re l i ab i l i ty i s at least . 9 9 5 . perhaps h e may have the accep tance tes t on that par t icu la r lo t of ammuni t ion where , out of a thousand rounds f i red at
' a n d e r n ou t of th e l o t only four fai led; therefore, the r e l i a b i l i t y of tha t
^ar t i cu la r lot c r ammuni t ion could be cons idered . 9 9 6 . hese are x n p i r i e a l ; er ived r e l i a b i l i t y f igures . a lcu la t ing the r e l i a b i l i t y of th e
revolver ml ht be a l i t t l e more di ff icu l t . ach m a n u f a c t u r e r g e n e r a l l y has i n i - a c c e p t a n c e tes t which his produc t must pass before i t i s de l ivered t o the
marne t . very g un i s tes t f i red. ur ing the in i t i a l f i r ings revo lvers have a high fa i lu re rate, sa y perhaps 20 out of a t h o u s a n d fa i l t o f i re the f i rs t
time. he revolver tha t fa i l s to f i re i s re tu rned to the factory produc t ion div i - io fo r r- fu rb i shment e g . t o rep lace a broken spr ing , den t hammer, etc .
w h i t * , o r n was that caused th e fa i lu re t o fire. h a t revo lver i s aga in returned f o r accep tance tes t an d on the second a t tempt t o f i re , genera l ly i t
would fire. y the t ime each pistol i s f i r ed ten t imes th e r e l i a b i l i t y rate
i.O
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
37/41
is very, v e r y h i g h , s a y m a y b e on e out of a t h o u s a n d w o u l d fa i l t o f i re . f a . s a m p l e of , say ten or more revolvers , w e r e t e s ted t o c o m p l e t e fa i lu re , or
to w e a r out , you w o u l d see a c u r v e t h a t looks s o m e t h i n g l ike t h a t shown i n
f igure 1 8 . s you n e a r th e s e r v i c e l i f e o f th e r e v o l v e r, th e s p r i n g s b e g i n to break, th e f i r i n g p i n s wear, e tc . , a s th e r e v o l v e r w e a r s out . h i s c u r v e i s ca l led a " b a t h t u b c u r v e " b e c a u s e i t ' s s h a p e d s o m e w h a t l ike a ba th tub , and
i t i s used fo r p r e d i c t i n g r e l i a b i l i t y o f h a r d w a r e . word of cau t ion : h en you are d o i n g f l i g h t t e s t you are a c t u a l l y p e r f o r m i n g on th e l e f t h a n d s i d e
of th e ba th tub , so th e f a i l u r e r a t e s ge ne r a l l y a re a lot h i g h e r than t h o s e
of a proven systemi
Now l e t ' s suppose fo r a m o m e n t tha t th e r e l i ab i l i t y o f th e r e v o l v e r an d
th e b u l l e t s are of s u f f i c i e n t l y h i g h order, t h a t you c an c o n s i d e r th e
p r o b a b i l t y b e i n g k i l l e d i n a round of R u s s i a n R o u l e t t e to b e 1 / 6 .
I f s ix people were g o i n g to play th e game, an d af te r each player p u l l e d
th e t r igger, s h o u l d h e surv ive , h e w o u l d sp in th e c h a m b e r and hand i t to the
n e x t guy.
f a player was f a t a l l y in ju red , th e r e ma i n i ng players would re load an d c o n t i n u e th e g a m e u n t i l a l l s ix h ad th e o pp o r tu n i t y to play.
W h a t i s th e p r o b a b i l i t y tha t s ix can play a n d someone be k i l l e d i n th e game?
T h a t i s the p r o b a b i l i t y t h a t on e g e t s ki l l ed , plus th e p r o b a b i l i t y tha t tw o
would be ki l l ed , plus th e p r o b a b i l i t y t h a t t h r e e would ge t ki l l ed etc . , up
to th e p r o b a b i l i t y tha t a l l s ix w e r e fa ta l ly in ju red . i n c e the sum of al l ind iv idua l p robab i l i t i e s m u s t e q ' j a l one, you c o u l d a l so so lve tha t p r o b l e m
by s u b t r a c t i n g th e p r o b a b i l i t y t h a t n o n e w e r e ki l led f rom on e o r uni ty :
P0 + P i + ?2 . . . Pn P 0 = 1 - P o - ow w h a t is th e p r o b a b i l i t y t h a t
3 40
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
38/41
K E a r l y
F a i l u r e s R a n d o mF a i l u r e s
B u i - , - i n i v r i o d
C o n s t a n t F a i l u r e R a t e
USEFUL L I F E P E R I O D
A g e - - >
We a r o u t n d R a n d o m F a i l u r e s
We a r o u t P e r i o d
B AT H T U B QJRVE
F I G U R E 18 4 1
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
39/41
none are k i l l e d ? e l l t h a t ' s th e p r o b a b i l i t y t h a t a player w o u l d s u r v i v e on
an y one round, an d th e n e x t guy w o u l d surv ive , an d th e t h i r d g uy would
surv ive , e tc . or , in o t h e r words , o ne m i n u s 5/6 ra i sed to th e s i x t h power,
and tha t t u r n s out to b e (5/6)6 or .3349. h e p r o b a b i l i t y t h a t someone is k i l l e d is then .6651.
N ow i f you w e r e to ask an a l t e r n a t e q u e s t i o n , nam e ly w h a t i s the
p r o b a b i l i t y t h a t e x a c t l y tw o were ki l led and e x a c t l y four surv ived , h ow
w o u l d you s o l v e t h a t ? ow ma ny c o m b i n a t i o n s ar e t h e r e o f two b e c o m i n g
ki l l ed an d four s u r v i v i n g ? h e f i r s t two c o u l d b e k i l l e d an d th e las t four
surv ive , th e f i r s t and th e l a s t on e ki l l ed , a n d th e mi dd le f o u r survive, 6
etc . h e number of c o m b i n a t i o n s are s h o w n as fo l lows: (21)(41) 1 5 . n
Th e genera l formula for d e t e r m i n i n g p ro ba b i l i t y is s h o w n be low: x (-x)
pX q n-x n i s th e total e v e n t s i n t h i s case si x players ; are th e events of
in te res t , i .e . , 2 dea ths ; p is th e p r o b a b i l i t y tha t d e a t h w i l l occur, i n
t h i s event on e s ix th ; q i s th e p r o b a b i l i t y t h a t d e a t h wil l not occur, i n
t h i s event , f ive s ix ths . f you s o l v e t h i s you f ind th e p r o b a b i l i t y of
e x a c t l y two k i l l e d an d four s u r v i v e is .16745.
St 1 d orn do you h a v e th e o p p o r t u n i t y to w o r k w i t h s p e c i f i c s such as
descr ibed above. e n e r a l l y you must c o n s i d e r a co n t i n uu m of events , or c o n t i n u u m of p r o b a b i l i t i e s . or t h a t you use a genera l fo rmula fo r empi r ica l dis t r ibu t ion , c a l l e d " a Poisson d i s t r i b u t i o n . " ere i s th e
formula:
. c -wt T ,.>* - a .r fe. * uK At
In he world f ccident
prevention ou re nterested n reventing he first ccident. What s he
probability of all possible umber f ccidents occurring: one, wo, hree,
3.MX
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
40/41
Etc? s before , s o l v e b y s u b t r a c t i n g p r o b a b i l i t y o f n o a c c i d e n t s o c c u r r i n g
f r o n t I . f Q ^ e . ^j / - P T h i s i s M u r p h y ' s L a w ; i t s h o w s t h a t i f I . j j u l i i s a n y t h i n g o t h e r t h a n z e r o , ( i f t h e r e ' s a w a y fo r s o m b o d y t o sc rew i1 1 1 ) 1 i h i n i l l l i c r f i s e n o u g h t i m e , ( somebody w i l l i n f . i c t n c t f w i t u p ) a nd \\v
p r \ j J > i b i l i t y Pany . i p p r o a c h e s u n i t y (acc iden t w i l l occur ) .
Limbda i s t he t a i l u r e r a t e o r t h e inverse o f t h e m e a n t i m e b e t w e e n
f a i l i r e ( M T b F ) . f s o you ' re r u n i n g a t e s t r i g t o d e t e r m i n e m e a n t i m e
b e t w e e n fa i lu re , y o u c a n ex t rac t t h e f a i l u r e r a t e d i r e c t l y.
V s t e m : ; Safe ty, a t a p p l i e d h e r e a t t h e F l i g h t Te s t Cente r, i s m o r e
i i t - v ^ : r l a t e l y ca l led Te s t Program Sa fe ty, s i n c e w e a re h e r e more in te res ted
v . i t . i , 1 t ( ) [ .l i i - t , i n - t e s t t h a n w o ar e o f t h e p h y r. i o , i ] d e s i g n o f t h e
i k u ' d w i r . e < i , a s t e s t . e n g i n e e r o r the f l igh t t e s t pi lo t , p r e p a r e a t e s t
i i r i . h i l e y o u ; i v > > r s p a r i n g d t e s t p l a n , yo u m u s t c o n s i d e r ' t h e h a z a r d s
i n v o l v e d . s ing t h e A F S C F o r m 5028 d e s c r i b e d e a r l i e r i n th e t e x t , l i s t a l l
' h e I . . i ' . . i r d .c t h e i r causes , t h e i r e ffec t s , a n d c o n t r o l s y o u m i g h t e m p l o y t - - .
p r w < t h e occur rence o f t h a t hazard .
. . : ' i v . i . e : ; i : p l n u has b e e n w r i t t e n , i t w i l l b e r e v i e w e d by e n g i n e e r -
i " : > t . - - f i n i c a l con ten t . l i i s r e v i e w i s p r i m a r i l y t e c h n i c a l a l t h o u g h sa fe ty
i puy.be cons ide red . nc e t h a t r e v i e w ha s b e e n accompl i s h o o t h e n you
^ i I I meet a s y s t e m dfe ty R e v i e w Board ( S R B ) . o u , y o u r superv i sors , a nd
t h < - " ' t e m Safe ty D i v i s i o n w i l l m u t u a l l y d e t e r m i n e ^h e c o m p o s i t i o n o f the
b o a r d . r i e b o a r d w i l l u s u a l l y c o n s i s t o f o p e r a t i o n a nd e n g i n e e r i n g represen ta t ives p l u s o t h e r s a s requ i red : h e o p e r a t i o n s r e p r e s e n t a t i v e i s
prefe rab ly c u i r e n t i n t h e type a i r c r a f t u n d e r t e s t a n d i s i n a s u p e r v i s o r y
U T
-
8/12/2019 USAF TPS Volume IV Test Management Chapter 3 Systems - Test Safety
41/41
or independent posit ion. he engineer ing representatives ar e supervisors o r e x p e r t s from the disciplines involved: . e . , propulsion, aerodynamic, f l u t t e r , e t c . ccas ional ly there will b e a maintenance personnel
represented i f there i s s ignif icant maintenance involvement. any t e s t , need representation t h e Fire Depar tment o r t h e hospital i f lasers o r t o x i c
materials a r e used or a r e p a r t of th e t e s t .
When t h e del ibera t ions are complete a risk assessment i s determined,
( h a z a r d o u s t e s t , r a e d i u n risk or low risk a s shown b y the matrix discussed
earlier). T h e complete AFSC Form Form 5028 i s s igned by board members and
supervisors a s shown o n t h e f r o n t side of th e Form 5 0 2 8 . Afte r th e commanders . have s e e n these documents, the test plan a n d Form 5028 i s br iefed t o th e
Commander, Vice Commander, a n d the Test Wing Commander regardless of the
r i s k l e v e l .