1-4244-2509-9/09/$20.00 ©2009 IEEE

Bayesian Reliability Demonstration Test in a Design for Reliability Process

Mingxiao Jiang, PhD, Medtronic, Inc. Daniel J. Dummer, PhD, Medtronic, Inc.

Key Words: Bayesian reliability demonstration test, design for reliability

SUMMARY & CONCLUSIONS

This paper illustrates how a Bayesian reliability demonstration test (BRDT) approach can be, and sometimes must be, integrated into a Design for Reliability (DFR) process. A simplified and effective BRDT algorithm is given, based on the prior distribution characteristics of reliability in the DFR process. The BRDT can significantly reduce sample size in reliability demonstration test, and serve as a powerful validation tool in DFR.

1. INTRODUCTION

Due to increasing competition, decreasing development time and increasing product complexity, it is often difficult to meet customers’ expectations of product performance, quality and reliability. To achieve high product quality (“out-of-box” product performance often quantified by Defective Parts Per Million), many companies successfully practice Design for Six Sigma (DFSS) to design high quality into products in early development stages. Similarly, to achieve high product reliability (often as measured by annualized failure rate, hazard rate, or survival function, etc) some companies have implemented the DFR process to design high reliability into their products early in the development phase prior to transition into full production.

DFR and DFSS share similar engineering philosophies with DIDOV (Define, Identify, Design, Optimize, Validation) and other sequential engineering procedures. One major challenge in the DFR process is the reliability demonstration test (RDT) activity in the Validation phase. After the product reliability requirement is allocated down to the critical subsystem or component level, reliability requirements at the subsystem or component level may be too high to be economically validated. Bayesian reliability techniques can be applied very successfully to such challenges, by integrating all the relevant reliability information gained in the product DFR earlier phases, e.g., FMEA (Failure Mode and Effect Analysis) in Define phase and PoF (Physics of Failure) in Optimize phase.

Our paper starts with a brief introduction of a DFR process. Then the challenges of RDT in the Validation phase will be illustrated by applying a classical RDT (CRDT) approach, which may require a large sample size to demonstrate the required high reliability at acceptable

confidence levels. This is true in demonstration of the required reliability at subsystem or component level, after product reliability requirement allocation activity in the DFR process.

Bayesian reliability demonstration test (BRDT) approach can be adopted to significantly reduce sample size or testing duration (e.g, Kececioglu, 1994; Kleyner A et al, 1997; Krolo, 2002; Lu and Rudy, 2001; Martz and Waller, 1982). In our present work, BRDT is employed in several ways: • BRDT is integrated into the whole DFR process by

linking it to FMEA, PoF, and reliability requirement flow down or allocation.

• Successful application of a Bayesian approach depends on the prior experience or life data (testing or field) from previous generations of the product under design. BRDT can still be used successfully for a totally new product design and development, based on the prior distribution characteristics of reliability in a DFR process.

• Bayesian reliability approaches involve challenging mathematical operations for engineers. The approach given in the paper can be used very easily by engineers with any standard spreadsheet calculation methodology.

2. THE DESIGN FOR RELIABILITY PROCESS

Customer expectations and market forces drive new product development to incorporate rigorous DFR principles throughout the design process prior to release. DFR is intimately paced with various phases in the new product development cycle. Many companies serious about reliability are formally invoking DFR in the phase gate review process. DFR comprises the sequentially phased elements of reliability requirement setting, early prioritization of reliability risks and allocation, design & optimization, validation, and control. These elements are sometimes mapped onto the DIDOV discipline. Six fundamental pillars of DFR are: 1) Physics of Failure (PoF) study; 2) quantification of environmental and usage stressors; 3) failure analysis; 4) stress testing; 5) parametric data analysis; and 6) manufacturing control. A flowchart of a DFR process is illustrated in Figure 1.

Market forces set the reliability requirements, for example, in terms of annualized failure rate over the mission life. Once established, these requirements set the overall scale for the risk prioritization and allocation. Initial estimates of environmental and usage stressors, failure analysis on similar

prior products, and FMEA’s combine to prioritize DFR resources. FMEA’s and other design analysis methods are particularly useful for new technologies and for products poised to enter new market spaces. The FMEA Risk Priority Number (RPN) can be used to rank order the reliability risks. Allocation methods can be used to scale the prior product failure Pareto; individual failure mechanism’s weighting can be further adjusted using Physics of Failure understanding of the new product design relative to the prior product. The allocation, the FMEA RPN score, and other inputs quantifiably rank the new product reliability risks. Only the

top risks warrant the resources that the DFR process consumes. The reliability risk prioritization ranking must be continuously updated as stress test data, new failure analysis, refined quantification of environmental and usage stressor distributions, and parametric data analysis assessment arise throughout the development process. The reliability requirements and this initial reliability risk prioritization are presented at the new product concept phase review to cement engineering and management buy-in of the risks. Engineering stakeholders are typically assigned to each of the top priority risks at this time.

Figure 1. The DFR process during development

The FMEA prioritization tool typically contains an action column – the most important column in the document. In a rigorous DFR process, the action column for the prioritized reliability risks can be split into six action columns with sub-headings that reference the six fundamental pillars of DFR. These actions are largely undertaken in the design optimization phase. In the early design phase, the Physics of Failure comprises initial development of reliability transfer functions by listing key input variables and early modeling such as Finite Element Analysis. There is significant focus at this early stage for quantification of environmental and usage stressor distributions especially for those failure mechanisms that are stress-induced or wear-out. These data are used for staging stress testing, developing new stress tests and possibly stress testing key components. Parametric analysis at this early design phase focuses on developing degradation metric capability to be used in stress tests and reliability demonstration test environments. Generally there are only small quantities of prototype units – teams are often tempted to ignore failure analysis (FA) and manufacturing control. FA focuses in this early phase on metrology gap analysis. The

manufacturing control focus is on two key sub-elements: manufacturing metrology gap analysis and Design for Manufacturability. For both FA and manufacturing, the metrology needs must be assessed early since equipment lead times are often very long and the metrology is needed in the later design & optimization and validation phases. The Design for Manufacturability effort, especially useful for defect related failure mechanisms, focuses on designing for cleanliness, commonality, simpler processing, and fewer parts. These six pillars of actions are often reviewed at a formal design prototype phase review to document findings, pace actions, and highlight interactions.

Middle and late stages of the design & optimization phase focus heavily on stress testing and parametric degradation analyses. For stress-induced failure mechanisms, small quantities of test units are typically stressed-to-failure. The design margin is assessed by convolving the strength distribution with the estimates of the environmental stress distribution. For wear-out failure mechanisms, stress tests take the form of Accelerated Life Tests (ALT). Typical life test and ALT analyses track the Time-to-Failure for failed

units as well as the run time of censored units during test. The resolution of such an analysis depends strongly on the quantity of units that “fail”. For efficient, fast, and high resolution ALT, it is crucial to enable in-situ monitoring of key parametric continuous variables wherever possible. Degradation of these variables can be used as performance metrics to enable all of the units to be analyzed – this dramatically reduces the need for many parts to “fail” in a high resolution test. Often the stress tests are intentionally loaded into a Design of Experiments (DOE) with “edge of the distribution” units as guided by PoF and manufacturing process understanding. Results of these DOE’s are used for defining design targets, quantifying the reliability transfer functions and guiding manufacturing control. Stress test design throughout this phase can be optimally guided by, for example, Bayesian methods that harness the continual advancement of the PoF understanding and reliability growth analysis. FA of the stress test failures also plays a key role in this phase as supplying input to the PoF understanding necessary for defining design corrective action. Parametric analysis of the manufacturing process variables for parent populations and especially those populations loaded into the stress tests help uncover new key variables for the PoF understanding and guide manufacturing process development. The design optimization phase review focuses DFR emphasis on these stress test results, PoF understanding, reliability growth, and manufacturing process readiness.

The validation phase typically focuses on performing reliability demonstration tests. Parametric review of key manufacturing process variables for the material chosen to load these tests and parent populations leads rather than lags testing. Again, Bayesian methods can be used to harness the continual advancement of the PoF understanding and reliability growth analysis from the prior phase to guide the validation test designs. Design engineers working with manufacturing and supplier engineers utilize PoF understanding to stage and plan statistical process control (SPC). The formal validation phase review addresses RDT results, PoF understanding, FA process and documentation, and manufacturing control strategy. Any residual reliability risks are highlighted at this review.

The control phase begins as production ramps up to full volume. Warranty analysis and returns FA are used to check PoF understanding and validation phase RDT results. They also feed future product reliability risk prioritization. SPC is maintained on those parameters identified through the PoF activities of the prior phases. Any possible excursions can usually be swiftly addressed due to the documented DFR activities during the development phase.

3. RELIABILITY DEMONSTRATION TEST

The bulk of the DFR activities occur well before the validation phase when the reliability demonstration test is scheduled. The higher the product reliability requirements, the more extensive the test must be in order to demonstrate the required reliability. According to sampling reliability theory (e.g., Kececioglu, 1994), RL, the lower one-sided confidence

limit on the reliability, can be obtained from

( ) ( )∑ −=−⎟⎟⎠

⎞⎜⎜⎝

⎛

=

−r

k

knL

kL CRR

kn

01 1 (1)

where, n is the test sample size, r is the given allowable number of failures, and C is the confidence level. This equation is cumbersome to use. The following, which can be very easily programmed in any standard spreadsheet application, can be used instead,

)(2;22; 11

1

rnrCL

Frn

rR

−+−+

+= (2)

where, F( ) is the F distribution function. If r is specified to be zero (called “success run” test), then eq (1) is simplified as

nL CR1

)1( −= (3) Equations (1) to (3) could require unfeasibly large sample

size to demonstrate high reliability at reasonable confidence levels.

Bayesian reliability demonstration test (BRDT) approach has been explored in reliability test planning to reduce test sample size (e.g, Kececioglu, 1994; Kleyner A et al, 1997; Krolo, 2002; Lu and Rudy, 2001; Martz and Waller, 1982). In the remainder of this section, the primary equations of BRDT are listed from literature. We start from the Bayesian theorem:

∫

=∞∞− θθθ

θθθdShf

ShfSob )|( )(

)|( )()|(Pr (4)

where, S represents a group of observed events, θ is a random scalar or vector, Prob(θ|S) is the posterior probability density function of θ, f(θ) is the prior probability density function of θ, and h(S|θ) is the conditional distribution of S. If θ is the reliability, R, then we have the following posterior reliability estimation:

∫

= 10 )|( )(

)|( )()|(PrdRRShRf

RShRfSRob (5)

The confidence level C for the true reliability within interval [RL, 1] can be obtained as

∫

∫=≤≤ 1

0

1

)|( )(

)|( )()1(

dRRShRf

dRRShRfRRC LR

L (6)

For a certain product with a true reliability R, in its binomial reliability testing case (testing outcome of a certain sample being either fail or pass), with S denoting the outcome of testing the whole population of sample size n, we have

rrn RRrn

RSh )1( )|( −⎟⎟⎠

⎞⎜⎜⎝

⎛= − (7)

where r is the number of failures. For a “success run” test, with r = 0, eq. (6) becomes

∫

∫=≤≤

10

1

)(

)()1(

dRRRf

dRRRfRRC

nR

n

LL (8)

The Beta distribution is widely used as a prior distribution function of R, or

( )( )baBe

RRRfba

,1)( −

= (9)

where,

( ) ( ) ( )( )2

11,++Γ

+Γ⋅+Γ=

bababaBe (10)

With the Beta distribution as the prior distribution of reliability, eq. (8) becomes

( )( )∫+−

=≤≤+

1,

1)1(LR

bnaL dR

bnaBeRRRRC (11)

Equation (11) provides the trade-off relationship among confidence level C, reliability requirement RL, and testing sample size n.

4. BRDT IN A DFR PROCESS

Successful application of the Bayesian approach depends on the amount of prior reliability data (testing or field). However, for a case when the product or design is totally new and there is no prior product life data from testing or field, we need to extend the Bayesian reliability demonstration test approach.

4.1 Prior distribution of reliability

If a product development adopts a DFR process, the prior distribution of reliability for the components or subsystems to be validated can be reasonably assumed to be of Beta distribution being heavily weighted to the right end of (0, 1) (see Figure 2), with a > b. The reasons are:

0

4

8

12

16

20

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Reliability

Den

sity

a = 10, b = -1

a = 10, b = 0

a = 10, b = 1

a = 10, b = 2

a = 20, b = -1

a = 20, b = 0

a = 20, b = 1

a = 20, b = 2

Figure 2.Prior distribution of reliability, in a DFR process

• In the DFR risk prioritization phase, the reliability allocated to a specific component or subsystem could be very high. For example, a product under development may have an overall reliability requirement of 90% (for example, first year). Through FMEA and prior product Pareto assessment, about 10 critical components and subsystems are identified. For the sake of argument, assuming equal allocation of reliability requirement to each critical component or subsystem (a much better allocation approach can be done based on consideration of cost, risk level, etc) we have approximately 99% reliability as the requirement at one of these individual components or subsystems.

• Throughout the DFR process with stress testing and PoF

driven corrective actions, the reliability growth is tracked. Of course, this is subject to RDT to validate.

4.2 Simplified algorithm for BRDT

Step 1: Construct a prior reliability: ),...,,( 21 mP xxxFR = (12)

where, PR is the prior reliability, and kx is the key input variable identified through the PoF activities. Equation (12) is a fruitful result obtained in DFR activity. Step 2: Obtain the prior distribution of PR :

With the samples of random variables of kx as inputs to eq (12), random samples of PR can be obtained as outputs of eq (12), using a Monte Carlo simulation approach. Then we can have the histogram, mean ( RPm ) and variance ( RPV ) of PR . Step 3: Fit the Beta distribution as the prior distribution of reliability:

The two parameters in the eq (9) can be obtained as (Kececioglu, 1994):

( ) ( )

RPRPRPRPRP

VmVmm

b21 2 −⋅+−⋅

= (13)

( )

RPRP

mbm

a−

−+⋅=

112

(14)

Step 4: Conduct the trade-off study among RL, C, and n: Equation (11) is very difficult for engineers to use in

trade-off study among RL, C, and n. The simplified approach we propose in the following will change eq (11) to be an explicit algebra equation, which is very easy to use in any standard spreadsheet application calculation. Denote:

[ ]aA int= (15) Eq (11) can be simplified as:

( )( )

( )( )

( ) ( )∫ ∑ ⋅−⋅⎟⎟⎠

⎞⎜⎜⎝

⎛ ++

=

∫+

⋅−=∫

+−⋅

≈

− +

=

+

−++

L

LL

R nA

k

bkk

RbnA

RbnA

dzzk

nAbnABe

dzbnABezzdR

bnABeRRC

10

0

10

1

1,

1

,1

,1

∑=+

=

nA

knkG

0),( (16)

where, G( ) is a function of dummy variable k and unknown n:

( )

( ) ( ) ( ) 11,1

1),( ++−

+++

⎟⎟⎠

⎞⎜⎜⎝

⎛ +⋅−

= bkL

k

RbnABebk

knA

nkG (17)

Eq (16) seems to be still very difficult to solve, since the summation upper limit contains unknown n. In fact, since in a DFR process, RL at critical component or subsystem level could be high, absolute value of G(k,n) decreases very rapidly as k increases. So eq (16) can be approximated as:

∑≈=

N

knkGC

0),( (18)

where, N is a large number, for example, 100. Eq (18) is very easy to use in any standard spreadsheet

application. In the calculation, simply have two columns (one

of k, one of G(k, n)) up to N rows, and summation of G(k, n) as confidence level C. Adjust n several times until a preset confidence level C is met. Then we can have n (sample size) needed in BRDT with specified RL and C.

4.3 Numerical example

DFR for a product development identified a critical component, whose reliability needs to be at least 99% at 5-year usage time, based on reliability requirement allocation. After DFR activities and corrective and preventative actions have been implemented, this component reliability needs to be validated in a RDT, which will be run at a higher stress level than field stress condition. Weibull distribution can be used for this component reliability. PoF study shows that the AF (ratio of the characteristic life values under RDT condition and field condition) is about 50. The RDT duration can be determined to be:

yryrAFyrT 1.050/5/5 === (19) So we need to conduct RDT at higher stress level with

testing duration of 0.1 year to demonstrate the 99% reliability requirement. PoF study also shows that the failure mechanism is of wear-out type, with the two Weibull distribution parameters as uniformly distributed random variables (counting the uncertainty in PoF study), under RDT condition (shape parameter is independent of stress condition):

[ ]4 ,1U~β [ ]1.4yr ,7.0U~ yrη (20) Step 1: Construct the prior reliability:

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛−=

β

ηTRP exp (21)

where the parameters are given in (19) and (20). Step 2: Obtain the prior distribution of PR :

20,000 samples have been randomly generated according to reliability prior distribution in eq (21) via Monte Carlo simulation: The average and variance of the prior reliability samples from simulation are: 9860.0=RPm 0005.0=RPV (22)

0.990.960.930.900.87

7000

6000

5000

4000

3000

2000

1000

0

Reliability

Den

sity

Prior pdf of reliability

Figure 3. Prior pdf (probability density function) of reliability

Table 1: Results of required sample size at various confidence level to demonstrate 99% reliability

C 0.8 0.9 0.95 0.99n (Classical RDT) 161 230 299 459n (Bayesian RDT) 34 81 132 263

Step 3: Fit the Beta distribution as the prior distribution of reliability:

Using eqs (13) and (14), the two parameters of Beta distribution can be obtained as:

87.24=a 63.0−=b (23) Step 4: Conduct trade-off study:

Using a standard spreadsheet application to calculate eqs (17) and (18), we have the following results (we also add in the classical RDT sample size results for comparison purpose):

The required sample size for BRDT is much smaller than that for classical RDT.

5. DISCUSSION

We have shown how a comprehensive DFR process may be structured emphasizing those DFR activities which are likely to yield a successful reliability demonstration test at the end of development. The Bayesian approach outlined in the present paper integrates those DFR activities to help better estimate the prior reliability of the product entering the reliability demonstration test. We have detailed a simple spreadsheet based approximation used to design a BRDT that is significantly more efficient and cost effective than a classical RDT design. The test design we have focused on is a zero-failure (success run) test. However the methods described in this paper can be extended to BRDT designs with nonzero failures. Estimates of the prior reliability of the product provide a method for quantifying the interim effectiveness of the DFR process. This can feed into reliability growth analysis useful for the BRDT design. The more effective the upstream DFR effort is, the more efficient, and often earlier, the BRDT will be. The DFR process and the subsequently integrated BRDT can be used in cases of incremental new product migration but are especially useful for radical new product development efforts.

ACKNOWLEDGEMENT:

The authors would like to acknowledge Dr. Harrie Netel of Seagate Technology, Longmont CO, for fruitful joint effort with the authors in developing a robust DFR process.

REFERENCES

1. Kececioglu D, Reliability & Life Testing Handbook, Vol.2, PTR Prentice Hall, 1994.

2. Kleyner A et al., Bayesian Techniques to Reduce the Sample Size in Automotive Electronics Attribute Testing, Microelectronics Reliability, Vol. 37, No. 6, 879-883, 1997.

3. Krolo A et al., Application of Bayes Statistics to Reduce

Sample-size Considering a Lifetime-Ratio, Proceedings of Annual Reliability and Maintainability Symposium, 577-583, 2002.

4. Lu M-W and Rudy R, Reliability Demonstration Test for a Finite Population, Quality and Reliability Engineering International, Vol. 17, 33-38, 2001.

5. Martz H and Waller R, Bayesian Reliability Analysis, Krieger Publishing Company, 1982.

BIOGRAPHIES

Mingxiao Jiang, Ph.D. Medtronic, Neuromodulation Minneapolis, MN 55432

e-mail: mingxiao.jiang@medtronic.com

Dr. Jiang received his B.S. in Engineering Mechanics from Zhejiang University, China, M.S. in Reliability Engineering from University of Arizona, and Ph.D. in Mechanics of Materials from Georgia Institute of Technology. He is currently working as a Senior Principal Reliability Engineer at

Medtronic Neuromodulation. He is a CRE, a senior member of ASQ and a senior member of IEEE. He has 3 US patents and over 20 publications in referred journals and conference proceedings.

Daniel J Dummer, Ph.D. Medtronic, Cardiac Rhythm Disease Management Minneapolis, MN 55112

e-mail:daniel.j.dummer@medtronic.com

Dr. Dummer received both his B.S. in Physics and Ph.D. in Experimental Solid State Physics from the University of Minnesota. He performed postdoctoral research on elementary particle detectors at the Max-Planck Institut-fuer-Physik in Munich. He studied optical metrology and optical properties of materials at NIST in Gaithersburg MD. He was Senior Director of Advanced Reliability at Seagate Technology where he led Seagate’s award winning corporate initiative on DFR. He is currently working as a Senior Manager at Medtronic Cardiac Rhythm Disease Management.