Customer-Centered Reliability Methodology

Russell D. Brunelle University of Washington Seattle Kailash C. Kapur University of Washington Seattle

Key Words: Multi-state reliability, Binary-state reliability, Reliability measures, Lifetime weight measure

SUMMARY & CONCLUSIONS

This paper proposes a series of reliability measures for multistate reliability models. As quality and reliability should be defined from the viewpoint of the customer, so should reliability measures. The circumstances under which each measure approximates customer satisfaction are discussed, and a new measure based on lifetime weighting functions is introduced in an attempt to make the multistate reliability model more responsive to time-varying customer demands. Examples are given of the calculation of these measures for a typical continuous-time Markov model, and the extension of these measures to continuous and mixed models is discussed.

1. INTRODUCTION

For many types of systems, the system’s state and the states of its components will change over time through a finite set of stages which are distinct and noticeable to the customer. When there are only two distinct stages for each component and the system, binary reliability theory applies. When there are more than two distinct stages for any of the components or for the system, multistate reliability theory applies and includes binary reliability theory as a special case. Early work in multistate reliability theory is documented in Refs. 1-2, and extensions to the basic theory are documented in Refs. 3-5.

IS0 8402 defines quality as “the totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs.” (Ref. 8) A typical modern reliability text (Ref. 9) defines reliability as “the ability of an item to perform a required function, under given environmental and operational conditions and for a stated period of time.” Since it is clear that an item’s ability to perform its required function bears on its ability to satisfy stated or implied needs, we conclude that reliability is a time-oriented quality characteristic (Refs. 10- 11). As quality must be defined from the viewpoint of the customer, so must reliability. As different customers may value to greater or lesser degrees different aspects of the dynamic performance of a given item, we seek in this paper to propose and develop a variety of reliability measures which may be made available to the customer for reliability assessment within the context of the multistate reliability model.

2. NOMENCLATURE, NOTATION AND ASSUMPTIONS

2.1 Definitions and Assumptions

$(t) = The state of the system. $(t)=@m iff the system is in its maximal state of functioning @ ( t ) ~ {@I, &., ..., (Pm.1] iff the system is between its maximal

+(t)=@o iff the system is in its minimal state of functioning and minimal states of functioning

@ e 0 P(t> i) = P+* ( t ) = PrtNt) = Ot 1

@,E [O,-) b‘ ie (0, 1, ..., m } $4, ‘t i<j

2.2 Discussion

Utilizing this notation, we may form a traditional binary model (Ref. 12) by setting m=l and + l = l . By allowing system states to be any positive real number, we may set the value of each state equal to the customer-defined utility (or some other numerical-valued characteristic) associated with each state. By associating a real number containing direct interpretation in terms of system characteristics with each state, we cause measures such as Var{$(t)} to be unaffected by the addition of unused states, and to have direct interpretation in terms of the operation of the system.

3. MULTISTATE RELIABILITY MEASURES

It should be emphasized that all of the measures presented here may be used for components as well as for systems. Also, as we will discuss later in this paper, these measures are all easily extended to continuum and mixed reliability models.

Static measures may be calculated even when the system state probabilities are known for only one moment in time, while dynamic measures require knowledge of these system state probabilities either up to some time t* (for finite-time dynamic measures) or for all t (for infinite time measures). Dynamic probability data may be obtained through continuous-time Markov modeling or through data-collection processes which collect and retain data continuously while items are on test.

3.1 Static Measures

3.1.1 Upper State Probabilities [USP(t,i)]

This measure (1) was considered in Ref. 15 and may be found throughout the literature. It has the interpretation of being the probability that the system is in a state greater than or equal to the given one. When m=O1=l (a binary model), it

286 0-7803-3783-2/97/$5.00 0 1997 IEEE

1997 PROCEEDINGS Annual RELIABILITY and MAINTAINABILIT” Symposium

follows that USP(t,l)=E{ $(t)}=Pr{ $(t)=l}. The customer may find this measure useful when it is possible to identify some cutoff state, at or above which there is a qualitative difference in the worth of the system. It is also of value when there is some action the customer must take when the system falls below a certain level of performance.

This multistate reliability measure was introduced in Ref. 1. It has the interpretation of being the expected value of the system state. This measure might be especially valuable to the customer when it is important that the system on average maintains a high state of functioning, but when no single state is identifiable as catastrophic. This measure has range [ $ o , $ ~ ] . It may be expressed as (2)

m

Of course, one may also consider the expected value of powers of $(t), in order to obtain various moments of the distribution of the system state as a function of time. One may then consider Kurtosis, Skewness, Variance, the Mean etc. of the system state or a function of the system state.

3.1.3 State Variance [SV(t)]

This measure was explored in Ref. 7. The customer might find the system state variance valuable for situations where consistent performance of the product is critical, or when a repair schedule must be maintained without the need for expediting. It may be expressed as (3)

m m 2

SV(t) = VarI$(t> 1 = [&%+J (t)l- [COj Po, (t>I (3)

Many customers may be interested in finding the time i’ at which the system state variance is at its maximum. This may be found through numerical methods, although in certain simple cases symbolic solutions may be practical. For cases where a non-repairable system begins in its maximal state, and spends a length in each non-minimal state governed by an exponential distribution with a common parameter p, we may find the following closed-form solutions.

TABLE 1: b=pt’Values for State Variance

j=O j = O

\2688b3 + 4608b’ + 5184b + 2880 ) 3,04585 la4 + lMb3 + 7 2 8 ’ + 20166 + 2592

5 e b =

3.1.4 Multistate Hazard Function [h(t,i)]

Throughout the binary reliability literature, the hazard function h(t) has yielded intuitive understanding of the dynamic behavior of binary systems (Refs. 9, 16). It is defined as the instantaneous failure rate given that a failure has not occurred prior to that point. Recently, the concept of the hazard been extended to the multistate case (Ref. 7). Here, we define this measure as the rate of passing into or below state i given that the system is above state i at time r (4). Thus, this measure reduces to the binary case when the system states are { 0,l } and one sets i=O.

mW$(t) 5 $ i 11

PrI$(t) $ i 1 h(t , i ) = dt (4)

3.2 Dynamic Measures

3.2.1 Finite-Time

3.2.1.1 Cumulative Peqormance Expectation [OE(t)]

This measure was introduced in Ref. 5. If one may interpret the state of the system as being akin to “utility per unit time”, then one may interpret this new random variable, Y(t), as expressing the total utility received in [O,t]. This measure has the range [ t ~ $ ~ , t $ ~ ] , and may be expressed as (5)

f m

O W ) = E{Y(t)1 = j [ x $ i P o , (z )3dz

where Y ( t ) = jQ(T)dT

(5 ) 0 j=O

r

0

3.2.1.2 Cumulative Performance Variance [OV(t) and OVUB(t)]

This measure may be expressed as (6)

O V ( t ) = Var{Y(t)} =

This measure expresses the variance of the total output received in [ O J ] , and so can be used to discern how much one may count on receiving a total utility near the expected value, possibly through application of Chebychev’s theorem. Expression (6) may not be calculable in all cases. Using the Schwarz inequality, an upper bound (7) may be found for it, as suggested in Ref. 5:

1997 PROCEEDINGS Annual RELIABILITY and MAINTAINABILITY Symposium 287

3.2.2 Infinite- Time

3.2.2.1 Dwell Time Expectation [DTE(i)]

This measure was initially explored for the multistate case in Ref. 7, though its equivalent for the binary case is well known (Refs. 9, 16). It is interpreted as the expected time the system will remain in the given state over its entire lifetime. If the maintenance cost associated with each state is known, this measure may allow easy calculation of total maintenance costs for given repairheplacement schedules. In the binary case, DTE(i) = MTBF. It may be expressed as (8)

4. MILITARY EXAMPLE

4.1 Model Definition

This model is based upon an example originally presented by Boedigheimer and Kapur (Ref. 18). We consider the effectiveness of a military force consisting of separate attack units and artillery units. The military force begins battle with six attack units and five artillery units. The system, consisting of the attack units (xl) and the artillery units (xz) has as the state of each component x1 and x2 the number of remaining units of that type. So, a component state vector of (3,4) would indicate that there are three remaining attack units and four remaining artillery units. We assume that it is not possible to replace units that are destroyed, and that each component spends a length of time in each non-minimal state governed by an exponential distribution with a common parameter.

Thus, we have the following expression (12) for the dynamic probability of each component being in each one of its states as a function of time, which is related to the Erlang distribution.

DTE( i ) po, ( t)dt (8) 0

Several variants of this measure are available, including "on- stream-availability", which is the proportion of the system's useful lifetime that it spends in and above a given state (Ref. 5) .

3.2.2.2 Total Pe@ormunce Expectation [TOE]

This measure was initially suggested in Ref. 7. If the integral in question (9) converges, it is interpreted as the average total utility that will be delivered by the product.

m .- (9)

0 j = O

3.2.2.3 Lifetime Weight Expectation [LWE[U(t)]]

as (10) This measure was created for this paper. It may be expressed

0 ] = O - where J = j U ( t ) g ( t ) d t (10)

0

This measure indicates the degree to which the product satisfied customer expectations over its entire lifetime. It considers the performance of the product over all time, weighted by a function U(t) which indicates the customer's level of interest in the product at each particular moment in time. It reduces to TOE if one chooses U(t)=l, and it reduces to OE(t*) if one chooses (1 1)

1 O l t l t "

0 otherwise U ( t ) =

This measure will be of value to customers and classes of customers who do not have a firm target lifetime in mind (i.e. a point at which the product will be discarded), but who find that at different times the product's performance (or lack thereof) plays greater or lesser roles in determining satisfaction with the product. Although it is possible that the U(t) functions could be approximated through customer surveys, it is also possible that for particular well-known product lines certain forms of U(t) may be assumed.

( p i t ) " ' - j e - ~ r i

(m- j ) ! (12) for j = m,m - 1, ..., 1

Pr{x, =xi;} = I - CPrIx , = x,, 1 for j = o 1 ,=I

Let us assume that based upon the state of each of the two components of this system, there is an overall effectiveness rating for the entire military force. Our customer, the military, indicates that they wish to consider six distinct levels of performance for the entire military force, [ 0, I, 2, 3, 4, 5 ) . It is assumed that the state values for the system have some direct interpretation in terms of the real performance of the system: that is, state 2 is half as desirable as state 4. Based on their experience with this type of military force, the customer indicates that the following state vectors (Table 2) for the components are such that any decrease in the state of either component would cause a decrease in the system state. These, therefore, are the lower boundary points.

TABLE 2: Customer-Specified Lower Boundary Points

From this information, using commonly known techniques for multistate systems (Ref. 34) we can determine the system state associated with any particular state of the set of components. If the model were large enough and complicated enough that computational time was a concern, then it might be worthwhile to decompose the system into a series of binary

288 1997 PROCEEDINGS Annual RELIABILITY and MAINTAINABILITY Symposium

models (Ref. 35) and then utilize some of the more advanced algorithms which have recently been developed to assist in such cases (Refs. 19,20).

Now, let us assume that through historical battle records or other simulations it is found that the transition rate for the first component is pl= 1.2000, and the transition rate associated with the second component is p2=1.9000. With this information, using direct enumeration based on the structure function calculated from the lower boundary points, we may find the probabilities of being in any given state as a function of time.

O.* t \ 0

0.

0 /

0

0 1 2 3 4

Figure 1. Dynamic System State Probabilities

With this time-based information, we may calculate values for each of the reliability measures of interest. Measures will be calculated both for the general time-based case, and specific results will be found for t=2, which is assumed to be of special interest to the customer.

4.2 Pegormance at the Customer’s Time of Interest (t=2)

TABLE 3: State-Dependent Measures

~

1997 PROCEEDINGS Annual RELIABILITY and MAINTAINABILITY Symposium 289

1 p(2, i) USP(2, i) h(2, i) 0 0.1876 1.0000 0.3229 1 0.4100 0.8124 1.0759 2 0.1593 0.4024 1.3077 3 0.1595 0.2431 1.8263 4 0.0690 0.0836 2.6694 5 0.0146 0.0146 NA

TABLE 4: State-Independent Measures

SE(2) SE(2)/4, SV(2) TOE 1.5562 0.3112 1.5303 8.4583

4.3 Interpretation of Measures at the Customer’s Time of Interest (t=2)

We can conclude that the system is in a state of disrepair, as the customer’s current average utility is only 31% of maximum. It can also be instructive to examine the ratio of OE(t) and TOE, as this will represent the expected fraction of the total benefits from the system which the customer has already received. At t=2, this ratio is 76%.

One may examine OVUB(t) with Chebyshev’s inequality (Ref. 21) to get a better sense of the behavior of OE(t). According to Chebyshev’s inequality, we may say that

t

(13) Pr{ OE(t) - h , / m e j$(z)dz I

0

OE(t) + hJOUUB(t)) 2 1 - 1 h

Of course, as OVUB(t) is already an upper bound for the variance of the quantity under consideration, this bound will be quite conservative. For h=2, we have for this example

t

(14) 3

Pr(1.380 c Sg(2)m 5 11.550) 2 - 4 0

4.4 Assessment of System Dynamic Per&ormance

We start by examining SE(t), the average state of the system as a function of time:

Figure 2. System State Expectation

SE‘ ( t )

OE(2) OVUB(2) 6.4648 6.555 1

Figure 3. System State Expectation Derivative

We can now examine the integral of the average state of the We use t* rather than t in this case, to system, OE(t*).

emphasize that this is not a static measure.

OE (t*)

1 2 3 4

Figure 4. Cumulative Performance Expectation

OE(t*) asymptotically approaches TOE, which is 8.4583 in this example; TOE represents the expected total utility one will receive from the system. As this is a non-repairable system, we would expect the average length of time (DTE(i)) the system spends in each state to be finite for all states above the minimal one and infinite for the minimal state. Calculations bear this out:

TABLE 5: Values of DTE(i)

DTE(0) DTE( 1) DTE(2) 00 1.5055 0.3867

We now examine SV(t), and note that it has a maximum at k1.37. One can easily imagine situations where the variance of the state of the system might be of equal or greater interest to the customer than the expected state of the system.

SV( t)

t

0 . 6

0.5

0.4

0.3'

0 . 2 '

0.1;

1 2 3 4 h ( 4 . t)

1 2 3 4

Figure 6. State 0 and State 4 Hazard Functions

functions, itemized below. The LWE measure was calculated for six different U(t)

U,(t) = 2e"' U( t)

LWE = 4.1944

0 . 0 . 0 . 0 . 0 . 0 .

1 2 3 4 5 6

Figure 7. U(t) Function #1

U, ( t ) = 9&3 + 2 t p LWE = 2.9228 U( t)

1 2 3

Figure 5. System State Variance

maximal and minimal states: It is also instructive to examine

4

the hazard function for the

1 2 3 4 5 6 Figure 8. U(t) Function #2

V(r) functions #1 and #2 may accurately model the time- varying nature of the customer's interest in the product in cases where the product or service is of greatest value when it is initially purchased, but where there is a steadily decreasing

290 1997 PROCEEDINGS Annual RELIABILITY and MAINTAINABILITY Symposium

positive utility for all no. For our military example, this could be reflected in a situation where a city defended while it evacuates.

LWE = 0.62776 t -; U3( t ) = G e

population is being

t

Figure 9. U(t) Function #3

U, ( t ) = 2te-t2 LWE = 3.435 1 U( t)

t 1 2 3 4 5 6

Figure 10. U(t) Function #4

U(t) functions #3 and #4 reflect cases where customer

interest grows to a maximum (at t=3 for #3 and at t = for #4) and then steadily fades. For our military example, this general form of U(t) could be appropriate when an important offensive is being planned at some unspecified time in the future, but when the military force in question must fight until the time of that offensive.

An observant reader will note that the U(t) functions illustrated here take the mathematical form of various continuous probability density functions. #1 is an exponential distributions, #2 is an F-distribution, #3 is gamma, and #4 is Weibull. By choosing U(t) functions which take the form of PDF’s, we allow additional interpretation of the returned result. Iffdt) is the PDF for a non-negative random variable T which represents the time of a particular event (such as an emergency for which the product or service under consideration will alleviate matters), then by selecting U(t) = fdt) we find that LWE is the expected state of the system at the time of that event. For our military example, this could have significant real-world value.

Additionally, as PDF’s for non-negative RV’s have the property thatJoTr(t)& = 1, they cause the range of LWE to be

[&, Qm]. This allows easy comparison of different systems. As one can see, and as one would expect, U(t) functions

which are large for small t and small for large t produce large

~

1997 PROCEEDINGS Annual RELIABILITY and MAINTAINABILITY Symposium

(“good”) values for L W . The U(t) functions which have non- trivial values at large values of time produce small (“poor”) values for LWE.

5 . CONTINUUM MODELS

Continuum models have been proposed and developed by a number of researchers (Refs. 22-31), and in essence extend the system and component state spaces to some segment of the real number line, typically [0,1]. One aspect of continuum models which to our knowledge has not been commented on in the literature is that any realistic continuum model should allow certain states of the system to have non-zero probabilities of occurrence. For example, if a non-repairable system may assume a continuum of states in [0,1], then when the system reaches its 0 state, it must remain there. Over time we expect a growing non-zero probability of this system being in its minimal state.

In general, the expected values of functions of a mixed random variable (RV) may be calculated through Stieltjes integration, as in Ref. 21. However, this can cause significant computational difficulties. It is more convenient to use a little- known theorem from probability theory (Ref. 17) when calculating measures which are functions of central moments:

w

(17) ELY” ] = dy ny”-’[l-- F, (y) + (-l)’* F, (-y)] 0

6. SOFTWARE NOTE

A set of Mathematicsl (Ref. 33) software packages which performs the calculations shown in this paper, calculates additional reliability measures not shown in this paper, and which performs the continuous and mixed measure calculations described in section 5, is available over the internet at the following FTP site:

ftp.u.washington.edu Ipubluser-supportedlreliabil

REFERENCES

1. E. El-Neweihi, F. Proschan, J. Sethuraman, “Multistate coherent systems”, J. Applied Probability, vol 15, 1978 Sep, pp 675-688.

2. R. E. Barlow, A. S. Wu, “Coherent systems with multistate components”, Math. Operations Research, vol3, 1978 Nov, pp 275-281.

3. J. C. Hudson, “The structure and reliability of multistate systems with multistate components”, Ph.W. Dissertation, 1981 ; Wayne State University, Detroit.

4. J. C. Hudson, K. C. Kapur, “Reliability analysis for multistate systems with multistate components”, IIE Trans., vol 15, 1983 Jun. pp 127-135.

5. T. Aven, “On performance measures for multistate monotone systems”, Reliability Engineering and System Safety, vol41, 1993, pp 259-266.

6. S. M. Ross, “Multivalued state component systems”, Ann. ofProb., vol 7,1979 Apr, pp 379-383.

7. K. Yang, J. Xue, “Dynamic reliability measures and life distribution models for multistate systems”, Int. L Reliabiliry, Quality, and Safety Engineering, vol2, 1995, pp 79-102.

8 . IS0 8402. Quality Management and Quality Assurance - Vncabulary, 2nd Ed., 1994.

9. A. Hoyland, M. Rausand, System Reliability Theory, 1994; John Wiley & Sons, Inc.

291

10. K. C. Kapur, “Reliability Engineering and Robust Design”, Proc. Ford 2000 Conference on Integrntion of Quality Methods, 1994 Nov 17-18.

11. M. Phadke, Quality Engineering and Robust Design, 1989; Prentice- Hall, Inc.

12. Z. W. Bimbaum, J. D. Esary, S. C. Saunders, “Multi-component systems and structures and their reliability”, Technometrics, vol3, 1961 May,

13. R. A. Boedigheimer, “Customer-driven reliability models for multistate coherent systems”, PhD Dissertation, 1992; University of Oklahoma.

14. K. Yu, I. Koren, Y. Guo, “Generalized multistate monotone coherent systems”, IEEE Trans. Reliability, vol43, 1994 Jun, pp 242-254.

15. D. A. Butler, “A complete importance ranking for components of binary coherent systems, with extensions to multi-state systems”, Naval Research Logistics Quarterly, vol26, 1979 Dec, pp 565-578.

16. K. C. Kapur, L. R. Lamberson, Reliability in Engineering, 1977; John Wiley & Sons, Inc.

17. P. Moran, An Introduction to Probability Theory, 1968; Oxford University Press, New York

18. R. A. Boedigheimer, K. C. Kapur, “Customer-driven reliability models for multistate coherent systems”, IEEE Trans. Reliability, vol43, 1994 Mar,

19. 0. Coudert, J. C. Madre, “MetaPrime: An interactive fault-tree analyzer”, IEEE Trans. Reliability, ~0143,1994 Mar, pp 121-127.

20. A. S. Heger, J. K. Bhat, D. W. Stack, D. V. Talbott, “Calculating exact top-event probabilities using Ell-Patrec”, IEEE Trans. Reliability, vol 44, 1995 Dec, pp 640-644.

21. E. Parzen, Modem Probability Theory and Its Applications, 1960; John Wiley & Sons, Inc.

22. L. A. Baxter, C, Kim, “Axiomatic characterization of continuum structure functions”, Operations Research Letters, 1987, pp 297-300.

23. L. A. Baxter, C. Kim, “Bounding the stochastic performance of continuum structure functions I”, J. Applied Probability, vol 23, 1986, pp

24. L. A. Baxter, “Continuum structures 11”, Mathematical Proc.

25. L. A. Baxter, “Continuum structures I”, J. Applied Probability, vol21,

26. H. W. Block, T. H. Savits, “Continuous multistate structure functions”,

27.0. Kaleva, “Fuzzy performance of a coherent system”, J. Mathematical

28. J. Montero, J. Tejada, J. Yanez, “General structure functions”,

29. J. Montero, “Observable structure functions”, Kybemeres, vol 22,

30. 3. Montero, J. Tejada, J. Yanez, “Structural properties of continuum

31. J. Montero, “Fuzzy coherent systems”, Kybemetes, vol 17, 1988, pp

32. H. Karlin, T. Howard, A First Course in Stochastic Processes, 1975;

33, S. Wolfram, Mathemafica, 1991; AddisonWesley. 34. H. W. Block and T. H. Savits, “A decomposition for multistate

35. B. Natvig, “Two suggestions of how to define a multistate coherent

pp 55-77.

pp 46-50.

660-669.

Cambridge Philosophical Society, vol99, 1986, pp 331-338.

1984, pp 802-815.

Operations Research, vol32, 1984, pp 703-714.

Analysis and Applications, vol 117,1986, pp 234-246.

Kybemetes, vol23, 1994, pp 10-19.

1993, pp 31-39.

systems”, European J. Operations Research, vol45, 1990, pp 231-240.

28-33.

New York Academic Press.

monotone systems,” J. Applied Prob., vol 19 no 2, 1982, pp 391-402.

system,” Advances m Applied Prob., vol 14, no 2, pp 434-455.

BIOGRAPHIES

Russell Brunelle Industrial Engineering University of Washington Box 352650 Seattle, Washington 98195 USA

Russell Brunelle is a doctoral student in Industrial Engineering at the University of Washington, Seattle. He received a BS in Physics and Mathematics from University of Puget Sound in 1992 and an MS in Industrial Engineering from University of Iowa in 1993. Russell Brunelle’s research has included computer simulation of JlT manufacturing environments, Kalman- filtering analysis of arctic sea-ice evolution, stochastic modelling of student flow through undergraduate engineering programs, and multistate reliability analysis of aviation communication systems. His doctoral dissertation work is in the area of customer-centered multistate reliability models.

Intemt (e-mail): sfpse@u.washington.edu

Kailash C. Kapur, Director and Professor Industrial Engineering University of Washington Box 352650 Seattle, Washington 98195 USA

Dr. Kailash C. Kapur is the director and a professor of Industrial Engineering at the University of Washington in Seattle. Previously, he was the director and a professor of the School of Industrial Engineering at the University of Oklahoma in Norman. He received his B.S. in Mechanical Engineering with distinction from Delhi University; his M.S. in Operations Research and Ph.D. in Industrial Engineering are from the University of California in Berkeley. Dr. Kapur has worked with General Motors Research Laboratories as a senior research engineer, with Ford as a visiting scholar, and with the U.S. Army Tank-Automotive Command as a consultant. He has served on the Board of Directors of American Supplier Institute, Inc., and has done extensive consulting in design of experiments, Taguchi Methods, design reliability, statistical process control and quality function deployment. Dr. Kapur co-authored the book Reliability in Engineering Design, has written chapters on reliability engineering for several handbooks such as Industrial Engineering and Mechanical Design, and has published numerous research papers in technical and research journals. He received the Allan Chop Technical Advancement award from the Reliability Division and the Craig Award from the Automotive Division of ASQC, and has been elected a Fellow of ASQC and IIE. He is a registered professional engineer.

Infemet (e-mail): kkapur@u.washington.edu

292 1997 PROCEEDINGS Annual RELIABILITY and MAINTAINABILITY Symposium