Capability Study and Measurement System Analysis
A Case Study at Bosch Rexroth AB
Victor Hultman
Industrial and Management Engineering, masters level 2016
Luleå University of Technology Department of Business Administration, Technology and Social Sciences
MASTER’S THESIS
Capability Study and
Measurement System Analysis A case study at Bosch Rexroth AB
Victor Hultman
2016
Supervisors:
Jörgen Selin, Bosch Rexroth AB
Erik Lovén, Luleå University of Technology
Master of Science in Engineering
Industrial Engineering and Management
Luleå University of Technology
Department of Business Administration, Technology and Social Sciences
II
“Quality is more important than quantity. One home run is much
better than two doubles”. – Steve Jobs (1955-2011).
III
Acknowledgements
This thesis is the final part of my education, Master of Science in Industrial Engineering and
Management at Luleå University of Technology, where the thesis is the result of a 20-week
study, project, and course within the field of Quality Management. The thesis was written
during January-May 2016 on mission from Bosch Rexroth AB at their manufacturing facility
in Mellansel, Örnsköldsvik, Sweden.
The work has been a big readjustment from the usual studies with both difficulty and fun but
especially educational. I believe that the experience of writing a Master’s thesis solely by
myself has been a great learning experience and will prove to be beneficial in my coming career.
I would first like to express my gratitude to everyone at Bosch Rexroth AB for the warm
treatment from day one. I would especially like to thank the production engineer Mikael
Larsson, Anders Westerlund, and Torsten Svensson in the R&D department for their expertise
in motor functionality and test processes; thank you for helping me succeed with my thesis,
without you it would have been impossible to finish the thesis. I also want to give an invaluable
love to my family for their support in everything from housekeeping, food, and access to a car
for transport to work in Mellansel; your daily support has meant everything to me. Finally, I
want to thank my supervisors Erik Lovén at Luleå University of Technology and Jörgen Selin
at Bosch Rexroth AB for their valuable advice and comments.
Mellansel, May 2016
Victor Hultman
IV
Abstract
Measurement System Analysis helps, along with statistical methods, provide deep knowledge
about the capability of a company’s measurement system. Through a better understanding of
how measurement systems perform, companies can base decisions on facts to promote quality
work.
This study examines the capabilities of Measurement System Analysis and examines how well
a test bench performs while testing motors at Bosch Rexroth AB in Mellansel. The study
examines both the repeatability and the reproducibility of the measurement system, i.e., the
variability that occurs when the same operator is doing repeated measurements on the same
motor and when different operators perform measurements on the same motor.
Bosch Rexroth AB manufactures high torque hydraulic motors. The case study covered the
motor type CA 50, which is the most produced motor today at the company. The tests were
made at one of the two existing test benches. The DMAIC method within Six Sigma was
selected to get a structured workflow in the case study.
Three parameters were examined for the motor type CA 50 – high pressure, external leaks, and
cleanliness – and touched on Destructive Testing. It was clear from the Measurement System
Analysis that the operators did not have a significant impact on the variability of the motor tests
for any of the three parameters. The precision of the measurement system differed between the
parameters. Concerning the parameter high pressure, the measurement system could be
considered to be acceptable, but with room for improvement. The parameter external leaks
showed an excellent measurement system with strong margins. The parameter cleanliness,
however, showed an unacceptable measurement system.
The capability study regarding the same three parameters showed different results. The
parameter high pressure showed a decent capability (𝐶𝑝𝑘= 1.17). Parameter external leaks
showed a high capability (𝐶𝑝𝑘= 1.17), but cleanliness showed a poor capability (𝐶𝑝𝑘= 1.09, 1.0
respectively 1.13) for the three different particle sizes 4µm, 6µm and 14µm.
This thesis resulted in four main suggestions for improvement concerning both the test bench
facility and motor type CA 50. The recommendations could help reduce the uncertainty in the
measurements of the parameter cleanliness and the precision of the parameter high pressure.
Furthermore, the thesis presents a suggestion on how outliers of parameter high pressure might
be handled and suggestions for adjusted tolerances on the parameter external leaks.
V
Sammanfattning
Mätsystemsanalys hjälper, tillsammans med statistiska metoder, att ge kunskap om hur företags
mätsystem presterar. Genom en bättre förståelse av hur processerna med deras mätsystem
presterar så kan företag basera beslut på fakta vilket främjar kvalitetsarbete.
Syftet med studien var att, med hjälp av mätsystemsanalys och duglighetsstudier undersöka hur
väl en testbänksanläggning presterar vid testning av den vanligaste produkttypen på Bosch
Rexroth AB i Mellansel. Studien undersöker både repeterbarheten och reproducerbarheten för
mätsystemet, dvs. de variationer som uppstår när dels mätsystemet får göra upprepande
mätningar på samma motor när samma operatör utför mätningarna samt när olika operatörer
utför mätningar på samma motor.
Bosch Rexroth AB tillverkar hydraulikmotorer där kundernas behov är motorer med ett väldig
högt vridmoment. Fallstudien omfattade motortypen med beteckning CA 50, vilket är den mest
producerade motorn idag på företaget. Testerna genomfördes på en av företagets befintliga
testbänkar. För att få en strukturerad arbetsgång i fallstudien användes Sex Sigmas arbetsmetod
DMAIC.
Tre parameter undersöktes för motortypen CA 50. Dessa var högtryck, externläckage och
renhet varav den sista komplicerades av att involvera förstörande provtagning. Vid
mätsystemsanalysen för dessa tre parametrar fastställdes det att operatörerna inte hade någon
märkbar inverkan på variationen för motorproverna men där resultatet för mätsystemets
precision skiljde sig åt beroende på vilken parameter det gällde. Parameter högtryck uppvisade
vad som kunde anses vara ett acceptabelt mätsystem men med utrymme för förbättring och
parameter externläckage på ett utmärkt mätsystem, dessutom med stor marginal. Parameter
renhet visade däremot på ett icke acceptabelt mätsystem.
Duglighetsstudien för de tre parametrarna visade på olika resultat. Parameter högtryck visade
på en hygglig duglighet (𝐶𝑝𝑘= 1.17). Parameter externläckage uppvisade en hög duglighet
(𝐶𝑝𝑘= 3.01), medan den tredje parametern, renhet, visade på en dålig duglighet (𝐶𝑝𝑘= 1.09, 1.0
respektive 1.13) för de tre olika partikelstorlekarna 4µm, 6µm and 14µm.
Detta examensarbete resulterade i fyra huvudsakliga förbättringsförslag som berör både
provbänksanläggningen och motortypen CA 50 där två av rekommendationerna kan hjälpa till
att minska mätosäkerheten för parameter renhet samt precisionen för parameter högtryck.
Dessutom skapades en rutin för hur avvikelser för parameter högtryck bör hanteras samt
underlag för en justerad tolerans rörande parameter externläckage.
VI
Abbreviations and definitions
BRAB – Bosch Rexroth AB. ISO – International Organization for Standardization. Measurement system
– Includes all the factors which is used to measure a desired dimension.
This can include sources as operators, measuring equipment, cabling,
software and tools (transducers, sensors, etc.).
TB – Test bench.
SEK – Swedish crowns.
External leaks – Parameter that indicates the amount of oil that leaks out from the
motor through joints and gaskets during use. Cleanliness – Parameter that indicates how pure a product is based on the
number of solid contaminant particles in the oil of the motor.
VII
TABLE OF CONTENTS
1 Introduction ........................................................................................................................ 1
1.1 Background .................................................................................................................. 1
1.2 Bosch Rexroth AB ........................................................................................................ 3
1.3 Problem discussion ...................................................................................................... 3
1.4 Aim of the study ........................................................................................................... 5
1.5 Limitations ................................................................................................................... 6
1.6 Thesis disposition ........................................................................................................ 6
2 Method ................................................................................................................................ 7
2.1 Study purpose .............................................................................................................. 7
2.2 Study approach ............................................................................................................ 7
2.3 Study strategy .............................................................................................................. 8
2.4 Data collection ............................................................................................................ 9
2.5 Reliability ..................................................................................................................... 9
2.6 Validity ....................................................................................................................... 10
2.7 Literature review ....................................................................................................... 10
2.8 Workflow – DMAIC ................................................................................................... 11
3 Theoretical Framework ................................................................................................... 13
3.1 Six Sigma ................................................................................................................... 13
3.2 DMAIC ....................................................................................................................... 13
3.2.1 Define............................................................................................................ 14
3.2.2 Measure ........................................................................................................ 14
3.2.3 Analyze.......................................................................................................... 14
3.2.4 Improve ......................................................................................................... 14
3.2.5 Control .......................................................................................................... 15
3.3 Statistical Process Control ........................................................................................ 15
3.3.1 Variability ..................................................................................................... 15
3.3.2 The Normal Distribution .............................................................................. 16
3.3.3 Autocorrelation ............................................................................................. 17
3.3.4 Control charts ............................................................................................... 17
3.3.5 Capability studies ......................................................................................... 19
3.3.6 Measurement System Analysis ...................................................................... 21
3.4 Sampling in pairs ....................................................................................................... 23
3.5 Process mapping ........................................................................................................ 24
VIII
3.5.1 SIPOC chart ................................................................................................. 24
3.6 Pareto chart ............................................................................................................... 24
4 Case Study through DMAIC ............................................................................................ 26
4.1 Define ......................................................................................................................... 26
4.1.1 Problem definition ........................................................................................ 26
4.1.2 Potential savings........................................................................................... 29
4.1.3 SIPOC chart ................................................................................................. 31
4.1.4 Process map .................................................................................................. 32
4.2 Measure ..................................................................................................................... 33
4.2.1 Data collection ............................................................................................. 33
4.3 Analyze ....................................................................................................................... 34
4.3.1 Measurement System Analysis ...................................................................... 34
4.3.2 Control charts ............................................................................................... 37
4.3.3 Capability study ............................................................................................ 38
4.3.4 Observations and interviews ........................................................................ 39
4.4 Improve ...................................................................................................................... 40
4.4.1 Improvement of PID control system ............................................................. 40
4.4.2 Adjusted tolerance limit ................................................................................ 40
4.4.3 New intern decision rules ............................................................................. 40
4.4.4 Production improvements ............................................................................. 42
4.4.5 Prioritization of improvement proposals ...................................................... 42
4.5 Control ....................................................................................................................... 43
5. Conclusions ...................................................................................................................... 45
6. Discussion ......................................................................................................................... 47
6.1 Personal reflections and results ..................................................................................... 47
6.2 Validity and reliability .................................................................................................... 48
6.3 Further studies ................................................................................................................ 49
References ................................................................................................................................ 50
APPENDICES
Appendix A – TB results ................................................................................................... 4 pages
Appendix B – MSA .......................................................................................................... 27 pages
Appendix C – Hypoteshis tests from Minitab ................................................................... 5 pages
Appendix D – Control charts ......................................................................................... 12 pages
IX
Appendix E – Capability study ....................................................................................... 11 pages
Appendix F – Unstructured interviews…………………………………………………………1 page
Appendix G – Integrated Contamination Monitoring System (CMS)………………………1 page
Appendix H – ISO Cleanliness Code ISO 4406 -1999……………………………………….1 page
1
1 Introduction This first chapter of the report clarifies issues and the choice of topic. The chapter presents
background, problem discussion, and a presentation of the company that the case study has
been performed at. Furthermore, the aim of this project and its boundaries are presented.
Finally, the disposition of the report is described.
1.1 Background
As globalization increases with emerging markets and increased trade between countries
(Arntz-Gray 2016), it is in society’s interest to guarantee sustainability of products and
manufacturing processes. Hoekstra (2015) also argues that it is impossible to claim that a
process can be considered sustainable if the activities of the process are not because each
process is the result of several other activities. According to Bergman and Klefsjö (2012), the
way people adapt to the increasingly complexity in the world affect the long-term sustainable
development and will improve both employees’ and clients’ quality of life. An increased
knowledge of what is important for the customers gives companies the possibility to constantly
improve their processes in terms of quality.
Addressing quality issues is needed to minimize the risk of rework, scrap, and poor products
reaching the customers, rendering unnecessary internal costs, customer complaints, warranties,
and even lost customers. If quality work is not good enough, companies risk weakening their
market shares (Garstenauer, Blackburn & Olson, 2014), and thus their sustainable competitive
advantage (Asif et al., 2013). Therefore, it is not considered to be a coincidence that the interest
within companies has increased regarding their own quality work due to the improvement of
performance such as the better use of existing resources and exploring new skills, capabilities,
and resources throughout the organization (ibid).
It is also important to carry out experiments to be able to create an understandning about
parameters as it will give more satisfied customers (Montgomery 2013; Bergman & Klefsjö
2012). This increased understanding can thus be a good basis for acting and improve the process
(Ljungberg & Larson, 2012).
According to Berman and Klefsjö (2012), there are two different aspects of quality: an objective
side that can be measured and a mental side that can depend on how customers have experienced
the product. A well-chosen quotation from the Japanese engineer Taguchi also summarizes
quality regarding how much shortcomings exists in product quality: “society’s overall losses
caused by the product after its delivery” (p. 22, Bergman & Klefsjö, 2012).
Many companies have adopted some or many components of the concept Total Quality
Management as they implement methods and tools for quality improvement to reduce internal
costs and increase customer satisfaction. Three key elements of the concept are learning,
continuous improvement, and customer satisfaction (Wiengarten, Fynes, Cheng & Chavez,
2013).
Bergman & Klefsjö (2012) underline the importance of making decisions based on facts and of
seeking knowledge about the outcome of processes and their variations. Statistical Process
Control is a methodology that in many cases can be very helpful in tracking and evaluating
processes and in the loch run reduce the variability in the processes (Gejdoš, 2015). It is also
2
useful for examining the capability of the operations as it provides the possibility to lower
quality costs and the need for inspections and provides a better knowledge of the processes and
machinery (Brännström-Stenberg & Deleryd, 1999).
Capability analysis is used to evaluate and increase the performance of the company's
manufacturing processes. Today, many companies do not work with capability analysis that
otherwise would have been beneficial to their process performance (Yang, 2013). Capability
studies, together with Statistical Process Control, can therefore help companies become more
efficient at investigating quality problems and solving them thereby allowing companies to
survive market competition (Shinde & Katikar, 2012).
There are a number of successful projects carried that focus on capability studies inspired by
the management strategy Six Sigma. One of these projects was carried out at Motorola. The
project included a fivefold growth in sales and a climb in profits by almost 20 % and could thus
obtain total savings of $14 billion (Zailani, Iranmanesh & Ramayah, 2015). Another example
comes from General Electric, which obtained savings of as much as $700 million in 1997 after
have invested $400 in the Six Sigma concept (ibid).
Bergman and Klefsjö (2012) describe the problem solving method DMAIC within Six Sigma
together with the use of statistical tools as key elements in management of complex problems.
Sörqvist and Höglund (2007) mention the DMAIC method as the primary approach when
dealing with the problem of unwanted variability.
The company Bosch Rexroth AB (BRAB) produces several direct drive motors of different
types. These motors have full torque from a stationary start with zero velocity (number of
revolution). One of these motors is the CB motor, which is suitable for many heavy-duty
applications such as shredders, feeders, and roll mills. BRAB also manufactures a motor with
the name CMB, which has the world’s highest torque-to-weight ratio (Boschrexroth.com, 2016;
U. Stromberg, Director Product Management, April 28, 2016). Other motors that the company
manufactures are CBP, CAB, and Viking. Viking is different from the other motors as it rotates
on its drive shaft whereas the other motor types have the cylinder blocks rotate inside the
motors. Finally, there is the CA motor. This motor is a compact motor. The name Compact
come from the purpose, i.e. for heavy-duty applications where size and weight are significant
issues. This motor is popular because it is small, has numerous mounting options, and can deal
with large shock loads, all characteristics that give it real competitive advantages (ibid).
Having an overall understanding and estimation of the variability in the company's various
processes has long been very important for the company’s long-term success. To make sure that
the customers are satisfied, the company has established high tolerance requirements for
individual motor components. The critical quality parameters are measured in the measurement
room without any wear on the motors. When a motor has been fully assembled, a final test is
made on the test bench (TB) where tests based on tolerance requirements for the different motor
models are conducted. Up until the beginning of 2016, no official capability study has been
carried out on the TB, and therefore it was of interest to BRAB to evaluate the capability of the
TB process as for some time the company has dedicated efforts to create capable processes in
production. Therefore, the company needs quality assurance with respect to TB, i.e. how well
the TB performs in its measurments. These results are very important as it sets the capability
on the final products of the company in terms of the parameters which are evaluated and is
therefore the last quality check before motors are sent to customers.
3
1.2 Bosch Rexroth AB
BRAB is one of the world's front runners of hydraulic motors, produces motors that can produce
significant torque for applications in industries such as the mining industry and various
manufacturing industries requiring long conveyor belts (Boschrexroth.com, 2016) The
production plant, located in the small village Mellansel outside the town of Örnsköldsvik, has
over 300 employees and a turnover of about one billion Swedish krona (ibid).
Previously BRAB was only producing motors to order, but recently it has introduced the well-
known hauling system Kanban1 to obtain a more efficient production and decrease tied capital
by visualizing any need of material for production purpose. The company also hopes that the
Kanban system will reduce waste. Up until today, several Six Sigma projects have been
conducted at BRAB. These projects also include small specific improvements carried out by
small improvement groups within the company. There also exists a special problem-solving
group that consists of members from several departments whose task is to handle quality
problems at meetings held on a weekly basis. This group is designated to solve the listed
problems. Quality work like capability studies and Measurement System Analysis are important
since the quality of the products is one of its main competitive advantages. Today, BRAB is
ISO certified for ISO 9001: 2008 (a Quality Management System) in 2013, and ISO 14001:
2004 (an Environment Management System) in 2014 (L. Lindblad, personal communication,
January 28, 2016). The company now faces a constant challenge within the industry from
competitors. Therefore, BRAB is significantly investing in producing high quality products.
This focus on quality creates a base for a higher order intake in the future. However, the order
intake for the motors is not as high as the company wants it to be; peak orders were in 2011 and
the order intake from 2012 until 2015 has decreased to a much lower level. According to the
market department’s Director for Product Management, it is not possible to point out one
specific reason for why the demand in terms of new orders has declined since different motor
models have different applications. These applications include, for example, the paper industry,
the plastic industry, and the fishing industry and thus depend on different market prices such as
the price of crude oil, copper, and iron ore. These prices also depend on the oil industry and the
mining industry. Currently, the company’s financial goal is to return to sales level from five
years ago by improving quality (L. Lindblad, personal communication, January 28, 2016).
1.3 Problem discussion
According to the Head of Measurement Technology, Technical Arrival & Control and
Capability at BRAB, there is a follow-up of deviations every morning (if necessary in the
afternoon) during a meeting with the head of delivery testing. According to a hydraulic engineer
at the research and development department at BRAB, the TB has four purposes: to verify that
the motor works, to obtain measurements of the external leaks, to flush the motor system to
make it clean, and to break in the motor.
At the end of the TB motor tests, according to production engineers and operators, some motors
need to be retested, in some cases several times, since not all tolerances were met as determined
by the computer testing software. This lack of 100% compliance has created doubt among the
personnel with respect to the TB and the products (i.e., the hydraulic motors). The staff can be
1 A tool that helps control the logistics chain of production to achieve just-in-time delivery (Al-Baik & Miller,
2014).
4
said to have experienced a possible uncertainty about whether or not the test results can be
trusted. Is the motor really as poor as the measurements show or is there something wrong with
the test procedure? In some cases, a motor has not even been approved despite several re-tries.
This lack of consistency has in turn resulted in rejected motors and thus extra work as the
company had to send the motor back to the assembly to look for unwanted wear or failure due
to improper installation, inadequate washing, or other causes of problems.
The process chart in Figure 1.1 below shows the workflow and how the different processes are
connected and where the TB process is involved, not including storehouses and inventory. As
can be seen, not every manufactured motor part is measured in the measuring room for each
article; some parts are selected for inspection at regular intervals depending on which article it
concerns, since the criticality of the parts decides the need for inspection and consequently
inspection intervals.
Figure 1.1 Process chart of the workflow
In the TB, 13 different motor parameters of the CA motor are tested. The parameters include
the parameters high and low pressure, number of revolutions, temperature for both intake and
motor house, external leaks, and three different contamination parameters concerning
cleanliness (ISO 4406). There is also a maximum pressure test for the motor house. These two
are done in a static position. To test a motor, an operator secures the motor onto the TB and
connects the oil tubes and the pressure connection with a jackhammer. The motor test can then
be started from the control room. The test takes about 20 minutes for each motor.
Up until the beginning of this study, the company had no information about the accuracy of the
TB tests, resulting in an uncertainty about the variability in the measurement results and if the
variation comes from differences in the products (i.e., the motors) or if measurement system is
flawed or a combination of these two factors. Therefore, it was of interest for the company to
determine how capable and reliable the test process with TB actually is and how suitable the
measurement system is to deliver reliable measurements and sort out products outside
specifications so that no defect motors are sent to customers.
5
Figure 1.2 One of the two TB facilities for the CA motor.
1.4 Aim of the study
The aim of the study was to evaluate the capability in the TB process concerning the CA motor
at BRAB with help of Measurement System Analysis and to investigate if the TB facility
performs as it is supoosed to do and to what extent the products meet the stated specifications
(tolerances) of some critical parameters. To do this, it was vital that the obtained measurements
were reliable and that the variability coming from the measurement system was quantified and
if necessary reduced. Knowledge about the variation in the measurement system was also
important in order to estimate the actual variation in the products and the capability of the
production processes. Therefore, a Measurement System Analysis was a necessary step to
estimate the capability for both the TB facility and the products. The capability of the products
was regarded from a customer’s perspective, while the measurement system in the TB facility
was seen as an internal matter.
In order to fulfil the aims the following study questions were established:
1. What uncertainty (variability) has the measurement system in the TB for the motor type
CA?
2. What capability exhibits the motor type CA?
3. How should the procedures for process control, Measurement System Analysis, and
capability studies be designed in the company in order to control and assure that the new
levels of improvements are maintained, achieve a capable TB process and get knowledge
on how the capability for the products varies over time?
These study questions were helpful for the improvement phase and the control phase of the
project when suggestions for improvement were brought up. Furthermore, a secondary purpose
of the case study was also to come up with proposals for improvements that could be
implemented and monitored. Together with the requirement of a Measurement System Analysis
6
to be performed, i.e. that a collection of data needed to be made, a case study as research strategy
seemed to be best suited for the purpose.
1.5 Limitations
This case study was limited to examining the measurement system at one specific TB and
focusing on one specific motor type, this due to that BRAB desired that the study was conducted
on these. To make sure proper limitations were made, the decisions were made during the define
phase in section 4.1, based on frequency and volume. In the define phase an identification was
made of which motor type and TB that were considered most critical. This study showed that
the motor type CA 50 and its TB 1 were most relevant to limit the study to. The DMAIC method
was selected for the case study methodology structure since it was well known by the author
and to some extent also by the case study company, Bosch Rexroth, and since DMAIC has
proven to be successful in other companies in similar situations. Another option could have
been to use the PDCA cycle, which stands for Plan, Do, Check and Act, as a working method,
but DMAIC was considered to be a better choice since the project was to last for several months.
A more general description of Six Sigma and DMAIC can be found in section 2.8 and 3.1.
1.6 Thesis disposition
Instead of the traditional design of a scientific report with empiricism, analysis, and results, this
report will have the case study designed according to the working method DMAIC. This choice
was made to make it easier for the reader to follow the actual project phases carried out and to
give a more comprehensive overview of the focus area and thus a better structure. Figure 1.3
shows the traditional report (orange) disposition to the left, the DMAIC structure (green) in the
middle, and the case study’s five phases (blue) to the right.
Figure 1.3 Visual description of the disposition of the report
Introduction
Theoretical Framework
Method
Empiricism
Analysis
Results
Conclusions
Discussion
Introduction
Method
Theoretical Framework
DMAIC
Conclusions
Discussion
Define
Measure
Analyze
Improve
Control
7
2 Method This chapter describes the study's approach. Here the requirements that exist for a scientific
report are represented. The chapter will talk about which research purpose, strategy, and
approach selected for the study. Furthermore, an explanation is provided about how the data
were carried out and what actions were taken to keep high reliability and validity. A description
is also made of the study literature. Finally, the study’s workflow is described according to the
DMAIC method and the choices within its individual phases.
A summary of the choice of research purpose, research approach, and research strategy for this
thesis are presented in Table 2.1. Table 2.1 briefly shows the general plan of how the author
worked to answer the research questions.
Table 2.1 Summary of method choices in the thesis
Method Overview Chosen Method
2.1 Study Purpose Exploratory, Descriptive
2.2 Study Approach Deductive, Abductive
2.3 Study Strategy Case Study
2.1 Study purpose
Below, I describe the three research approaches according to Saunders, et al. (2012).
Exploratory research, according to Saunders et al. (2012), this type of study aims to
investigate what has occurred and describes several possible approaches to successfully
achieving the goal of the study. Examples of approaches are interviews and literature studies as
these will be helpful for creating a good understanding of the problem to be investigated
(Saunders et al., 2012).
Descriptive research provides an accurate picture needs to be created about situations and are
often used in conjunction with other types of studies (Saunders et al., 2012). According to
Ferreira, et al. (2013), this type of research usually includes observations and analysis.
Explanatory research, according to Saunders et al. (2012), describes a study's relevance when
a clarification of the relationship between the variables is needed to understand a specific
situation or problem.
Because this thesis consists of a case study where both collected and available data were used
and formal approaches like interviews with employees were done to examine the current TB,
an exploratory approach was used. The study also included a descriptive research purpose since
the beginning of the thesis contained a description of the current situation.
2.2 Study approach
Saunders et al. (2012) address three research approaches:
The deductive approach begins by creating a theory about the chosen topic and then develops
a question around the topic. The prepared question is then tested by already proven theories that
are considered to best fit the specific situation by collecting data and performing an analysis.
8
The inductive approach collects empirical material using different methods that try to fit the
established theory. This approach uses the collected data to design a new theory around the
subject (Saunders et al. 2012).
The abductive approach is a combination of the deductive approach and inductive approach
since the new theory is created based on the already existing theory and then moves to the
previous theory.
Since existing theories within the field including indexes for Measurement System Analysis
and capability studies were tested against a collection of data to see the outcome for a specific
case, a deductive research approach was used in this thesis. This use of existing theory was
needed to successfully be able to analyze and come up with conclusions from the collected data
and be able to answer the created study questions. As the approach of the study was based on
the DMAIC method in which the collected data during one of the phases were used in a
following phase, an abductive research approach was also used.
2.3 Study strategy
According to Saunders et al. (2012), the research strategy will serve as a plan for how the study
will be carried out and dictate the data collection method and subsequent analysis.
Yin (2007) notes the five most common research strategies.
Experiment is the basic strategy in terms of research. The strategy is used when an
investigating why or how something happens. The strategy is also said to fit best when the
purpose of a study is exploratory or explanatory (Saunders et al., 2012). Yin (2003) notes that
the experiment strategy addresses questions like how and why (ibid).
Survey is best suited as a descriptive or exploratory purpose combined with a deductive
approach. Data collection is normally carried out using surveys and interviews (Saunders et al.,
2012).
Archival Analysis, according to Saunders et al. (2012), is very adaptable to all possible
investigative purposes. It also describes how data is collected. According to Yin (2003),
archival analysis addresses question like who, what, where, how many, and how much, focusing
on past events.
History is performed when investigating a particular event at a particular time in the past. In
this type of strategy, questions like how and why are of interest (Yin, 2007).
Case Study, according to Yin (2007), describes a methodology and strategy with an approach
based on data collection techniques and specific analysis of the collected data. This strategy
addresses questions like why, what, and how (Saunders et al., 2012).
I used Yin’s (2007) recommendations to determine which strategy will work best for this thesis.
In addition, this study concerns current TB process. Since the study had the need for extensive
observations and interviews to create and understandning of the problem and TB process, a case
study seemed to fit best as research strategy, but also to make an appropriate use of methods
for the analysis and the data collection for the Measurement system analysis. These
observations and interviews were also used to determine analytical methods and data collection.
9
Using Measurement System Analysis (i.e., the data collection), this case study examines
complex problems at one TB process as an analysis unit by building on the DMAIC method.
2.4 Data collection
Saunders et al. (2012) describing two ways to perform data collection. The first is to collect
primary data: data are collected through, for example, maintained interviews, observations, or
measurements made. The other way is to collect secondary data: the data collected are already
available, for example, in a company’s database or in existing documents.
This thesis primarily uses secondary data from the company’s saved test protocols and
electronic documents. Primary data, also relevant for this work, were collected by interviewing
people using the TB and by observing the TB including the process. The observations, for
example, revealed information about the steps involved in the TB process. Primary data were
also collected togheter with operators in form of measurements from a reference motor for the
Measurement System Analysis (Gauge R&R study) concerning the two parameters high
pressure and external leaks. The people who were included in the interviews and how the data
were collected and analyzed are presented more in detail in Table 2.2, in which people/operators
were selected by recommendations of the relevant manager.
According to Saunders et al. (2012), a classification of data can be done in two ways –
qualitative data or quantitative data. Qualitativ data is any data that cannot be quantified, for
example, attitudes, experiences, and values that are captured through observations or interviews
(ibid). Walliman (2011) describes quantitative data as data that can only be described in words
rather than numbers. In addition, quantitative data, unlike qualitative data, can be analyzed with
statistical methods (ibid).
For this thesis, both qualitative data and quantitative data were used. The qualitative data came
from own observations as well as various forms of interviews (semi-structured and unstructured
interviews) to allow some spontaneity to occur, give opportunity for questions to develop during
the course of the interview, but also allowing new ideas to be brought up during the interviews.
The quantitative data came from historical samples of testing on the three chosen parameters:
high pressure, external leak, and cleanliness. When necessary, quantitative data also included
data collected from sample tests for the same three parameters. That is, this study relies on
multiple data collection methods (Saunders et al., 2012).
2.5 Reliability
Reliability reflects the trustworthiness of an investigation (Yin, 2007). That is, the same results
must be obtained when using the same method. Similarly, Saunders et al. (2012) describe high
reliability as the ability to reproduce the same results using several methods.
The reliability for the study was strengthened by the use of several sources, enhancing the
information from the collected data. In addition, several key people who were familiar with the
area also examined the contents of the report. Reliability was also ensured by using the
standardized DMAIC approach regarding planned working time, an approach inspired by the
Sörqvist and Höglund (2007). Reliability was also ensured by using pre-determined interview
10
questions. In addition, the study used pre-existing and repeatable experiments to collect data
needed for the study.
To further increase reliability, various types of interviews were used (semi-structured
interviews can be seen in Appendix F). As with most studies, a 95% confidence level was
selected (a p-value of 0.05; i.e., a 5% significance level). Because this study uses the established
Nested Gauge R&R approach for some parts of the Measurement System Analysis, future
researchers will be able to analyze similar destructive testing situations.
2.6 Validity
Yin (2007) describes validity as the ability of an investigation to measure what is supposed to
be measured. To achieve high validity, Saunders et al. (2012) disclose two divisions of
investigation perspectives – external validity and internal validity. External validity generalizes
the obtained results, whereas internal validity investigates the accuracy in terms of
measurements.
The validity in the study was therefore strengthened by making observations on how the
operators confirm their work complies with the instructions for the workplace. It was also
important to make sure the observations did not influence the measurements (e.g., the operators
may make an extra effort when connecting and disconnecting each motor). To prevent this
possibility, the author observed the TB under normal conditions.
To further increase the validity, brainstorming sessions were held with R&D workers. This
strategy created an accurate and comparable understanding of the process behavior to find
potential problems. Since both unstructured and semi-structured interviews were carried out
along with a workshop with people from the company, it could be said this study relied on a
type of triangulation. That is, triangulation uses several data collection methods to ensure that
the data accurately depict the phenomenon under study (Saunders et al., 2012). Finally, to make
it possible for others to repeat this study, the working method was based on the
recommendations from Sörqvist and Höglund (2007). That is, the level of validity was
increased. To know which data would be needed, the data collection took the survey questions
into account. Triangulation was then used to clarify what data would be needed. The data was
then collected from observing the operators. The planning for this was made through
conversations with a production planner to get relevant motors successively to the assembly so
that the same motors were observed in the trial tests. This was desired by the head of the test
process so that the implementation and the data collection would be as efficient as possible and
have a minimal impact on daily production.
2.7 Literature review
In this thesis, a literature review was made, which intends to work like a support and fortify
knowledge within the area for theories, models, and the planned DMAIC approach. The
literature was also continuous (i.e., a continuous collection of data was carried out throughout
the study) so the author could make additions when it was necessary. In addition to the extensive
literature review within the subject of quality technology, the search tool called “discovery tool”
from the library of Luleå University of Technology was used to find information. The tool itself
is available at the library’s website. The literature review was limited to publishing dates from
11
1990 to minimize the risk of using outdated literature since the author considered that the risk
of missing something essential before the 1990s could be seen as low. The following search
terms were used: Process Capability, Capability Studies & Indices, Measurement Systems
Analysis, Gauge R&R, and DMAIC. Moreover, the articles were all peer reviewed. The
literature reviewed was studied as the thesis proceeded, where most of the articles was gathered
in the introduction of the thesis when theory and problem description were described.
2.8 Workflow – DMAIC
The workflow that this thesis with containing case study followed was the well-defined working
method DMAIC. The DMAIC method considered more effective than, e.g. a PDCA cycle since
there would be a clear emphasis on statistical methods and tools in the study. Typical tools and
information sources within a Six Sigma project are a flowchart (e.g., SIPOC), which is used to
clarify the problem, and a control chart that helps identify systematic variabilities (Sörqvist &
Höglund, 2007). Measurement System Analysis and capability studies are other tools
mentioned together with information sources like observations and interviews (Montgomery,
2013; Bergman & Klefsjö, 2012).
Below is a summary of each purpose for the different phases along with the quality tools and
information sources that were used in this study and how the information was obtained in each
of the phases: define, measure, analyze, improve, and control (Table 2.2). Information and the
related data collection took place in the beginning of the case study, while tools for analysis
were used later in the case study.
12
Table 2.2 Summary of the workflow according to the DMAIC method in the case study
D
• Purose: Clarify problem, get a better understanding of the process and calculate potential savings
• Tools: Pareto-chart, SIPOC, Process mapping
• Information sources: Observations: TB installation and assembly, Semi-structured interviews: Head of measurement technology , technical arrival & control and capability, operators, company provided documents
M
• Purpose: Collect enough data for the project
• Tools: Excel
• Information sources: Qualitative data: unstructured interviews with operators, sample tests on TB with operator. Quantitative data: Data stored electronically, On place
A
• Purpose: Analyze the collected data to evaluate the capability and the variability in the measurement system, products and the process
• Tools: Measurement System Analysis (MSA), control charts of type shewhart, capability studies
• Information sources: Minitab 17 Statistical Software, historical test protocols, own measurements
I• Purpose: Establish proposals and recommendation for
improvements to the company
• Tools: Previous phases, Brainstorming
• Information sources: Quality Engineer, Operating Engineer, operators, Head of measurement technology , technical arrival & control and capability
C:• Purpose: Suggest methods for follow-up and Establish
routines and instructions for staff, as well spread the methods to other motors and processes (TB)
• Tools: -
• Information sources: Previous phases
13
3 Theoretical Framework This chapter lists the theories and tools that were used in the thesis. Initially, a more detailed
description is given of Six Sigma, DMAIC, and Statistical Process Control. Next, there is a
description of Sampling in pairs, process mapping, SIPOC chart, and finally Pareto chart.
3.1 Six Sigma
Six Sigma is a business strategy founded by Motorola in the 1980s. The purpose of the strategy
is to improve the efficiency, performance, and customer satisfaction in organizations by
reducing operational costs and increasing profitability. Currently, Six Sigma is considered as
one of the main quality tools developed in the 21st century. (Sin Zailani, Iranmanesh &
Ramayah, 2015). Six Sigma can be viewed in several different ways, partly as a level of quality,
but also as a problem-solving approach and a management philosophy. The name Six Sigma is
derived from the Greek letter “σ” and is used as a statistical measurement of the variability in
a process where the level of 6σ standard deviations refers to when a process only produces 3.4
defects per million opportunities. Within this boundary, the variability in the mean is allowed
to be 1.5σ standard deviations, a variability that in practice equivalent is 99.9997% of the
exchange (Sin Zailani et.al, 2015); (Bergman & Klefsjö, 2012). In a Six Sigma project the
improvement work usually follows the structured problem-solving framework with the name
DMAIC – Define, Measure, Analyze, Improve, and Control. Six Sigma strategies will therefore
be put to good use when the goal is to reduce the variability in the organization’s processes
(Easton & Rosenzweig, 2012).
3.2 DMAIC
There are a variety of optional improvement tools that aim to achieve continuous improvement
in terms of quality. The DMAIC model is one of them (Sörqvist & Höglund, 2007). Sörqvist
and Höglund (2012) also describe the DMAIC methodology as a systematic way of working
where the method is used as a project model with a focus on in-depth data collection,
measurements, and analysis. The method is based on five key success factors. These factors
create understanding of the problem, base the problem solving on facts, identify root causes,
implement effective solutions, and secure the achieved results.
The method usually works as a core tool in Six Sigma projects but can be used as a framework
when other applications for improvements need to be made. The name DMAIC is an
abbreviation for the different phases: define, measure, analyze, improve, and control (Gejdoš,
2015). A visual description of the different phases can be seen in Figure 3.1.
14
Figure 3.1 Working method DMAIC
3.2.1 Define
The define phase defines the problem. The actual problem and nothing else will be the focus
for improvement areas. The define phase clearly describes the problems. For the formulation
of the problem to be extra clear, it is recommended that questions like what, where, when, and
who are asked (Sörqvist & Höglund, 2007). According to Gejdoš (2015), it is good to establish
a project plan. Sörqvist and Höglund (2007) suggest a determination of what the project’s
potential savings will be, the significance of the project, and an identification of customers and
their needs. Finally, the mapping of the process should specify the purpose of the study situation
(ibid).
3.2.2 Measure
The second phase of DMAIC, Measure, involves collecting data and information in a systematic
manner (Sörqvist and Höglund 2007). In this phase, on-going elements occur such as deciding
what information is needed and identifying important dimensions of measurements. After a
determining what should be measured, the measurements are made (ibid). Gejdoš (2015) also
points out the importance of determining what should be measured.
3.2.3 Analyze
According to Montgomery (2013), the purpose of this third phase is to identify the cause and
effect relationship to the variability or defects that occur. Montgomery notes that Measurement
Systems Analysis is an important way to perform statistical analyses in these situations.
Bergman and Klefsjö (2012) and Sörqvist and Höglund (2007) specifically mention SIPOC
Chart and cause-effect diagrams as helpful analytical tools. Cause-and-effect diagrams are also
something that is recommended by Gejdoš (2015).
3.2.4 Improve
For the fourth phase the improvement will come from the work in the earlier phases, especially
the analysis phase. Previous work can be seen as a foundation to determine good solutions to
the problem or problems that have been identified and analyzed. Usually several possible
Define
Measure
AnalyzeImprove
Control
15
solutions are developed using various creative methods (Sörqvist & Höglund, 2007). After the
solutions are tested, the method that seems to be the best will be implemented. The type of
solution can be for either the individual process that is being tested or for a solution for the
entire operation depending on the situation (Gejdoš, 2015).
3.2.5 Control
According to Sörqvist and Höglund (2007), it is necessary to follow-up the results to see if the
improvements carried out have succeeded and if they can be held although a final follow-up of
the project is made with verification of the results that the project has given. The result should
provide updated procedures and management systems. It is also very important to spread the
knowledge throughout the organization (ibid).
3.3 Statistical Process Control
Statistical Process Control (SPC) is a proven method to achieve capable processes and improve
the capability of these by reducing the variability that exists. The method comes in handy when
improvements in terms of quality and productivity are of interest (Montgomery, 2013).
Bergman and Klefsjö (2012) note that SPC finds possible contributions to the variability that
should be eliminated. According to Gejdoš (2015), SPC is much more than just a method to
reduce this discernible variability as it also describes the SPC as a way to find the cause of a
variety of problems concerning quality, delivery times, procedures, materials, equipment, and
how things are made. Any process that can be measured can be evaluated using SPC. Analytical
tools and control charts make it possible to detect critical causes in an earlier stage and
experiments that use Measurement System Analysis can then be used to understand or explain
the variability (Montgomery, 2013). This strategy allows changes to be implemented so that the
desired results can be achieved (Gejdoš, 2015).
3.3.1 Variability
Variability is something that exists everywhere and is often a reason for the failures that occur.
Due to this, variability will affect the results since it cannot be controlled as much or as often
as desired. This in turn makes the understanding of how variabilities appear very important. In
terms of processes, the identified critical dimensions will be very important as this will allow
for an understanding of how the variability looks for these dimensions by analyzing the
collected data using statistical methods. This understanding will then be the basis for solving
the problem (Sörqvist & Höglund, 2007). The variability that is included within a process could
have many different causes that together add up to the total observed variability. Berman and
Klefsjö (2007) divides this variability in two different types – discernible variability and
random variability. Eliminating the discernible variability can create positive results like
reduced costs and increased customer satisfaction. If this is possible, then the process can be
considered to be in statistical control (Figure 2.1). When the process becomes stable, work can
be conducted to improve further the capability of the process (Berman & Klefsjö, 2007).
16
Figure 3.1 Visual description of the difference between a non-stable process and a stable process
when discernible variability has been eliminated, inspired by Bergman & Klefsjö (2012)
3.3.2 The Normal Distribution
Normal distribution will be the most important of distributions to take into account
(Montgomery, 2013). The normal distribution, which can be seen as a mathematical model, can
be described in a symmetrical manner with the observed values obtained in situations. The
reason that the distribution is symmetrical is that there are only random variabilities that occur,
as the values mainly lie around the mean, μ (Sörqvist & Höglund, 2007).
Figure 3.2 The Normal distribution, inspired by Sörqvist & Höglund (2007)
The normal distribution explains how the expected value (μ) and standard deviation (σ) are
distributed. The different levels of the standard deviation that can be seen in Figure 2.2 above
and will contain different amounts of the observed values . For example, the level of 1σ includes
approximately 68.2%, whereas the level of 3σ includes as much as 99.7% of all observations.
(Montgomery, 2013). It is also possible to use analysis software to obtain graphical results of
these observation values to so-called normal contribution plots. This makes it possible for two
different people to reach different conclusions from the graphical results even though they are
using the same data in their analysis. Therefore, it becomes important to not make a full
conclusion before an objective addition has been made to the analysis, for example, with the
use of Shapiro-Wilk test.
17
3.3.3 Autocorrelation
Data from autocorrelation means is dependent; that is, the values that are above the mean will
be followed by another value above the mean, also called a positive autocorrelation. Similarly,
values that fall below the mean will be followed by another value that falls below the mean.
For this, the values of the data may have longer runs on either side of its mean (Montgomery,
2013).
The type of behavior with the runs above or below average (i.e., positive autocorrelation) is
also said to increase the frequency of false alarms in the process. Values that depart from each
other have a negative autocorrelation, a condition that results in fluctuating data (ibid).
To break up the autocorrelation that seems to exist after an autocorrelation test, either reduce
the frequency of sampling from the process or fit an exponentially-weighted moving average
(EWMA) chart or a Cusum chart and use these when plotting the residuals instead of the true
values. This approach will counter the negative effects that occur in Shewhart control charts,
reducing its performance (Montgomery, 2013; Franco, et al., 2014).
3.3.4 Control charts
In 1924, Walter Andrew Shewhart developed control charts to check that manufactured parts
remain within their control limits. The theory is based on reducing the spread of the processes
by finding discernible causes of variability. By measuring one or more important dimensions
regarded, a control chart of the measurement values can then be established with the associated
control limits, which can be done by calculation. Control limits determine if the process is in
statistical control. If all plotted values are maintained inside the control limits, the process can
be considered to be in statistical control.
Control charts are also included as one of the tools in the “seven enhancement tools” (QC tools)
(Bergman & Klefsjö, 2012). In addition to these measurements and boundaries, a typical control
chart has a centerline in the form of the mean of the average (Fleet, 1999). For the process to
be considered to be in control, it needs as much as 99.73% of all measured values to be within
control limits. The following analysis of a control chart looks for any systematic patterns that
need to be followed (ibid).
When control charts are constructed, the choice of its control limits will be important. After a
several experiments in the 1920s, Shewhart calculated an upper and a lower control limit with
three standard deviations (3σ) that he considered to provide the conditions to minimize financial
losses. After this, the upper and lower control limits were placed with a total distance of 6σ
standard deviations. This will bring 3.4 deviations of a million observations (ibid). According
to Montgomery (2013), the Wester Electric rules can be used to determine whether the process
is not in statistical control. The Wester Electric rules are not applicable if any of the following
criteria for the control chart exist:
1. One is outside the 3σ control limits.
2. Two of the three following points are outside the 2σ limits.
18
3. Four of five consecutive points lie on or outside the 1σ limits.
4. Eight subsequent points are on the same side of the centerline.
Figure 3.3 Example of a typical control chart with its 14 observed values; mean in the form of a
centerline and control limits
Montgomery (2013) also describes a number of reasons why control charts have become so
popular. One of these reasons is because of their ability to provide information on how capable
the process is, and thus can be used to take advantage of capability studies. The type of data
that is used is also important to take into consideration when choosing the most suitable type of
19
control chart (ibid). To do this, one can take advantage of an established guide to Univariate
Process Monitoring and Control (Figure 2.4) (ibid).
Figure 3.4 Guide to Univariate Process Monitoring and Control, inspired by Montgomery (2013).
3.3.5 Capability studies
Capability studies are very important to constantly improve quality and productivity (Wu, Pearn
& Kotz, 2009). Furthermore, Jeang (2008) notes that capability studies are used to measure and
control processes based on quality standards on the basis of capability measurements where the
collected data are analyzed to find potential quality improvements (ibid). The most common
capability measures, according to Berman and Klefsjö (2012), are the capability index 𝐶𝑝 of
the process and the corrected capability indexes 𝐶𝑝𝑘 , 𝐶𝑝𝑚 , and 𝐶𝑝𝑚𝑘 of the process. The
capability index 𝐶𝑝𝑚 also takes into account if the average value of the process deviates from
its target value, T. The generalized capability index 𝐶𝑝𝑚𝑘 takes instead into account the cases
when T is located in the middle of the tolerance range. Each capability index can be calculated
from equation 3.6, 3.7, 3.8, and 3.9 below, where 𝑇𝑢 𝑎𝑛𝑑 𝑇𝑙 stands for the upper lower tolerance
limit, respectively, and 𝜇 𝑡ℎ𝑒 𝑚𝑒𝑎𝑛, 𝑎𝑛𝑑 T the target value (ibid).
𝐶𝑝 = 𝑇𝑢− 𝑇𝑙
6𝜎 (3.6)
𝐶𝑝𝑘 = 𝑚𝑖𝑛 (𝑇𝑢− 𝜇
3𝜎,
𝜇− 𝑇𝑙
3𝜎 ) (3.7)
𝐶𝑝𝑚 = 𝑇𝑢− 𝑇𝑙
6√𝜎2+(𝜇−𝑇)2 (3.8)
20
𝐶𝑝𝑚𝑘 = (𝑚𝑖𝑛 (𝑇𝑢− 𝜇,𝜇− 𝑇𝑙)
3√𝜎2+ (𝜇−𝑇)2 ) (3.9)
For the process to be considered capable, the capability indexes 𝐶𝑝 and 𝐶𝑝𝑘 at least need to
reach a value of 1.33. Today, it is common to have even higher standards where the critical
value can be as high as 2.0 for manufacturing companies before the process can be considered
to be capable enough (Sörqvist & Höglund, 2007). The capability index 𝐶𝑝 , however, takes
into account only the spread in the process. Therefore, a supplementation to the study will be
necessary in terms of the index 𝐶𝑝𝑘, which will also take the centering of the process into
account, and then an analysis can be performed.
There are cases in which tolerance requirements may only have one important tolerance limit
that is relevant to the process capability. For these individual capability indexes are used which
can be seen in equation 3.10 and 3.11 (Bergman & Klefsjö, 2012).
𝐶𝑝𝑢 = 𝑇𝑢− µ
3𝜎 (3.10)
𝐶𝑝𝑙 = µ − 𝑇𝑙
3𝜎 (3.11)
An example of how the mean value and the upper and lower tolerance limit relates to each
other in a normal distribution can be seen in Figure 3.5.
Figure 3.5 Visual description over the mean with the tolerance width and its lower and
upper limits, inspired by Bergman & Klefsjö (2012)
Montgomery (2013) also mentions that 𝐶𝑝 and 𝐶𝑝𝑘 are only point estimates and they can
create some uncertainty in the measurement of the capability index. Therefore, a confidence
interval can be calculated for the various capability indices to provide a more accurate picture
of the process capability. The interval of the index 𝐶𝑝𝑘 is calculated by equation 3.12, where n
𝑇𝐿 µ 𝑇𝑢
21
is the number of observed values from the process and Z is taken from the table on the basis
of used significance level.
𝐶𝑝𝑘 [1 − 𝑍𝛼/2√1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] < 𝐶𝑝𝑘 < 𝐶𝑝𝑘 [1 + 𝑍𝛼/2√
1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] (3.12)
Before an accurate assessment can be made of a process capability, it needs to be stable (i.e.,
be in statistical control) (Shinde and Katikar, 2012). If the process is in statistical control, the
estimate of process capability cannot be used as a prediction for the future variability as it will
only be able to demonstrate the capability of only the studied period.
3.3.6 Measurement System Analysis
Control charts can help understand statistical control. However, it is important to keep in mind
that these control charts show the total variability for both the production process itself as well
as the measurement process included. Therefore, it is also important to examine whether the
measurement system is working as it should so that not wrong conclusions are drawn from the
process variability, but more important, about the capability. The variability that comes from
the measurement system will be especially important to consider when it comes to capability.
Otherwise, it will be incorrectly concluded that the manufacturing process is not capable only
because of the poor capability in the measurement system (Diering, Hamrol & Kujawińska,
2015). To improve product quality, a Measurement System Analysis is carried out. This type
of analysis can be performed in several different ways in order to assess the suitability of a
measurement system. Assessing the suitability of a system can be done using the Gauge R&R
study (Dalalah & Hani, 2015).
According to Montgomery (2013), a Measurement System Analysis has three purposes:
1. Examine how much of the total observed variability comes from the measurement or
tool;
2. Delimit the components that cause a variability in the measurement system; and
3. Assess whether the measurement or tool is capable.
In a Gauge R&R study two phenomena are investigated: (1) repeatability – the ability of the
measurement system to give the same result at repeated measurements when same operators,
methods, and equipment are used and (2) reproducibility – the variability that occurs when
different operators perform the same type of work (measuring) (ibid).
The basic idea of a Measurement System Analysis, according to Montgomery (2013), is further
defined in the equation 3.13: where y is the total observed measurement for the process, x is the
true value of the measurements performed on a single unit, and 𝜀 is the measurement error.
𝑦 = 𝑥 + 𝜀 (3.13)
22
Here the assumption is also made that x and 𝜀 are normally distributed and independent
variables with respective variances 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡2 and 𝜎𝐺𝑎𝑢𝑔𝑒
2 . Based on this, the obtained variance
for the total observed measurement is obtained (equation 3.14) (ibid).
𝜎𝑇𝑜𝑡𝑎𝑙 2 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2 + 𝜎𝐺𝑎𝑢𝑔𝑒2 (3.14)
In addition, precision-to-tolerance (P/T) – the estimation of gauge capability in relation to
tolerance width – is an important variable to measure (Montgomery, 2013). The standard
deviation of the measurement error is written as 𝜎𝐺𝑎𝑢𝑔𝑒. The calculation of (P/T) is performed
according to the following equation 3.15 (ibid).
P/T = 6𝜎𝐺𝑎𝑢𝑔𝑒
𝑈𝑆𝐿−𝐿𝑆𝐿 (3.15)
There are situations when only one specification limit exists for a product. In this case, the
process is called a one-sided tolerance, where P/T is defined by equation 3.16.
P/T = 3𝜎𝐺𝑎𝑢𝑔𝑒
|𝑈𝑆𝐿 𝑜𝑟 𝐿𝑆𝐿−𝑃𝑟𝑜𝑐𝑒𝑠𝑠 𝑀𝑒𝑎𝑛| (3.16)
For a P/T value to be seen as acceptable (i.e., to suggest that the measurement is sufficiently
capable), the criteria P/T ≤ 0.1 must be met. According to Al-Refaie and Bata (2010) the
measurement is considered unacceptable if this ratio exceeds 0.3. Before P/T can be calculated,
the value 𝜎𝐺𝑎𝑢𝑔𝑒2 must be accessed using equation 3.17, where the sum of the variance of the
repeatability and reproducibility gives the variance of the measurement error (Montgomery,
2013).
𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = 𝜎𝐺𝑎𝑢𝑔𝑒
2 = 𝜎𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦2 + 𝜎𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦
2 (3.17)
There are also cases when a traditional Measurement System Analysis cannot be performed.
These cases occur when or where the measurement of a specific part or parameter makes the
object either different, destroyed, lose its properties, or in another way does not provide the
same conditions when the same operator or another operator perform the measurement. These
scenarios are called Destructive Testing (Gorman & Bower, 2002).
A system or process with destructive sampling means that the variability between samples will
be part of the total variability and will therefore be included in the Measurement System
Analysis. The use of the so called Destructive Testing will therefore play an important role in
cases like wire pulls or ball shear when measurements of raw materials and finished products
are made. The validity of the measurements in scenarios with Destructive Testing is in question
23
because of its process not being capable to demonstrate repeatability and reproducibility in a
normal way (Ackermann, 1993).
One approach to solve this is called Nested Gauge R&R, a method that identifies sufficiently
identical batches of homogeneous parts. In these cases, it can be assumed that the various parts
being measured can be considered the same part. Nested Gauge R&R is necessary to use when
the identified batches are limited in size and have only the opportunity to test a couple of times
by each operator. This makes it possible to carry out a proper study to investigate the
performance of a measurement system when Destructive Testing is identified. (Gorman &
Bower, 2002). Ackermann (1993) mentions also that it is important to take into account the
additional source of variability that will be added to the measurement system (i.e., sample to
sample variability or material variability).
3.4 Sampling in pairs
According to Vännman (2002), the method sampling in pairs can be used when there is an
interest in comparing different expected values. The method itself investigates this by
establishing a confidence interval that is based on a t-distribution. The reason for the type of
distribution is because the standard deviation is not known. Instead it is estimated using the
equation 3.18.
𝑠 = √1
𝑛−1∑ (𝑧𝑖 − 𝑧)2𝑛
𝑖=1 (3.18)
The value 𝑧𝑖 is calculated by 𝑧𝑖 = 𝑦𝑖 − 𝑥𝑖 and can be the difference between two methods’
observed values. The value 𝑧 is in turn calculated by equation 3.19, where n is the number of
observed values for each method (ibid).
𝑧 = 1
𝑛∑ 𝑧𝑖
𝑛𝑖=1 (3.19)
To finally calculate the confidence interval, equation 3.20 is used if the methods are assumed
to have a significant difference in the average if the range for the confidence interval does not
contain 0 (ibid).
[𝑧 − 𝑡𝛼/2𝑠
√𝑛, 𝑧 + 𝑡𝛼/2
𝑠
√𝑛] (3.20)
For this method, the decision rule will be that with the security of 1-α, there are no significant
differences in the expected values for the different methods (ibid).
24
3.5 Process mapping
Today, businesses are interested in creating a better understanding of the manufacturing
processes to manage waste. Companies now see the importance of pursuing sustainable
production and energy efficiency with respect to profitability (Rybicka, Tiwari Del Campo &
Howarth, 2015). Process mapping is now used to gain a sustainability advantage (ibid). There
are various tools that can be used when process mapping is being performed. A flow chart, for
example, graphically presents the logical order of the steps that are involved in a process (ibid).
Bergman and Klefsjö (2012) found that there is great value to working in this systematical way
with the description of the current process. The principle is based on the identified inputs in the
form of activities and where in the organization they are performed. This collection of processes
creates a common overall picture of how different processes and activities are linked (ibid).
3.5.1 SIPOC chart
A SIPOC chart (Figure 3.7) shows a raw value flow of a process (Kumar and Kaushish, 2015).
The components of the SIPOC chart are described below (Montgomery, 2013):
1. Suppliers – The people who provide the materials, information, or other parts used in
the process.
2. Input – The information or materials provided to the process.
3. Process – The collection of the steps required to carry out the planned work.
4. Output – The product, service, or information being sent to the customer.
5. Customer – External customer or the next step in internal activities.
Figure 3.7 Example of a SIPOC chart, inspired by (Montgomery, 2012, p.53)
3.6 Pareto chart
A Pareto Chart is an improvement tool and also belongs to the so-called “seven- improvement
tools” or QC tools. The Pareto chart, which is named after the Italian statistician named Vilfredo
25
Pareto, serves as a decision support when the priority needs to be done for a variety of problems
(Bergman & Klefsjö, 2012).
As Rawson, et al. (2015) note, the graph below demonstrates that 80% of the problem (i.e.,
cost) is due to 20% of the causes.That is, a very small error accounts for most of the total error
or costs (Bergman and Klefsjö, 2012). The rule is also known as “the vital few and the useful
many” (ibid).
The Pareto chart is often used in the measure phase in the DMAIC model. However, the chart
can be used in the initial define phase to identify deficiencies or the category that is most
relevant to investigate first (Dreachslin & Lee, 2007).
26
4 Case Study through DMAIC This chapter presents the workflow for the case study according to the DMAIC method and
builds principally on Sörqvist and Höglund’s (2007) interpretation for each phase. The case
study will begin with the define phase where the reader will be introduced to the established
problem and the potential savings.
4.1 Define
In the introduction of the define phase, the problem was clarified with help of informants where
everyone agreed, by describing the current situation through different analysis of the collected
data with a problem definition. A SIPOC chart was also made to describe the TB process, but
a more detailed process map was made of a motor test procedure. Finally, a discussion over
possible potential savings within the TB process was done.
4.1.1 Problem definition
According to the Head of Measurement Technology, Technical Arrival & Control and
Capability, BRAB always strives to do better, for example, by reducing the motor returns.
Today, the company has five motor options for CA, its most common motor model, and the
number of CA motors manufactured of each model varies. The different motor types – CA 50,
CA 70, CA 100, CA 140, and CA 210 – indicate progressively larger motor sizes. The
difference between a CA 50 and CA 100 and CA 70 and a CA 140 are the number of cam rings;
the higher motor number will have two rings instead of one. Therefore, the difference between
a CA 70 and a CA 210 is three rings instead of one. The motors have the same radius, but
increases in the axial direction depends on the number of cam rings.
The interest was to investigate the most produced motor option. With the help of a production
engineer, data was found for the frequency of each CA motor option produced during the past
year. The result is shown in Figure 4.1 in the form of a Pareto chart.
27
Figure 4.1 Pareto chart over the annual throughput of approved motors for each model of the CA
motor
In the Pareto chart, shows that 873 CA 50 motors were produced each year, which is much
more than the other CA motors. The other motor options are nowhere near the annual frequency
and are all less than 400 motors each year. This difference in production runs led to an interest
in understanding the demand for the various motor options. Using the company’s business
system (Axapta), the demand for each motor model could be predicted for each week. From
this data, the conclusion could be made that there would be no difference in future forecasts.
Because the CA 50 motor has the highest throughput of CA motors, it could be predicted that
future demand would remain stable. As a two-week lead time is necessary for the CA 50 to
meet existing demand, it is very critical that the motors are delivered on time. Other motor
models have lead times between six and 16 weeks. Therefore, a CA 50 motor that does not pass
quality control inspection results in a larger negative impact than other CA models that do not
pass quality control inspection.
Furthermore, this study looked at how many returns occurred during the last three years (2013-
2015) to estimate the cost (time) of returns. The results are presented in Figure 4.2.
28
Figure 4.2 The number of returned motors in the past three years (2013-2015) for each CA model
These returns include the motors that were not approved despite repeated tests in the TB facility.
This in turn leads to overhead costs for the company when relevant personnel must be called in
to investigate the problem. The CA 50 motor has the highest amount of returns during the past
three years, with both highest numbers of errors in form of failed assembly and other causes.
Because it is unclear what these other causes are, a hypothesis was made that the numbers
mostly reflect failed motor tests.
As a result of this data, a Measurement System Analysis on the TB for the CA 50 motor and a
capability study on the CA 50 motor as a product were deemed to have the greatest potential
for improving the CA 50 motor. In addition, it was hypothesized that such an understanding
could then be applied to other models.
There are currently two TBs for the CA 50 motor. To get a more accurate picture, motors that
have been tested and approved were filtered out using historically documented motor tests
documented in Excel files. More motors were tested using TB 1 (4964 tests) than using TB 2
(2443 tests). TB 2 was more often used to test motor brakes (483 tests) than TB 1 (81 tests). As
the project had a limitation due to time, it was necessary to use just one of the TBs; TB 1 was
chosen since was used the most for motor tests.
In 2015, data on test alarms were collected as an established project was on hold after a supplier
wanted a certain number of the TBs for the CA motor. Meanwhile, this study wanted to examine
why there was an alarm. That is, the study wanted to identify the cause of the errors as either
due to component flaws, general motor flaws, or TB flaws. Figure 4.3 shows the data on the
test alarms for most for the CA 50 motors.
Figure 4.3 Pareto chart over the amount of occurred alarms within each alarm type
Of the 520 alarms that occurred during the project, the greatest number of alarms was for
cleanliness, temperature, and pressure (Figure 4.3). Both cleanliness and pressure have been
the most common test alarms, so they were investigated. Pressure was chosen over the
temperature as pressure depends on motor parts, whereas temperature only depends on the
29
motor oil. In addition, an ongoing project already existed for temperature and in particular the
control system that adjusts the cooling for the oil.
However, the motors do not directly engage the high pressure signal alarm. A control system
(PID) manages parameter high pressure around a certain value (XXX bar). The interest is
therefore to examine the precision and impact of the PID control system. After a discussion
with an engineer in the department of research and facility (R&D), it was determined that
external leaks and cleanliness reveals the quality of the motors. Therefore, high pressure,
external leaks and cleanliness are good choices for critical dimensions that should be
monitored. These parameters were selected because together they influence high and low
pressure as well as both external and internal leaks.
From the discussion above, the selected parameters that were selected to be examined by a
Measurement System Analysis are high pressure, external leaks, and cleanliness. These
parameters were then chosen to be made on TB 1 and a capability study of the CA 50 motor.
Because cleanliness can be automatically improved (i.e., cleaned after a motor is tested), the
case also involved the so-called Destructive Testing for the parameter cleanliness, which means
that the alternative method of Measurement System Analysis called Nested Gauge R&R was
needed. In both cases, the situation made it possible to use two operators to examine both the
repeatability and reproducibility of the process.
Therefore, this case capability study performed experiments on TB 1 designed specifically for
the CA motor. The following critical parameters were used.
1. High pressure P3
Measured in the unit bar and has a two-sided tolerance limit of XXX ± X bar
2. External leaks QY
Measured in the unit liters per minute and has a one-sided upper tolerance limit of Xl/min.
3. Cleanliness (by ISO 4406: 4μm, 6μm respectively 14μm)
Measured in number of particles of each particle size and showed as the corresponding
contamination class according to ISO 4406. One-sided upper tolerances used within the
company for particle size is Mx (X), Mx (X), and Mx (X).
The capability study did not consider other parameters such as splines2, low leaks, number of
revolutions, or flow despite their importance to the customer since this part of the motor did not
depend on the TB.
4.1.2 Potential savings
There was also an interest in reviewing a possible savings potential for the project. After
reflecting on these potential savings, a relevant parameter was needed that related to the TB
facility.
2 Is the gear-like portion of a motor cylinder in which the drive shaft is mounted to drive the motor
30
Today, occasionally unapproved motors pass inspection according to the results from the TB
despite various retests of the motors. That is, the returned motors required dismantling and
examination. After some discussions and interviews with operators about production techniques
for the TB facility, they felt it impossible to give specific estimate for how long it typically
takes to resolve a motor problem as the problems and their resolutions varied. Instead, the focus
went to understanding how much these returns cost the company.
After an analysis of the collected data for the history of alarms for the CA motor, exclusion was
carried out. This exclusion included alarms that remained over a weekend and other types of
alarms that did not give a fair picture for the total downtime. This active downtime is the time
it takes from the alarm being detected and received by an operator, where the downtime lasted
anywhere from a few seconds to a few minutes. The total downtime was then calculated for the
total number of hours and applied to TB 1 for the year 2015, where the total downtime hours
came to 412. In addition to downtime, costs accrue because motors need to be tested several
times (costs in form of time and work force) and, above all, because it is expensive to
disassemble, fix, and re-install motors.
The head of the finance department helped identify other costs related to the TB process. These
costs included a variable costs affected by the production volumes as well as fixed costs such
as the cost of operating a TB. The variable costs are essentially salaries to operators and cost of
electricity. Of the fixed cost, 504 Swedish crowns (SEK) are depreciation and capital costs of
the machines, about 65 SEK are costs of risk due to re-works and scrap, and the rest of the
remaining cost (544 SEK) includes overhead costs, for example, rent and support functions
such as technology and managers’ salaries.
The variable cost can thus be seen as direct savings where the cost is 672 SEK each hour,
equivalent to 1113 SEK each hour for the fixed cost. This fixed cost, expressed by the head of
the finance department, are too high so these costs were considered as a potential improvement
area in terms of the TB. These potential savings are only based on the downtime. A much higher
savings potential could be expected if the number of motors being repaired is reduced.
However, it is difficult to put a specific number on this.
If the downtimes can be reduced, the company would be more effective, more profitable. A
total annual cost was then calculated from the information provided and is presented in Table
4.1 below.
Table 4.1 Total cost of the annual active downtime for TB 1
Total downtime for
TB 1
Variable cost
Fixed cost
Total annual cost
412 h 672 SEK/h 1113 SEK/h 735 720 SEK
This is, however, a subjective estimation of what the unidentified cost is for the downtime
between test alarms. The total cost is based on an expected value of zero (i.e., the saving
potential if all reported downtime is eliminated). In practice, this is not possible, but if, for
example, a reduction in downtime of 20% could be obtained, it would give annual savings of
147 000 SEK.
31
The calculated total annual cost can still be used to understand that some improvement
opportunities exist for the TB facility based on these data. If the total time goes down
significantly, the operators could create more value for the company. In addition, if fewer
motors are returned, upstream production would improve as less “re-work” would need to be
done.
The number of satisfied customers will increase when the number of CA motor missing delivery
because of failed tolerances on the TB decreases. This improvement can be interpreted in the
Kano model3 as the implied needs are met, which itself is expected by the customer. Despite
this, it will still be as important as it helps to create long-term relationships with its customers.
Should the number of motor returns or in any case the number of test alarms be reduced that
usually lead to motor tests being cancelled, operators and other personnel involved in the
reworking would be able to focus on things that bring value to the company and therefore the
company will be more productive in terms of work. In terms of the sustainability, a reduced
number of motor returns will increase the sustainable development from an environmental
perspective as, for example, less returned transport from the customer needs to be done.
4.1.3 SIPOC chart
For a better overall view of the process stream, a SIPOC chart was made for the TB facility
through observations. This was considered relevant as the chart shows the various flows
included in the process and how they are interconnected in the form of suppliers, inputs,
process, outputs, and customers. The established SIPOC chart can be seen in Figure 4.6 below.
Figure 4.4 SIPOC chart showing the process stream for the TB facility
3 A tool to help pinpoint customer needs (Berman & Klefsjö, 2012).
32
4.1.4 Process map
In addition to the SIPOC chart, a process mapping of the actual motor test performance was
also conduced through observations. A process map helps identify which process step or steps
most likely contribute to the variability in the process. Figure 4.7 below presents the process
map for the TB with its detailed approach steps identified by observations. The operators
provided comments and suggestions on the first draft.
Figure 4.5 Process map that describes the approach for the TB process of motors at the TB facility
The first step in the process is receiving the motor from the assembly through a roller conveyor
between the stations. After receiving the motor, the operator uses a set of protocols (e.g.,
checking the cam rings and the motor signature) to inspect the motor. Next, the operator mounts
the motor on the TB. Once these steps are completed, the operator uses one of the available
overhead cranes and lifts the motor from the runway to a lifting table which is placed outside
the door of the TB. When it is time to test the motor, it is lifted from the lift table into the test
cell where the mounting block, hoses, and speed sensor are connected to the motor. Here the
only option for the operator is the order in which the things are connected to the motor. After
this is done, the operator checks doors and ceiling to make sure that these are closed. Then the
operator keys the receipt card and follows a checklist. When these steps are done, the motor
test will be started. It is important for the operator to use the right receipt card for the specific
motor. When the motor test is finished, the test protocol is printed and put with the other week’s
test protocols in a special compartment inside the control room for the production technician to
collect at the end of the week. The next step in the procedure will be to open doors and roof
again, and after that oil tubes are disconnected and replaced with drainage tubes. When the
tested motor is fully drained, it is lifted to the table outside the TB again where a detailed
assembly takes place in form of plugs, lifting ears, and blocks. Finally, the motor is lifted to the
load carrier before being transported to the painting facility.
33
As can be seen, the operators have few options when it comes to preparations and the actual
motor tests as the procedure is strictly controlled routines. Thus, there exists very small room
for differences in individual handling and expectations.
4.2 Measure
In the measure phase, the information is presented in the form of data that are needed and thus
collected in order to implement a Measurement System Analysis, establish control charts, and
conduct a capability study for the CA 50 motor.
4.2.1 Data collection
To perform the Measurement System Analysis, primary data were collected in the form of
variable data for the three parameters high pressure, external leaks, and cleanliness for TB 1.
These data were collected under normal tests at the TB where data for the three parameters high
pressure, external leaks, and cleanliness were collected. This primary collection was not only
important to see how the test procedure was done, but also important to confirm the operators
who performed the tests, making it possible to replicate the tests. In addition, historical data
were collected that identified which motors had been tested by which operator (the name of the
operator was written on the test protocol). These data also ensured the validity of the project to
make it possible to apply to the present day. Moreover, new data were needed for the
Measurement System Analysis. The two main parameters based on BRAB promised tolerances
to customers deemed by the company to be external leaks and cleanliness, where the parameter
high pressure was more of an internal interest. The experiment in the form of motor tests of a
reference motor was necessary to do when the time was considered to be too demanding to
execute repeated tests for motors under normal production. The tests of the parameters high
pressure and external leaks were performed on a stop day (i.e., when production was closed)
so that useful data could be obtained, while the primary data for the motor tests for the parameter
cleanliness were planned together with the production planner on the basis of needs and order
priority. The collected data of the motor tests are collected in Appendix A.
A problem with the data collection concerning to the parameter cleanliness is when a motor test
is repeated. The measurement of this parameter was therefore done using Destructive Testing.
This means that the motor normally becomes cleaner after each test drive. A more detailed
description of this can be seen in Figure B.1 in Appendix B.
The solution for this was to group identical CA 50 motors and assume the same motor was
tested twice. For the other two parameters, high pressure and external leaks, a single reference
motor was tested by two different operators because these two measures are not expected to be
affected by repeated tests and the tests themselves do not change the motors performance
regarding the two parameters high pressure and external leaks. This approach was chosen when
the interest was to investigate TB and its measurement system and to save time as a motor test
usually takes at least 30 minutes excluding the changeover time and because previous controls
for this had been made at the company. The data for the parameter cleanliness could therefore
be considered concrete.
Because replicates were performed with the same reference motor, it was possible to shorten
the overall cycle time for a motor test including changeover time from 40-44 minutes to 22
minutes. Data collection was then based on BRAB’s own standard of at least a total of 25
34
measurements, which was considered a good estimate. The total number of measurements came
as close to that number as possible. Because the operators worked two shifts, all measurements
needed to be made during one shift for the two operators to perform the measurements with as
little variability from external factors as possible. The measurements were performed during a
stop day, which meant that operators could fully devote their time to the motor tests.
The primary data collection methodology for the parameters high pressure and external leaks
was designed so that the two operators had to measure the same reference motor as many times
as the time limit allowed to complete as many real replicates as possible to uncover all the
contributions of variability that existed. The number of measurements amounted to a total of
20 (ten for each operator) and can be seen in Appendix A.
For the parameter cleanliness with its need for a Destructive Testing approach, the assumption
was that no motor could be replicated. This was solved by establishing a Nested Gauge R&R
measurement plan where homogeneous motors were planned in conjunction with a production
planner so that they could be measured in the same batch. These batches were tested in pairs
during the same days between dates 2016-02-10 to 2016-02-24 and can be seen in Table A.3 in
Appendix A. This way, a planning based on the order of motors could be made despite the
limited view of opening orders within two days. A completion of motor pairs was also made to
give a good estimate of the weighted standard deviation for the parameter cleanliness.
For the capability study of the CA 50 motor, a collection of 50 data samples for each of the
three parameters was made through the last executed motor tests in form of test protocols
between week 2 (2016-01-13) and week 5 (2016-02-05). This was chosen to evaluate how the
normal production process looked in the company today for the CA 50 motor. It was important
to study the capability on the normal production as the included products in the study are the
products that customers actually receive from the company.
The collected data from the interviews and the observations was used to establish the SIPOC
chart (Figure 4.4) and the Process map (Figure 4.5) with purposes to create a better
understanding of the TB process in the earlier define phase. The results from the unstructured
interviews can be seen in Appendix B.
4.3 Analyze
In the analysis phase, the collected data were analyzed for the three parameters high pressure,
external leaks, and cleanliness with its three particle sizes. The results from the observations
and interviews conducted at TB process are also described. The analysis emphasizes the
measurement system and the results of this system led to a more reliable measuring station, i.e.,
the TB facility for the CA 50 motor. A capability study was also conducted with the aim to
investigate what the capability for the CA 50 motor looks like. In this way, safety can be created
in the measured values for internal use.
4.3.1 Measurement System Analysis
The Measurement System Analysis was performed to investigate whether the measurement
system can be considered capable in the tests for three parameters high pressure, external leaks,
and cleanliness. It was relevant to split the analysis into three separate analyses as the three
35
parameters are different and therefore different approaches were needed to calculate the
repeatability and reproducibility for the Gauge R&R study.
The first parameter, high pressure, is an internal customer interest rather than an external
customer interest when interest lies with the company itself to test all the motors on the pressure
level XXX bar. This target value is produced from the maximum pressure of 350 bar, which is
the maximum pressure the motor can run on where thrust bearings inside the motors are able to
resist forces that occur inside the motor within this maximum pressure. The measurement
system for the internal parameter is then controlled entirely by the PID control system that is
designed for the TB facility. The PID control system aims to fix the value around XXX bar.
The results of the Measurement System Analysis are presented in Table 4.1, and the full version
of the Measurement System Analysis can be seen in Appendix B.
Table 4.1 Summary of Measurement System Analysis
Parameter ��𝑹𝒆𝒑𝒆𝒕 ��𝑹𝒆𝒑𝒓𝒐𝒅 ��𝑮𝒂𝒖𝒈𝒆 P/T (%)
𝝈𝑷𝟐 𝝆𝑷(%) 𝝆𝑴(%) Confidence
interval
High
Pressure
0,2173 0,0177 0,2180 0,3270 0,2639 0,8474 0,1526 [-0.29,
0,25]
External
leaks
0,1017 0,0177 0,1032 0,0590 0,2533 0,9597 0,0403 [-0.12,
0,08]
Cleanliness
4μm
0,8503 0,1330 0,8606 0,6297 0,8205 0,5256 0,4744 [-0.75,
1.05]
Cleanliness
6μm
1,5492 0,1773 1,5593 0,9746 0,1400 0,0545 0,9455 [-1.08,
0.68]
Cleanliness
14μm
2,0856 0,0443 2,0861 0,8597 0,3028 0,0651 0,9349 [-0.88,
0.98]
In summary, it was possible to determine, based on the results of variability contributions from
various operators (i.e., σReproducibility) and the confidence interval with the method sampling
in pairs, that the operators had a very low impact on the result of the TB for all three parameters.
This means that the analysis did not detect any significant difference between different
operators conducting the motor tests.
The P/T ratio for the parameter high pressure was 32.7%. This result indicates that the precision
of the measurement system, which in this case was the PID control system, had poor accuracy
in terms of the tolerance measurement error as it takes up 32.7% of the space within the
tolerance width. However, this error was calculated with Six Sigma’s accuracy of 99.975% (k
= 6). If the more general k = 5 (15 with a corresponding accuracy of 99%) would be used, then
P/T ratio would be 28.1%, which demonstrates an acceptable measurement system. However,
it can be considered relevant to review opportunities for improvement of the control system
when the quality manager for Engineering and Development (E&D) proved that this system
does not always seem to work as it should, as it has difficulty maintaining a stable pressure
during certain motor tests, which in some cases generated to test alarm. This also explained
much of the considerably greater variability in terms of repeatability.
36
The parameter external leaks, however, showed a very good P/T ratio of 5.9% (Table 4.1). At
first glance, this would result in a situation so good there is no room for improvement. After a
deeper analysis of this value, it turned out that the motors were not even close to today’s
tolerance limit of six liters per minute but had a much lower mean. According to Research and
Development (R&D), the current tolerance had existed since the 1990s when a change was
made to the pistons that gave a new tracking channel with the help of Finite Element Method
(FEM) calculations to reduce external leaks. This result was considered large enough to pursue
further opportunities for improvement on this parameter in terms of the tolerance. Furthermore,
96% of the total variability did come from the reference motor used for the analysis; that is,
only 4% came from the measurement system itself.
For the parameter cleanliness, the measurement system showed an uncertainty that for all three
particle sizes when estimating the ISO classifications (Table 4.1). With P/T ratios of 63%,
97.5%, and 86% tells us that most of the variability that occurs within the tolerance limits was
in the form of the measurement error (uncertainty). The percentage of the total variability that
did come from the measurement system, which in this case was the existing particle counter,
showed that 47.7% had a particle size of 4μm, 94.6% had a particle size of 6μm, and 93.5% had
a particle size of 14μm. This proves that the method of measuring the number of particles in the
motors had an uncertainty. There could be several reason for this problem. One reason could
be the particle counter’s ability to transform particle numbers to the corresponding ISO classes.
Another reason could be that the light that the particle counter uses to identify particles only
highlights particles from one side, resulting in an inaccurate measure of the particle’s diameter.
As a result of this, future investigations should consider highlighting the particles from only
one side could be a possible cause of the problem. That is, particles of the smallest size (4μm)
were from the motors considered as relevant when the size of particles was ten times smaller
than a human eye is able to see. Therefore, the particles can exist in all possible areas in the
motors and thus risk shaking loose both during operation and during testing.
The standard deviation of the total measurement error, σGauge, for the parameter cleanliness
made it essential to evaluate the variability of the particle counter. Therefore, a comparison was
chosen made between the results in Table 4.2 and with the corresponding standard deviation of
the measurement error of the 20 measured values of the reference motor. These values can be
seen in Table A.2 in Appendix A, where the assumption was made that the reference motor had
been worn out by its hundreds of runs and was therefore considered stable in terms of
cleanliness. The result can be seen in Table 4.2 below.
Table 4.2 Summary of the total standard deviations for the measurement error for cleanliness with
the use of reference motor
Parameter ��𝑮𝒂𝒖𝒈𝒆
Cleanliness 4μm 1,0990
Cleanliness 6μm 1,3485
Cleanliness 14μm 1,9808
The total standard deviation of the measurement error for these values includes both the
measurement system’s repeatability and reproducibility, but since the reproducibility showed
very low results (4.1), the conclusion was that the existing variability from measurement error
came from the particle counter itself. One important thing to note here is that this standard
deviation demonstrates the worst case scenario (i.e., the highest possible variability). This
37
scenario existed because we were dealing with Destructive Testing and thus the product
variability and the measurement system variability were overlaid. Another impact of this result
is that the particle counter reports its results in integers in the form of ISO classes, so it
disregards information that could make the measurement system more precise.
4.3.2 Control charts
To conduct a proper capability study on the three parameters high pressure, external leaks and
cleanliness, it was relevant to control how the data for the various parameters could be
considered normally distributed or not on the basis of the collected data in Appendix A. The
normal distribution was determined using the analysis in the software Minitab, where both a
graphic and a normal plot analysis with associated p-values were used. For each parameter
control charts were established for individual measurements and the moving range (i.e., I-MR
charts).
The parameter high pressure could be assumed to have normally distributed data and was
considered a stable process. For the parameter external leaks, however, a pattern was detected
that showed a more unstable process. It should be investigated whether anything made the
measurement of the parameter external leaks became wobbly during the weeks when the data
were collected or if it was merely a coincidence. It was also necessary to do a transformation
of the data for parameter external leaks for it to show a normal distribution. There was no reason
to understand why the data were not normally distributed for the parameter external leaks as
only 50 values were used. Only 50 values were used because this number would satisfy the
theory that at least 30 measurements are needed as well as fit the required time perspective.
For the cleanliness parameter, all three particle sizes in a non-stable process were considered.
For all three particle sizes, several points adopted the same value as the parameter itself is
measured by only one decimal point. The particle size 4μm could be considered normally
distributed according to the normal plot. The process could be seen as a stable process when
the chance might have placed the numerous measurements (ISO classes) on the same side of
the centerline and several subsequent motors received the same ISO rating as the resolution of
the specific number of particles will be different between motors. This result was not surprising
when the parameter cleanliness is presented for a certain ISO class with one decimal point
where the classes often adopt the same size pretty. Therefore, this will contribute to a worse
resolution of the data; it actually should belong to a more Poisson distribution.
An alarm occurred outside the 3σ for particle size 4µm, which indicated that the process is not
stable. The same situation existed for the other two particle sizes, 6μm and 14μm. At regular
intervals, the motors were given an ISO grade of zero for 14μm particles, which meant in
practice that motor tests either contained none of the particles of that size or that the particle
counter was not able to estimate the number. This led to the MR chart giving an alarm for two
measurement points. After an analysis of the last year of motor tests and talking to production
staff, it was found that this pattern did not occur regularly but appeared occasionally at different
intervals. Control charts for total cleanliness were also established as a complement to the three
separate particle sizes, which even showed a non-stable process.
Because some of the parameters proved to be stable but in a few cases there were no alarms, it
was decided to leave them for further analysis. This decision was taken because occasional false
alarms can reasonably be expected and that the collected data showed only a certain time of the
38
whole process. The analysis assumed that the collected data were normally distributed, so the
study could be said to have good reliability.
4.3.3 Capability study
The capability study was performed to investigate how capable the process is for the three
parameters high pressure, external leaks, and cleanliness. The investigated capability was
calculated with the help from the capability indexes 𝐶𝑝 and 𝐶𝑝𝑘 for the three parameters, where
the capability index for one-sided tolerance limit has been used for the parameters external
leaks and cleanliness.
The results of the individual studies regarding variability and capability have been compiled in
Table 4.3. It was of interest to identify the capability in terms of the total variability, in this case
the process capability, and to identify how much better the capability would be if the variability
of the measurement system was pulled from the results (i.e., when only the product variability
was used).
Table 4.3 Summary of Capability study
Parameter ��𝒑𝒓𝒐𝒅𝒖𝒄𝒕 𝑪𝒑 𝑪𝒑𝒌 𝑪𝒐𝒏𝒇𝒊𝒅𝒆𝒏𝒄𝒆 interval
��𝑻𝒐𝒕𝒂𝒍 𝑪𝒑 𝑪𝒑𝒌 𝑪𝒐𝒏𝒇𝒊𝒅𝒆𝒏𝒄𝒆 interval
High
Pressure
0.5137 1.3
0
1.3
2
[1.04, 1.60] 0.558
1
1.1
9
1.1
7
[0.92, 1.42]
External
leaks
0.5033 - 3.4
7
[2.78, 4.16] 0.513
7
- 3.0
1
[2.40, 3.61]
Cleanlines
s 4μm
0.9059 - 1.5
1
[1.20, 1.82] 1.249
5
- 1.0
9
[0.85, 1.33]
Cleanlines
s 6μm
0.3742 - 4.2
8
[3.42, 5.13] 1.603
6
- 1.0
0
[0.78, 1.22]
Cleanlines
s 14μm
0.5503 - 4.4
1
[3.53, 5.29] 2.157
5
- 1.1
3
[0.88, 1.37]
In summary, it was possible to determine that the testing process delivered the products with
different levels of capability. The parameter high pressure, which was a production parameter
and thus was not dependent on the motors as much, showed on a 𝐶𝑝𝑘value of 1.17 for the control
system, which does not meet the requirement of 1.33. This value along with the lower limit of
the confidence interval of 0.92 indicated that the process does not keep the value of the pressure
close enough to the target value of XXX bar, but had too much variability and poor centering.
This is mostly due to the PID control system’s ability to fix the pressure during motor tests and
the PID cannot handle this task well enough. This variability can be considered large enough to
look for ways to improve this control. This view was supported by the production and quality
manager at the R&D department who revealed that several high pressure alarms have occurred
during motor tests. In contrast, the control system showed separately, with the variability of
measurement uncertainty (error) excluded in the capability index, a result that is very close to
the limit 1.33.
In contrast, the parameter external leaks showed good values of the capability index of 3.47
with regards to the product variability after the variability of the measurement error had been
39
removed and 3.01 with the variability of measurement error included. Both these values with
associated interval are significantly greater than the requirement of 1.33. The reason that the
capability indexes for external leaks were so good was because the leakage of the motors is so
far from the tolerance limit of six liters per minute. According to the R&D department, the
motor pistons have been reworked to reduce the leakage in the motors; however, the TB process
continued with the same tolerance because of the delay in this change. This means that the
company today usually delivers better products than what is specified in their literature, but this
also means the company might risk sending out a motor with high leakage.
The capability index for parameter cleanliness showed on a non-capable performance when
none of the particle sizes met the requirement of at least 1.33. These results reflect the total
variability in the process. When the variability of the measurement error is deducted from the
total variability so that each capability index was calculated based only on product variability,
the result showed a high capability for the parameter cleanliness. These results show that there
exist opportunities to improve the parameter cleanliness. After brainstorming with the Quality
Manager at the R&D department, one conclusion was that dirt in the motors could be derived
from the details of processing and during motor tests. This creates not only an opportunity to
improve the integrity of the TB process but also the motors by improving the TB processes.
4.3.4 Observations and interviews
The data collection made at the beginning of the project from own observations and
unstructured interviews with employees working at the TB process was used to establish the
SIPOC chart (Figure 4.4) and Process map (Figure 4.5) in the previous define phase. Based on
the established figures along with the semi-structured interviews (Appendix F) it was concluded
that the operators at the TB process have good conditions to carry out their work without
influence from their operations. This was further strengthened from the measurement system
analysis in Table 4.1, where the reproducibility is significantly lower than the repeatability for
all three parameters.
The result of the interviews also indicated that the instructions are sufficiently standardized and
easy to follow as the operators themselves experienced very low handling errors during their
operations, and that they are able to rotate in shifts between being working at the assemby
process and the TB process. However, operators felt that the greatest risk of making mistakes
in TB lies in the two TB facilities for the motor type CA 50 becauso of their mirrored inside
appearance.
During the interviews, it emerged that the measurement values for the tested motors can be
wrong if the testing protocols accidentally become incorrect. The majority of the risks the
operators considered be the present assembly process where errors can lead to e.g. motor or
breakdowns which takes time to identify the reason for.
As the operator impact was so low, the need for improvement was considered not as critical as
those identified during the analysis of the results from the completed Measurement System
analysis and capability Study. Therefore, it was decided that not take it further to the coming
improve phase.
40
4.4 Improve
In the Improve phase, improvements and recommendations are proposed for the three
parameters high pressure, external leaks, and cleanliness that have been developed during the
Analyze phase. Some of the proposals have not been able to test the thesis of various reasons,
because they require expertise in programming and various rebuilds or installation of TB
equipment.
4.4.1 Improvement of PID control system
The parameter high pressure is an internal production parameter where the company wants to
test other product parameters that are important to customers and to motor function. The control
system is designed and programmed to control and regulate the parameter high pressure in the
motors during the TB where its purpose is to maintain the real value as close to the target value
XXX bar as possible, so motors receive minimal wear and tear under the motor tests. After
discussions with a production engineer at the company and quality engineers in the R&D
department, it was found that the current PID control system does not need to be optimally
designed for the TB. This created room for improving the PID control to not fail in meeting its
capability requirement for a process at 1.33 and minimizing the risk for “false alarms”. A better
developed PID control system will therefore lead to eliminating the test alarms that occur
annually due to pressure drops outside one of the tolerance limits. This will require expert
knowledge of programming and algorithms. If an assumption is made that the 17 failed motors
did not meet the target value for high pressure after 15 minutes (i.e., in the middle of each motor
test), the improvement of the PID control system would correspond to a cost savings of 7586
SEK each year.
4.4.2 Adjusted tolerance limit
From the measurement systems analysis and the capability study in the previous Analyze phase,
it was concluded that the company is delivering better CA 50 motors than what is required
according to the test protocols. Therefore, one improvement could be to adjust the current
tolerance limit of six liters per minute for the parameter external leaks to 3.85 liters per minute.
This tolerance was calculated considering both the equation for P/T ratio and capability index
Cpk. With this new tolerance limit for the parameter external leaks, the measurement system will
still satisfy the highest requirement for an excellent measurement system of ≤ 10% and a
capability index of 1.61, which in turn meets the requirement of 1.33. This action is also
important with respect to working with continuous improvements to eliminate the risk of
sending motors with bad leakage to customers.
Another proposal would be that the company tells their customers via the website and the
product brochure that they have changed their promised tolerance claim, improving the
desirability of their products. That is, the company should actively promote this change to
improve quality of the CA 50 motor and eventually this will improve its market position. This
could be generalized for all the CA motor sizes (i.e., the Compact series).
4.4.3 New intern decision rules
ISO 4406 is based on an ideal measurement system. Given that the motors are just as dirty in
the current situation and given that there is an uncertainty in the measurements of the parameter
41
cleanliness with the high values on ��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 according to Table 4.1, which means that the
measurement system is not working well enough for its current task, the recommendation is to
improve the measurement system through the introduction of new internal tolerances for each
particle size. The following steps are made to make sure that the process reaches the tolerance
requirements of the ISO 4406 – Mx (X), Mx (X), and Mx (X) for particle size 4μm, 6μm, 14μm,
respectively. The improvement proposal can be seen as a refinement of the system by evaluating
the parameter cleanliness with new internal tolerances. In this way, quality assurance can be
made in terms of cleanliness of the motors so the company ensures that no motors would pass
that actually should have been stopped under the motor test and that no motors being sent to
customers do not fulfill the ISO 4406.
To calculate where the internal decision rules should be, one-side tolerance limit was used as
USL – K��𝐺𝑎𝑢𝑔𝑒 , where ��𝐺𝑎𝑢𝑔𝑒 is based on the previous results (Table 4.1). These values are
the highest that the standard deviation for the occurred measurement error can adopt. Since the
replicates were faked by matching homogenous motors with each other, an entirely separate
estimation could not be done for ��𝐺𝑎𝑢𝑔𝑒 and ��𝑃𝑟𝑜𝑑𝑢𝑐𝑡 because they were superimposed with
each other. This means that the sizes on ��𝐺𝑎𝑢𝑔𝑒 will contain both product variability and the
variability from the repeatability, i.e., the inaccuracy from the particle counter (measurement
system). More realistically feasible internal tolerances were identified using two assumptions.
The first assumption was that a smaller value of K is sufficient instead of K = 3, which the
original equation used when it otherwise takes the worst case scenarios in to account, i.e., the
points all the way to the tails of a normal distribution curve. Here it was a struggle between
theory and practice. Based on this and because so few motors land at the edge of the distribution,
it was not relevant to such a large constant but the value of K was instead chosen to vary for
the three particle sizes so that the respective tolerance was lowered with one ISO class. The
second assumption made was that half of the values of the adopted ��𝐺𝑎𝑢𝑔𝑒 came from the result
from the Nested Gauge R&R approach. With these assumptions, the calculated internal
tolerances could be selected for particle size 4μm, 6μm, and 14μm, which can be seen in
equations 4.1, 4.2, and 4.3, respectively:
𝑀𝑥(𝑋) − 2,32396 ×0.8606
2= 17 (4.1)
𝑀𝑥(𝑋) − 1,28263 ×1.5593
2= 15 (4.2)
𝑀𝑥(𝑋) − 0,95873 ×2.0861
2= 12 (4.3)
Since the ISO classes are identified as only whole numbers, it was natural to round to the nearest
classification. The recommendations from this calculation for the three particle sizes are Mx
(17), Mx (15), and Mx (12). This will further mean that six CA 50 motors should have had an
alarm between the 2015-03-03 and 2016-02-25. In total, 1185 CA 50 motors were tested
between 2015-01-13 and 2016-02-15 for 2990 CA motors irrespective of size. These numbers
include failed/stopped motors at the two TBs. From these motors, 158 CA motors were stopped
because the parameter cleanliness exceeded any of the tolerances for the various particle sizes.
Of these 158 alarms, the CA 50 motors were responsible for 14 failures.
42
Since the time of the failures varied, it was necessary to assume that the motors had stopped in
the middle of the sample (i.e., after about 15 minutes). With the new internal tolerances, this
meant an additional stop of six CA 50 motors, which will give 90 minutes extra motor testing
each year. In the current situation, the motor testing for the Compact series has a variable cost
of 754 SEK each hour. This cost includes things like having staff on site, material costs, raw
material, and price of electricity. According to the economy department, the marginal cost of
the motor testing is negligible for the current situation, which means that these 90 minutes extra
of testing time of the CA 50 motor will not entail any additional costs for the company. Since
all CA motors use the same type of particle counter, it would also mean that 94 additional CA
motors will be stopped, regardless of size. This result corresponds to 1410 minutes of extra
motor testing (23.5 hours) each year, which also means a negligible marginal cost. This is with
the assumption that a retest of the motors would mean at least one lower ISO class. If the
occupancy rate of the motor testing for the Compact series (CA motors) goes up due to
increased orders of CA motors, the assumption was made that one-third would not be possible
to be put into general production, so those extra 90 minutes for the CA 50 motor and 1410
minutes for CA motors correspond to costs at 377 SEK and 5906 SEK, respectively (T.
Fahlberg, Finance Department, May 16, 2016).
An alternative to these internal tolerances would be to build an extension system of the testing
time to ensure that the motor is not stopped at the critical time of the final assembly without
prolonging the test time, for example, a maximum of 15 minutes. This means that the motor
has a chance to be flushed and thus possibly meet the requirements of the parameter cleanliness.
This would be relevant when even a single particle of any size can stop a motor.
With these additional tolerances for the assessment of particle numbers with the associated ISO
classes and thus the motor unit, the TB process becomes a more rigorous testing process that
will reduce any future complaints from customers. This can also provide a market advantage if
the company announces that the motors generally are to be delivered cleaner than the current
tolerances used for ISO 4406 in the TB process. This improvement proposal will therefore be
a quality investment rather than productivity investment. One question that the company must
ask is whether it wants to invest in productivity in the first place or in the collateral quality of
the tests. Here the company needs to weigh the benefits with the costs.
4.4.4 Production improvements
To achieve a high capability for the TB process in terms of the parameter cleanliness it would
be appropriate to improve and to test the washing process, the design of the motors, and/or the
assembly process to create better conditions for the motors before they being tested. For the TB
process, a change of the filter system could reduce particle levels even further. This is something
the company should test in the long term. Work in these areas should possibly give a lower
mean and a lower standard deviation of the process, and thus make it possible for the capability
index, 𝐶𝑝𝑘, to reach the requirement of 1.33. In this project, there was no time to test these
identified opportunities for improvement in practice when more comprehensive planning is
required concerning the normal production; therefore, more studies are needed (see 6.3).
4.4.5 Prioritization of improvement proposals
Based on the measures and recommendations, changes were prioritized in terms of how easy
they would be to perform, their costs, and their results (Table 4.4). The simplest action would
43
be to update the tolerance limit and specification for the CA 50 motor regarding the parameter
external leaks. This should be given a high priority, as it gives a very positive result.
If the testing process evaluates the cleanliness parameter for the CA motor with new tolerances,
it will improve quality as the ISO 4406 improves security and eliminates the risk that some
motors would not be stopped even if they had too high ISO class. The improvement of the
current PID control system would ensure that the parameter high pressure meets the
requirement of 1.33 as well as eliminates TB alarms due to the parameter being outside the
tolerances, reducing costs for the company.
A more comprehensive recommendation is to carry out improvement work concerning the
conditions for the cleanliness in the motors before the motor tests are performed. This could
mean improvements in the washing process, construction, and/or the assembly process.
Table 4.4 Prioritization of improvement proposals
Priority Action
1 Update tolerance limit for parameter external leaks, base for X.XX l/min
2 Establish new internal tolerances for parameter cleanliness: Mx(X) Mx(X) Mx(X)
3 Improve the design of the PID control system for parameter high pressure
4
Improvement work of the conditions for motors cleanliness before motor tests,
examples: Filter system, washing facility, construction, and/or assembly process
4.5 Control
This control phase deals with how the operation can be standardized by establishing routines
and instructions for staff as well as pointing out the importance of disseminating these
practices to the other TB processes.
Due to the time limit, this project had a detailed follow-up of improvement proposals and
recommendations of BRAB could not be implemented. Thus, it will be up to BRAB to continue
with the implementation of those highlighted improvements during the Improve phase. The
company should to continue to work with Statistical Process Control, preferably with similar
control chart in this case study, to in the long term obtain a stable TB process for the studied
parameters. This can be done by a weekly monitoring of the test protocol as a Shewhart chart
for individual measures. Furthermore, it is possible to discuss how and when future
Measurement System Analysis should be carried out. The performed Measurement System
Analysis can be used as a template for future Measurement System Analysis on the other TBs
within the company.
Here, standardization can be designed for the cases when a Measurement System Analysis for
a TB is going to be made. This could be done several times each year when service of TB is
made, when new TB procedures developed, or when new staff are being trained. This will
enable the company track the actions. It is important to constantly check new procedures so any
improvement can be maintained.
44
It is also important to continue to perform capability studies on the products to see if the
capability of the products change over time, to see if the improvements actually give good
results, and to control the variability in the process. This could, for example, be conducted four
times a year, once every quarter, or when changes are done.
Because there exist occasional situations where the PID control system cannot regulate the
high pressure within desired tolerances, a procedure/routine was chosen for handling this type
of cases:
If a high value for high pressure is obtained under a motor test (e.g., 177,1 bar) that
produces an alarm and a failed motor/test stop, the following three-step procedure is
recommended.
(1) Set the motor aside.
(2) Test the next two motors in the production order and then test the failed motor
again.
(3) Conclusion no alarm: High probability that the PID control system caused the
alarm.
Conclusion still alarm: High probability that the motor caused alarm, so it is worth
sending the motor back for control/investigation.
In a company-specific purpose, it would be important to continue to investigate the TB facility
by performing continuous analysis of the measurement results. A very simple way to do this is
to log all the parameters in a specific Excel document to, in a more easily way, use the data to
make an analysis using the software Minitab. Finally, it is recommended that the company
allocates improvement work to create better conditions for the motors in terms of cleanliness
and hopefully reach a reduced variability of ISO classes. Relevant places to perform these are
the suggested washing process, motor design, and/or the motor assembly to successfully
achieve an excellent capability for parameter cleanliness, which will also give a reduced
number of stopped motors when both the average value of ISO classes and the standard
deviation are reduced. Another important issue is whether the company believes it is worth
trying to upgrade all the filters that are included in the filter system. Presently, a 10 micro filter
with 75% efficiency (i.e., 75% of particles with the size of 10μm and larger are removed) is
used in most places. However, this is a fairly expensive investment. A final recommendation
for the company in terms of creating better conditions for motor cleanliness and the PID control
system’s ability to regulate the parameter high pressure is to improve pumps, valves, pipeline,
and fittings as dirt easily sticks to these parts as loose dirt falls out during motor start-up.
45
5. Conclusions The following chapter provides a conclusion of the aim of the conducted study at BRAB and
in what way the aim has been fulfilled.
The aim of the study was to evaluate the capability in the TB process for the CA motor with
help of Measurement System Analysis and to investigate if the TB facility performs as it is
supposed to do and to what extent the products meet the stated specifications (tolerances) of
some critical parameters. From the results, proposals were developed to improve safety, the
measurement system, and the capability of the TB process. In addition, proposals were made
regarding long-run production. In this chapter, the study questions that were founded in section
1.4 are reviewed and answered. The questions are answered in the original order of priority.
1. What uncertainty (variability) has the measurement system in the TB for the motor type
CA?
The uncertainty and variability of the measurement system (i.e., the TB) has been established
for the three critical parameters high pressure, external leaks, and cleanliness. For all the
parameters, variability between operators was very low. The results also showed that the
parameter high pressure in terms of variability and uncertainty is acceptable, but even that
improvement should be designed for the existing PID control system to obtain an excellent
measurement system in the future and to avoid costs associated with full alarms.
The parameter external leaks showed a low variability of the measurement system and high
precision of the measurements in terms of the level of tolerance. In contrast, the results showed
that in reality the motors had lower leakage than current tolerance requirements demand. For
the parameter cleanliness, the measurement system showed an uncertainty with respect to
measures needed to be investigated in order to obtain a more quality safe measurement system.
2. What capability exhibits the motor type CA?
The capability of the CA 50 motors as a product was determined in terms of the two parameters
external leaks and cleanliness. The parameter high pressure could not be investigated for the
motors, but became a production parameter of internal interest. The parameter high pressure
and cleanliness showed decent results for a test process, but needs improvement in order to
fulfill the demand of Six Sigma capability (i.e., 1.33). The parameter external leaks received
very good capability mainly because the company used a level of tolerance that was up to date
after an improvement of the product had been performed.
3. How should the procedures for process control, Measurement System Analysis, and
capability studies be designed in the company in order to control and assure that the
new levels of improvements are maintained, achieve a capable TB process and get
knowledge on how the capability for the products varies over time?
No previous Measurement System Analysis or capability study has been performed for the parts
concerning the parameters and processes that this project worked with. In order to face the
uncertainty of the various parameters changes have been made that in turn may affect the testing
process and ultimately reach a completely capable TB process. The Measurement System
Analysis and capability study done in the project can therefore be used for periodic inspections
of the process. For short-term management of uncertainty for the parameter high pressure and
46
cleanliness, a routine was developed for handling test alarms and new internal tolerances to
make a quality assurance of the ISO 4406 (i.e., measuring the parameter cleanliness).
This study has shown how the use of Six Sigma’s DMAIC approach can be used for extensive
Measurement System Analysis and a capability study for a measurement process in the form of
final tests of hydraulic motors. The study’s results could be used to develop other TB procedures
within BRAB as well as with other companies that use similar types of final testing of products,
particularly with regard to situations where Destructive Testing is used. This improvement in
the form of a case study could therefore also be applicable to other organizations that desire to
reduce customer complaints, which in a sustainable perspective is a positive result. The
completed case study also shows how important it is to work with quality throughout the
production chain when a process depends on the present processes, for example, the cleanliness
of the products.
47
6. Discussion This chapter discusses personal reflections, results of the thesis, the study’s validity and
reliability, DMAIC, and the need for future studies.
6.1 Personal reflections and results
Measurement System Analysis with help of relevant theory has been a central part in the
execution of this project. The original purpose designed by BRAB for the thesis was rather
vague: to obtain stable processes in the production of critical dimensions based on their
function. However, after extensive discussions with the issuer of the project, the purpose
became clearer, i.e. to evaluate the capability in the TB process concerning the CA motor at
BRAB with help of Measurement System Analysis and to investigate if the TB facility performs
as it is supoosed to do and to what extent the products meet the stated specifications (tolerances)
of some critical parameters. The company had experienced unreliability in their TB process as
well as for a number of critical dimensions. Since the TB process is the last step before the
hydraulic motors are sent to customers, it was of interest to reduce the risk of motor returns and
reworking of motors. These issues helped form the aim of the thesis.
At the start of the project, the initial project meetings with the recommended persons from the
R&D department provided very good support for both identifying and limiting the dozens of
factors that the TB handled. The factors that were not included in the study were considered
either not relevant or were not important from a customer perspective. This provided good
conditions for an interesting work. Since a motor test for the CA 50 motor took just over 30
minutes, it was necessary for the author to carry out tests using a reference motor, where the
test could be reduced to a third of the time but still have representative measurements of the
two parameters high pressure and external leaks. The study could have been improved if motors
from the normal production were supplied for the measurements, a design that would have made
generalizations even better.
The project with its experiments in the form of Measurement System Analysis has resulted in
an increased knowledge about the TB process that contribute most to the variability of the
parameters high pressure, external leaks, and cleanliness. For the parameter cleanliness, an
alternative Measurement System Analysis was used – the Nested Gauge R&R. This was
necessary to apply Measurement System Analysis despite the character of the parameter with
its destructive kind (i.e., the conditions for motor cleanliness change after the motor had been
tested). Based on the conditions for implementation of the project during a normal production
at the company, this method was good enough to use based on the actual conditions of the
project and despite the little uncertainty that exists with the assumption of homogeneous motors.
This uncertainty is based on the variability contribution of the products and that the
Measurement System could not be separated completely when they were superimposed. It was
prescient that the R&D team paired similar motors if possible to form simulated replicates and
then to plan the production order with the production planner.
It was difficult to implement improvement measures for the TB process because it required
extensive and costly work. Therefore, the improvement suggestions that the author brought
forward could not be tested and verified within the time frame of the thesis. In contrast, the
author believes that if these improvements are introduced and implemented, the chances of
reaching high quality are likely to be achieved and eventually also create conditions for cleaner
motors under testing and reduces the downtime of the TB facility.
48
BRAB has several TB facilities within the company for different motor types. The
manufacturing process for the different motors differs to some extent, but the test bench almost
always works in an identical manner regardless of the motor type. Therefore, it may be useful
to generalize the results in this study to the other TBs and the motor types, so perhaps this study
can be used as a basis for other work. With the improvement proposals and recommendations,
this thesis also addresses sustainability from an environmental perspective as the company will
produce cleaner motors, so fewer motors will be sent back from the customers, reducing
unnecessary transport.
6.2 Validity and reliability
To achieve good generalization, it was important that the study had high validity. Based on this
study, it is possible to use the information and approach within the Analyze phase to perform
similar analyses of other TBs and machine types for similar organizations/facilities. The use of
DMAIC has resulted in a number of suggestions for improvement. Since the DMAIC method
is a well-known and structured method for improvement, it can be used in a very broad area.
This gives it a high generalizability. The project is largely based on Sörqvist and Höglund’s
(2007) interpretation of the DMAIC methodology with the goal to create a good structure and
simplicity to follow the different phases. The author believes that there is enough evidence to
repeat the same kind of study for the other TBs and products within the company, but also in
other companies and organizations.
Graphs and figures were used whenever it seemed suitable to increase understanding. Several
sources were used in the analyze phase to estimate, for example, the repeatability and
reproducibility. Several sources were also used to obtain information on which data should be
collected and how this would be done as well as the observations during normal production and
various types of interviews to compare to the current work practices. This created triangulation
that made the results more truthful. All this together is therefore helpful to meet both high
external and internal validity and high reliability in the thesis. The variability of contribution
that was identified and occurred due to the measurement error was chosen from the results of
the Nested Gauge R&R study. This was then compared with corresponding values of the
parameter cleanliness that were measured at the reference motor. This meant that the results
from the reference motor were used as references in which the results seemed to be almost
identical. The fact that the two different methods showed the same result strengthens the
reliability for the measurements. To further strengthen the reliability, several sources came to
the same conclusions.
49
6.3 Further studies
One of the results of the study was that a new tolerance for the parameter external leaks is
relevant to establish for the CA 50 motor. It would therefore be interesting to examine whether
this proposal for tolerance can be generalized for all the CA motor types. It would also be
interesting for external leaks to investigate why the process was wobblier after the measurement
(motor) 31 around 2016-01-25.
During the Define phase, the limitation in the work was made for the parameters high pressure,
external leaks, and cleanliness. Something that was identified at a later date, together with R&D
under the project, was that one of the most important functions during motor tests is that the
parameter cleanliness does not show an upward trend. Therefore, it is proposed for future
studies to investigate the so-called delta (i.e., the difference between the start and end of the
cleanliness of the motors). Same type of delta approach could in advance also be used as an
investigated parameter for the ratio between high and low pressure.
Something that is of great interest for the company is the idea to carry out similar analysis on
other TBs. It would be quite straightforward, based on the analysis part of this project, to
perform capability studies and measurement systems analysis on TB 2 for the CA 70, CA 100,
CA 140, and CA 210 motor and also the motor types CBM and CB with their associated TBs.
Since the cause of the problem of the uncertainty in the counting system within the particle
could depend on several reasons, a future recommendation will be to investigate the cause of
the problem to the particle counter measurement uncertainty. After analysis of the function
made by the author, the conclusion was that it could be due to two reasons. The first reason
would be the counting system’s ability to make an appropriate transformation between the exact
number of particles and the corresponding ISO classes. The other reason, which includes the
light that the particle counter uses to identify particles, will have the consequence that a portion
of the particles counted in the oil are incorrectly sized.
50
References Ackermann, C. (1993). Evaluating Destructive Measurements using Gage R & R. Proceedings.
IEEE/SEMI Advanced Semiconductor Manufacturing Conference and Workshop, 101-103. doi:
10.1109/asmc.1993.682490.
Al-Baik, O. & Miller, J. (2014). The Kanban Approach, between Agility and Leanness: A
Systematic Review. Empirical Software Engineering Empire Software Eng, 20(6), 1861-1897.
doi: 10.1007/s10664-014-9340-x.
Al-Refaie, A. & Bata, N. (2010). Evaluating Measurement and Process Capabilities by G R&
R with four Quality Measures. Measurement, 43(6), 842-851.
doi: 10.1016/j.measurement.2010.02.016.
Asif, M., Vries, H. J. & Ahmad, N. (2013). Knowledge Creation through Quality Management.
Total Quality Management & Business Excellence, 24(5-6), 664-677.
doi: 10.1080/14783363.2013.791097.
Arntz-Gray, J. (2016). Plan, Do, Check, Act: The Need for Independent Audit of the Internal
Responsibility System in Occupational Health and Safety. Safety Science, 84, 12-23. doi:
10.1016/j.ssci.2015.11.019.
Bergman, B. & Klefsjo, B. (2012). Kvalitet: Fran behov till anvandning (5th ed.). Lund:
Studentlitteratur AB.
Blunch, N. J. (2015). Introduction to Structural Equation Modelling using IBM SPSS Statistics
and EQS. Sage Publications.
Brannstrom-Stenberg, A. & Deleryd, M. (1999). Implementation of Statistical Process Control
and Process Capability Studies: Requirements or Free Will? Total Quality Management, 10(4-
5), 439-446. Doi: 10.1080/0954412997389.
Boschrexroth.com. (2016). About-Bosch-Rexroth-Sweden-I. Retrieved 22 January 2016, from
http://www.boschrexroth.com/sv/se/foeretaget/om-bosch-rexroth-sverige/about-bosch-
rexroth-sweden-1
Dalalah, D. & Hani, D. (2016). On the Actual and Observed Process Capability Indices: A
Signal-to-Noise Ratio Model. Measurement, 81, 241-250.
doi: 10.1016/j.measurement.2015.12.018.
Diering, M., Hamrol, A. & Kujawińska, A. (2015). Measurement System Analysis Combined
with Shewhart’s Approach. KEM Key Engineering Materials, 637, 7-11.
doi: 10.4028/www.scientific.net/KEM.637.7.
Dreachslin, J. & Lee, P. (2007). Applying Six Sigma and DMAIC to Diversity Initiatives.
Journal of Healthcare Management, 52(6), 361-367.
Easton, G. S. & Rosenzweig, E. D. (2012). The Role of Experience in Six Sigma Project
Success: An Empirical Analysis of improvement projects. Journal of Operations Management,
30(7-8), 481-493. doi: 10.1016/j.jom.2012.08.002.
51
Ferreira, M., Tereso, A., Ribeiro, P., Fernandes, G. & Loureiro, I. (2013). Project Management
Practices in Private Portuguese Organizations. Procedia Technology, 9, 608-617.
doi: 10.1016/j.protcy.2013.12.067.
Flott, L. W. (1999). Quality Control - Introduction to Control Charts. Metal Finishing, 110(6),
36-38. doi: 10.1016/S0026-0576(13)70220-4.
Franco, B. C., Celano, G., Castagliola, P. & Costa, A. (2014). Economic Design of Shewhart
Control Charts for Monitoring Autocorrelated Data with Skip Sampling Strategies.
International Journal of Production Economics, 151, 121-130.
Garstenauer, A., Blackburn, T. & Olson, B. (2014). A Knowledge Management Based
Approach to Quality Management for Large Manufacturing Organizations. Engineering
Management Journal, 26(4), 47-58. doi: 10.1080/10429247.2014.11432028.
Gejdoš, P. (2015). Continuous Quality Improvement by Statistical Process Control. Procedia
Economics and Finance, 34, 565-572. doi: 10.1016/s2212-5671(15)01669-x.
Ghorbani, M., Arabzad, S. M. & Shahin, A. (2013). A Novel Approach for Supplier Selection
Based on the Kano Model and Fuzzy MCDM. International Journal of Production Research,
51 (18), 5469-5484. doi: 10.1080/00207543.2013.784403.
Gorman, D. & Bower, K. M. (2002, August). Measurement System Analysis. Six Sigma Forum
Magazine, 16-19.
Hoekstra, A. Y. (2015). The Sustainability of a Single Activity, Production Process or Product.
Ecological Indicators, 57, 82-84. doi: 10.1016/j.ecolind.2015.04.022.
Jha, C. M. (2015). Thermal sensors: Principles and Applications for Semiconductor Industries.
New York, NY: Springer.
Kenny, D. A. (1994). Interpersonal Perception: A Social Relations Analysis. New York:
Guilford Press.
Klefsjo, B., Eliasson H., Kennerfalk L., Lundbäck A. & Sandström M. (1999). De sju
ledningsverktygen: For effektivare planering av forbattringsarbetet. Lund: Studentlitteratur.
Kumar, D. & Kaushish, D. (2015). Scrap Reduction in a Piston Manufacturing Industry: An
Analysis Using Six Sigma and DMAIC Methodology. The IUP Journal of Operations
Management, 14(2), 8-24.
Montgomery, D. (2001). Introduction to Statistical Quality Control (4th ed.). New
York: Wiley.
Montgomery, D. (2013). Statistical Quality Control: A Modern Introduction (7th ed.).
Singapore: Wiley Singapore Pte.
Rawson, J. V., Kannan, A. & Furman, M. (2015). Use of Process Improvement Tools in
Radiology. Current Problems in Diagnostic Radiology, 1-7.
doi: 10.1067/j.cpradiol.2015.09.004.
52
Rybicka, J., Tiwari, A., Campo, P. A. & Howarth, J. (2015). Capturing Composites
Manufacturing Waste Flows through Process Mapping. Journal of Cleaner Production, 91,
251-261. doi: 10.1016/j.jclepro.2014.12.033.
Saunders, M., Lewis, P. & Thornhill, A. (2012). Research Methods for Business Students.
Harlow: Financial Times Prentice Hall.
Shinde, J. H. & Katikar, R. S. (2012). Importance of Process Capability and Process
Performance Indices in Machine Tool. International Journal of Research in Engineering &
Applied Sciences, 2(2), 1211-1217.
Sin, A. B., Zailani, S., Iranmanesh, M. & Ramayah, T. (2015). Structural Equation Modelling
on Knowledge Creation in Six Sigma DMAIC Project and its Impact on Organizational
Performance. International Journal of Production Economics, 168, 105-117.
doi: 10.1016/j.ijpe.2015.06.007.
Sorqvist, L. & Höglund F. (2007). Sex Sigma: Resultatorienterat forbattringsarbete som ger
ökad lonsamhet och nojdare kunder vid produktion av varor och tjanster (1st ed.) Lund:
Studentlitteratur.
Vannman, K. (2002). Matematisk statistik (10th ed). Lund: Studentlitteratur.
Walliman, N. (2011). Research Methods: The Basics. London: Routledge.
Wiengarten, F., Fynes, B., Cheng, E. T., & Chavez, R. (2013). Taking an Innovative Approach
to Quality Practices: Exploring the Importance of a Company’s Innovativeness on the Success
of TQM Practices. International Journal of Production Research, 51(10), 3055-3074.
doi: 10.1080/00207543.2012.752609.
Wu, C., Pearn, W. & Kotz, S. (2009). An Overview of Theory and Practice on Process
Capability Indices for Quality Assurance. International Journal of Production Economics,
117(2), 338-359. doi: 10.1016/j.ijpe.2008.11.008.
Xu, Q., Jiao, R. J., Yang, X., Helander, M., Khalid, H. M. & Opperud, A. (2009). An Analytical
Kano Model for Customer Need Analysis. Design Studies, 30(1), 87-110.
doi: 10.1016/j.destud.2008.07.001.
Yang, L. (2013). Key Practices, Manufacturing Capability and Attainment of Manufacturing
Goals: The Perspective of Project/Engineer-To-Order Manufacturing. International Journal of
Project Management, 31(1), 109-125. doi: 10.1016/j.ijproman.2012.03.005.
Yin, R. K. (2003). Case Study Research: Design and Methods. Thousand Oaks, CA: Sage
Publications.
Yin, R. K. & Nilsson, B. (2007). Fallstudier: Design och genomforande. Malmo: Liber.
1
Appendix A – TB results
Table A.1 Data collection for the CA 50 motor in TB 1
2
Table A.2 Summary of data used for the parameters high pressure and external leaks MSA
Table A.3 Summary of data used for the cleanliness’s MSA through Nested Gauge R&R
3
Table A.4 Summary of data used for the cleanliness’s MSA through Nested Gauge R&R
4
Table A.5 Summary of data used for the cleanliness’s MSA through Nested Gauge R&R
1
Appendix B – MSA In Appendix B, a Measurement System Analysis was performed to investigate the variability
of contribution from the Measurement System in terms of repeatability and reproducibility. In
Tables A.1, A.2, and A.3 are the data collected for this analysis. Note that this is the maximum
repeatability and reproducibility that can occur based on the measured values that have been
used. Also note that the operator interaction was not significant for any of the three parameters.
High pressure P3 (bar) First, the repeatability was analyzed for the parameter high pressure (i.e., the control system’s
ability to make repeated adjustments of the pressure on the same motor). The values for the
coming equations are taken from previous calculations in Table A2. Equation B.3 gives a score
of 0.217 for the total variability that exist from the repeatability.
Operator 1:
��1 = 𝑠 = √1
(10−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,26013 (B.1)
Operator 2:
��2 = 𝑠 = √1
(10−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,16364 (B.2)
The weighted standard deviation of repeatability is then calculated according to equation B.3.
��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 = �� = √��12+ ��2
2
2= 0,2173081527 (B.3)
The next step in the analysis was to examine the reproducibility, i.e., if the result was different
when different operators conducted motor tests for the parameter high pressure. With the use
of Table B.1, the results were obtained in Equation B.4, B.5, and B.6.
��𝑚𝑎𝑥 = 𝑚𝑎𝑥(��1, ��2) = 175,09 (B.4)
��𝑚𝑖𝑛 = 𝑚𝑖𝑛(��1, ��2) = 175,07 (B.5)
𝑅�� = ��𝑚𝑎𝑥 − ��𝑚𝑖𝑛 = 175,09 − 175,07 = 0,02 (B.6)
2
To investigate if there was a systematic difference when different operators performed the
motor tests other than the next calculation, the established method sampling in pairs was also
made in which the result of the sample tests can be seen in the Table B.1 below where 𝑧𝑖 =
𝑂𝑝𝑒𝑟𝑎𝑡𝑜𝑟 2 − 𝑂𝑝𝑒𝑟𝑎𝑡𝑜𝑟 1.
Table B.1 Sampling in pairs for high pressure with the used input data
Motor test 1 2 3 4 5 6 7 8 9 10
Operator 1 175,2 175,4 175,5 175,1 175 175,1 174,9 174,9 174,6 175,2
Operator 2 175,1 175,1 174,9 174,9 174,9 174,9 175,3 175,2 175,3 175,1
𝒛𝒊 -0,1 -0,3 -0,6 -0,2 -0,1 -0,2 0,4 0,3 0,7 -0,1
The mean for z and the estimate of the standard deviation was calculated using equation B.7
and B.8.
𝑧 = 1
10∑ 𝑧𝑖 = −0,0210
𝑖=1 (B.7)
𝑠 = √1
10−1∑ (𝑧𝑖 − 𝑧)210
𝑖=1 = 0,3794733192 (B.8)
The confidence level was selected to 0.95 and to have a 95% confidence interval when it was
considered to be good enough for the study. The T-ratio became 𝑡0,025(9) = 2,262. Next, a
setup for the T-interval could be made using equation B.7.
[𝑧 − 𝑡𝛼/2𝑠
√𝑛, 𝑧 + 𝑡𝛼/2
𝑠
√𝑛] = (B.9)
[−0,02 − 2,2620,3795
√10, −0,02 + 2,262
0,3795
√10] =
[−0,29144, 0,25144]
3
As this confidence interval contains zero, we can say with certainty (1 – α; i.e., 95%) that the
operators had no significant impact on the outcome, i.e., reproducibility of parameter high
pressure can be considered low.
��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦 = 𝑅��
𝑑2 =
0,02
1,128= 0,0177304965 (B.10)
The estimated standard deviation of reproducibility was then calculated using equation B.10
whose results prove the previously calculated confidence interval with method sampling in pairs
in equation B.9 because the standard deviation was low. Here the 𝑑2 factor from Appendix VI
of Montgomery (2013) was used according to that 𝑅�� is the range of a sample size of two
according to equation B.10. Now we have both components of the measurement error standard
deviation, 𝜎𝑔𝑎𝑢𝑔𝑒 and could calculate the total estimated variance of the measurement ��𝑔𝑎𝑢𝑔𝑒2
by equation B.11.
𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = ��𝑔𝑎𝑢𝑔𝑒
2 = ��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦2 + ��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦
2 (B.11)
= (0,217)2 + (0,018)2 = 0,0475372038
The standard deviation of the total measurement error (in this case, the control error) could then
be obtained when both repeatability and reproducibility was taken into account according to
equation B.12.
��𝑔𝑎𝑢𝑔𝑒 = √𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = 0,2180302817 (B.12)
From this, a P/T ratio can be calculated for this standard deviation by equation B.13.
𝑃
𝑇=
6��𝑔𝑎𝑢𝑔𝑒
𝑈𝑆𝐿−𝐿𝑆𝐿=
6∗0,2180302817
𝑋𝑋𝑋−𝑋𝑋𝑋= 0,3270454226 (B.13)
Based on the results of the P/T ratio of 32.7%, it could be concluded that the Measurement
system is not acceptable. This P/T ratio of a measurement system would be considered to be
acceptable if it was ≤ 0.30 (i.e., a maximum of 30%), but this is not the case. However, it is
important to keep in mind that the analysis of the parameter high pressure is an analysis of an
output parameter and thus the control system contributes to a variation of the motor so the
estimate of ��𝑔𝑎𝑢𝑔𝑒 becomes the theoretical maximum variability contribution of measurement
error that may occur. With both the control system itself and the pump that supplies pressure of
the products (i.e., the contributed variability that comes from motors is included), it may be
reasonable to have a higher P/T ratio and still see that as acceptable and because of the task of
the existing regulator that controls the oscillations of pressure and that friction in the pump and
motors changes with the temperature.
4
In order to relate to the proportion of the variability that exists and that comes from the control
system, the equation B.14 is used where the variability of the Measurement System, 𝜎𝐺𝑎𝑢𝑔𝑒2 ,
has already been calculated in Equation B.12 and the total standard deviation, 𝜎𝑇𝑜𝑡𝑎𝑙 , has been
calculated on the basis of the measured values in Table A.1.
𝜎𝑇𝑜𝑡𝑎𝑙 2 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛
2 + 𝜎𝐺𝑎𝑢𝑔𝑒2 (B.14)
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛2 = 𝜎𝑇𝑜𝑡𝑎𝑙
2 − 𝜎𝐺𝑎𝑢𝑔𝑒2
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛2 = 0,55806
2 − 0,0475 = 0,2638840016
If the total variability of the process, 𝜎𝑇𝑜𝑡𝑎𝑙2 , is 100%, the variability of the control system,
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛2 , can measure 90% of the total variance, and the variance of the measurement
system, σGauge2 , is 10% of the total variability. To investigate this, equation B.15 and B.16 are
used.
𝜌𝑃 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.15)
𝜌𝑃 = 0,2688
0,3114= 0,8473539922
𝜌𝑀 = 𝜎𝐺𝑎𝑢𝑔𝑒
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.16)
𝜌𝑀 = 0,0475
0,3114= 0,152646008
The results show that the control system stands for 84.7% of the total variability in the process
and the measurement system for 15.3% of the total variability in the process.
External leaks QY (l/min)
Operator 1:
5
��1 = 𝑠 = √1
(𝑛−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,094868 (B.17)
Operator 2:
��2 = 𝑠 = √1
(𝑛−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,10801 (B.18)
The weighted standard deviation for the repeatability could then be calculated using equation
B.19.
��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 = 𝑠 = √��12+ ��2
2
2= 0,1016516048 (B.19)
��𝑚𝑎𝑥 = 𝑚𝑎𝑥(��1, ��2) = 0,77 (B.20)
��𝑚𝑖𝑛 = 𝑚𝑖𝑛(��1, ��2) = 0,75 (B.21)
𝑅�� = ��𝑚𝑎𝑥 − ��𝑚𝑖𝑛 (B.22)
= 0,77 − 0,75 = 0,02
To investigate if a systematic difference of the operators performing the motor test for the
parameter external leaks existed, sampling in pairs was also performed here. The result of the
tests can be seen in Table B.2, where 𝑧𝑖 = 𝑂𝑝𝑒𝑟𝑎𝑡𝑜𝑟 2 − 𝑂𝑝𝑒𝑟𝑎𝑡𝑜𝑟 1.
Table B.2 Sampling in pairs for external leaks with the used input data
Motor test 1 2 3 4 5 6 7 8 9 10
Operator 1 0,8 0,6 0,8 0,8 0,9 0,8 0,8 0,8 0,8 0,6
Operator 2 0,8 0,8 0,8 0,8 0,9 0,6 0,6 0,6 0,8 0,8
𝒛𝒊 0 0,2 0 0 0 -0,2 -0,2 -0,2 0 0,2
6
𝑧 = 1
10∑ 𝑧𝑖 = −0,0210
𝑖=1 (B.23)
𝑠 = √1
9∑ (𝑧𝑖 − 𝑧)210
𝑖=1 = 0,1348249894 (B.24)
Here the confidence level was once again chosen to be 0.95 to get a 95% confidence interval
which was considered good enough for the study. The T-ratio became 𝑡0,025(9) = 2,262.
After this, a setup of the range could be based on the same formula as before.
[𝑧 − 𝑡𝛼/2𝑠
√𝑛, 𝑧 + 𝑡𝛼/2
𝑠
√𝑛] = (B.25)
[−0,02 − 2,2620,1348
√10, −0,02 + 2,262
0,1348
√10] =
[−0,1164, 0,0764]
As this confidence interval also contains zero, we can with the certainty 1 - α state that operators
do not have any significant impact on the result, i.e., the reproducibility is very low even for
external leaks.
��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦 = 𝑅��
𝑑2 =
0,02
1,128= 0,0177304965 (B.26)
Here again the 𝑑2 factor from Appendix VI in Montgomery (2013) was used in the same sense
that 𝑅�� is the range from a test with size two according B.26.
Now both components for the standard deviation of the measurement error, 𝜎𝑔𝑎𝑢𝑔𝑒 , could
calculate the total estimated variability for the measurement error, ��𝑔𝑎𝑢𝑔𝑒2 , using equation B.27.
𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = ��𝑔𝑎𝑢𝑔𝑒
2 = ��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦2 + ��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦
2 (B.27)
= (0,102)2 + (0,01773)2 = 0,0106474193
Thus the standard deviation for the measurement error could be obtained from equation B.28
when both repeatability and reproducibility are taken into account.
7
��𝑔𝑎𝑢𝑔𝑒 = √𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = 0,1031863328 (B.28)
From this, a P/T ratio was calculated for the standard deviation by equation B.29 when the one-
sided specification limit exists, where the process mean for the 20 measurements are
𝑋𝑚𝑒𝑎𝑛 = 0.76.
𝑃
𝑇=
3��𝑔𝑎𝑢𝑔𝑒
|𝑈𝑆𝐿−𝑃𝑟𝑜𝑐𝑒𝑠𝑠 𝑀𝑒𝑎𝑛|=
3∗0,1031863328
|𝑋−0,76|= 0,0590761447 (B.29)
This result shows the percentage distribution that is based on the tolerances. Since the P/T ratio
is 5.9%, it shows a good measurement system; however, it is highly dependent on that the USL
is so much higher than the average value of the measurements for the process.
To calculate the proportion of the variability that exists and that comes from the product, (i.e.,
the reference motor in this case), equation B.30 was used where the variability of the
measurement system, 𝜎𝐺𝑎𝑢𝑔𝑒2 , has already been calculated in Equation B.27 and the total standard
deviation, 𝜎𝑇𝑜𝑡𝑎𝑙 2 , have been calculated on the basis of the measured values in Table A.1.
𝜎𝑇𝑜𝑡𝑎𝑙 2 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑘𝑡
2 + 𝜎𝐺𝑎𝑢𝑔𝑒2 (B.30)
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡2 = 𝜎𝑇𝑜𝑡𝑎𝑙
2 − 𝜎𝐺𝑎𝑢𝑔𝑒2
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡2 = 0,513733831
2 − 0,010647 = 0,2532754491
If the total variability of the process, 𝜎𝑇𝑜𝑡𝑎𝑙2 is 100%, the variability from the products,
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑠2 , can therefore measure a process with 90% of the total variance and the variance of
the measurement system, 𝜎𝐺𝑎𝑢𝑔𝑒2 , is 10% of the total variability. To investigate this, the
equations B.31 and B.32 were used.
𝜌𝑃 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.31)
𝜌𝑃 = 0,2533
0,2639= 0,9596586041
8
𝜌𝑀 = 𝜎𝐺𝑎𝑢𝑔𝑒
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.32)
𝜌𝑀 = 0,010647
0,2639= 0,0403429846
The results show that the CA 50 motor as product stands for 96% of the total variability in the
process and the measurement system stands for only 4% of the total variability in the process.
Cleanliness (4μm) - through Nested Gauge R&R
The Measurement System Analysis for parameter cleanliness (4μm, 6μm, and 14μm) is
different from the other two parameters with respect to destructive testing, so a Nested Gauge
R&R approach was used with the help of Minitab to compare the calculations by hand. This
means that the result was triangulated when several methods were used to reach the result.
The situation can be compared with the theory of destructive sampling for each particle size
4μm, 6μm, and 14μm, which in turn means that conditions being destroyed over time if the
same motor is tested repeated times. This is illustrated in Figure B.1 where a flattening of the
curves can be seen after about 100 hours for all three particle sizes. The big jumps upwards in
ISO classes in the graph occurred because the motor was stopped, and then, for example, the
solenoid valve or other parts were loosened for cleaning. This operation contributes to an
increased amount of dirt if the motor is restarted.
Figure B.1 Visual description over the degradation of the contamination (ISO) classes for each
particle size which is dependent on the run time, where the y-axis shows the contamination (ISO)
classes) and the x-axis indicates time in number of hours (h)
First, the repeatability was calculated with the help of the S-method for the average variability
in the measurement system, i.e., the TB.
9
Operator 1:
��1 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.33)
��3 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.34)
��5 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,7071 (B.35)
��7 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,7071 (B.36)
��9 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,7071 (B.37)
Operator 2:
��2 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 1,4142 (B.38)
��4 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 1,4142 (B.39)
��6 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,7071 (B.40)
��8 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.41)
10
��10 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,7071 (B.42)
The weighted standard deviation for the repeatability was thereafter calculated using equation
B.43.
��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 = 𝑠 = √��12+ ��2
2+ ��32+ ��4
2+��52+ ��6
2+ ��72+ ��8
2+ ��92+ ��10
2
10 (B.43)
= 0,8062180431
This result was then compared with the results in Minitab’s ANOVA Table B.5 (0.8944), which
indicates that the results differ only at 0.1 standard deviations. The result in equation B.43
(0.80622) prevails with the method of Nested Gauge R&R in Minitab. The average of these two
results was calculated and the results of the obtained, ��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 , were used. This was
assumed to be used in future calculations.
Table B.3
Table B.4
Table B.5
11
An individual analysis was performed in Minitab for cleanliness 4μm. Based on the results in
Figure B.2, Components of Variation had the same results as found in Table B.5, i.e., the largest
dimension is the source repeatability (% Study Var) at 99.84%. These results show an
unacceptable measurement system because it is over 30%. If we look at the corresponding
analysis of variance in Table B.3, we see that the operators’ effect is not significant at the level
α = 0.05. This is because the p-value = 0.339 (> 0.05). Therefore, the null hypothesis – Ho:
��2𝑂𝑝𝑒𝑟𝑎𝑡𝑜𝑟 = 0 – could not be rejected in this case, but since the operators never performed
measurements on identical motors, this result was not considered critical without a further
analysis. In the difference between motor groups in the source motor number (Operator) in the
same table, we see that the variability was very low, with a p-value of 0.508, which the
variability in turn is zero for part-to-part in Table B.4. This suggests that in addition to the
measurement system needs for improvement it could not detect the difference of the various
motor groups within each operator. Because the measurement system seems to be insufficient
regarding the repeatability error, it could highlight issues of the homogenous motor group
assumption that is made for this destructive testing regarding parameter cleanliness 4μm, but
since the motor’s models were not supposed to differ with the criteria they had, the conclusion
was for the weighted standard deviation of repeatability,��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦, which also showed a
high variability of the measurement system. However, we could not take into account the
reproducibility of this Nested method in Minitab for this situation as operators did not measure
identical motors.
12
Figure B.2 Summary of Nested Gauge R&R for 4μm
The R-chart in Figure B.2 shows that the level of variability within each motor group is very
low. Because the range showed the combined variability of both the repeatability of the
measurement system and the within group variability, this R-chart would help identify if
different operators had a difficulty consistently preparing and testing the parameter cleanliness
4μm and identifying specific motor groups that would not have been appropriate in the
homogeneous motor assumption. However, this is not the case here.
The X-chart in Figure B.2 shows that the motor group variability varies less than the control
limits. This result is not good because the control limits are based on the combined repeatability
and within-group variability, where the area between the control limits represents the
measurement system variability. If more than half of those points lie within these limits, this
type of instrument is unable to distinguish between the motors. This result shows that the
difference between the different motors is not likely to be discovered across the repeatability
error. The X-chart also shows the “By Motor number” (Operator) and “by Operator” have no
major differences in the average value between the operators’ measurements, which proves that
the randomization of motors to each of the two operators worked well as no operator seemed to
have motors with significantly higher values.
The collected data in Table A.4 and Table A.5 were used to estimate the respective operator’s
average and in the same way as before calculate an average variability that thus becomes an
estimate of the reproducibility of the average of the operator’s variability. This way the
estimated reproducibility of parameter cleanliness 4μm was triangulated, resulting in more
reliable results.
13
To calculate the reproducibility of cleanliness 4μm, sampling in pairs was first performed to
investigate whether there is any significant difference between the operators, and the R-method
based on Montgomery (2013) was used to calculate the size of the same.
A setup of a confidence interval was made based on the method of sample in pairs with the help
of the same previously used equation B.25 for the parameters high pressure and external leaks.
The result of the confidence interval can be seen in equation B.44.
[−0,7546, 1,0546] (B.44)
Since this confidence interval contained zero, we could say with certainty 1 - α that the operators
did not have any significant impact on the outcome, i.e., the reproducibility was very low for
the parameter cleanliness 4μm. Now, it remained to calculate how large this variability of
contribution from the operators was. This was done using the same tabular procedure based on
Montgomery (2013) where the values were taken from Table A.5.
��𝑚𝑎𝑥 = 𝑚𝑎𝑥(��1, ��2) = 14,55 (B.45)
��𝑚𝑖𝑛 = 𝑚𝑖𝑛(��1, ��2) = 14,4 (B.46)
𝑅�� = ��𝑚𝑎𝑥 − ��𝑚𝑖𝑛 (B.47)
= 14,55 − 14,4 = 0,15
��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦 = 𝑅��
𝑑2 =
0,15
1,128= 0,1329787234 (B.48)
Here the 𝑑2 factor was once again used from Appendix VI in Montgomery (2013): 𝑅�� is the
range from a test with size two according to equation B.48.
Now we had both components of the standard deviation from the measurement error, 𝜎𝑔𝑎𝑢𝑔𝑒,
and could calculate the total estimated variability for the measurement error, ��𝑔𝑎𝑢𝑔𝑒2 , using
equation B.49.
𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = ��𝑔𝑎𝑢𝑔𝑒
2 = ��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦2 + ��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦
2 (B.49)
14
= (0,8503)2 + (0,1330)2 = 0,74069909
Thus the standard deviation was obtained for the total measurement error using equation B.50
when both repeatability and reproducibility are taken into account.
��𝑔𝑎𝑢𝑔𝑒 = √𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = 0,8606387686 (B.50)
From this, a P/T ratio was calculated for the standard deviation by the equation B.51 when a
one-sided specification limit exists where the process mean, 𝑋𝑃𝑟𝑜𝑐𝑒𝑠𝑠 𝑀𝑒𝑎𝑛 = 13.9, was
calculated for the 50 measurements from Table A.1.
𝑃
𝑇=
3��𝑔𝑎𝑢𝑔𝑒
|𝑈𝑆𝐿−𝑃𝑟𝑜𝑐𝑒𝑠𝑠 𝑀𝑒𝑎𝑛|=
3∗0,8606387686
|𝑋−13,9|= 0,6297356843 (B.51)
This ratio of 62.97% was then compared with Table B.5 from Minitab at 59.72%, where both
indicate an unacceptable measurement system.
To determine the proportion of the variability that exists and where this variability came from,
equation B.52 was used where the variability of the measurement system, 𝜎𝐺𝑎𝑢𝑔𝑒2 , had already
been calculated in equation B.49 and the total standard deviation, 𝜎𝑇𝑜𝑡𝑎𝑙 , was taken from Table
A.1.
𝜎𝑇𝑜𝑡𝑎𝑙 2 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑘𝑡
2 + 𝜎𝐺𝑎𝑢𝑔𝑒2 (B.52)
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡2 = 𝜎𝑇𝑜𝑡𝑎𝑙
2 − 𝜎𝐺𝑎𝑢𝑔𝑒2
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡2 = 1,2495
2 − 0,7407 = 0,8205244904
If the total variability of the process, 𝜎𝑇𝑜𝑡𝑎𝑙2 , is 100%, the variability of the products, 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2 ,
could therefore be 90% of the total variance for a capable measurement system, and the
variability of the measurement system, 𝜎𝐺𝑎𝑢𝑔𝑒2 , could be 10% of the total variability for a
capable measurement system. To investigate this, equations B.53 and B.54 were used.
𝜌𝑃 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.53)
15
𝜌𝑃 = 0,8205
1,5612= 0,5255647062
𝜌𝑀 = 𝜎𝐺𝑎𝑢𝑔𝑒
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.54)
𝜌𝑀 = 0,7407
1,5612= 0,4744347112
The results show that the CA 50 motor as a product stands for 52.6% of the total variability in
the process and the measurement system for 47.4% of the total variability in the process.
Cleanliness (6μm) - Through Nested Gauge R&R
First, the repeatability was calculated with help of the S-method for the average variability in
the measurement system (i.e., the TB).
Operator 1:
��1 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 1,4142 (B.55)
��3 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.56)
��5 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.57)
��7 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.58)
��9 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.59)
16
Operator 2:
��2 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 2,8284 (B.60)
��4 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 2,8284 (B.61)
��6 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 1,4142 (B.62)
��8 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 1,4142 (B.63)
��10 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 1,4142 (B.64)
The weighted standard deviation for the repeatability was calculated using equation B.65.
��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 = �� = √��12+ ��2
2+ ��32+ ��4
2+��52+ ��6
2+ ��72+ ��8
2+ ��92+ ��10
2
10 (B.65)
= 1,54917848
This result was then compared with the results in Minitab’s ANOVA Table B.8 (1.54918),
which is very close to the calculation made by hand in equation B.65 (1.549178), which seems
to correlate with the method for Nested Gauge R&R in Minitab. If the average was taken for
these two results, the repeatability,σRepeatability, went to be 1.54917924. This value was then
used in future calculations.
Table B.6
17
Table B.7
Table B.8
An individual analysis was also performed in Minitab for cleanliness 6μm. Based on the results
in Figure B.3, the Components of Variation has the same results as in Table B.8, i.e., the largest
dimension is the source repeatability (% Study Var) at 94.46%. This result shows an
unacceptable measurement system because it is over 30%. If we look at the corresponding
analysis of variance in Table B.6, we see that the operators’ effect was not significant at the
level α = 0.05. This is because the p-value = 0.161 (> 0.05). Therefore, the null hypothesis –
Ho: ��2𝑂𝑝𝑒𝑟𝑎𝑡𝑜𝑟 = 0 – could not be rejected in this case either. Since the operators never
performed measurements on identical motors, this result was not considered critical without a
further analysis, a same condition as for cleanliness 4μm. In the difference between motor
groups in the source motor number (Operator) in the same table, we see that the variability was
very low, with a p-value of 0.567, which variability in turn is zero for part-to-part in Table B.7.
This suggests that, in addition to the measurement system, there is a need for improvement as
it could not detect the difference of the various motor groups within each operator. Because the
measurement system seems to be insufficient regarding the repeatability error, it could highlight
issues of the homogenous motor group assuming the destructive testing regarding the parameter
cleanliness 6μm. However, since the motors’ models were supposed to be the same, they the
previous results for the weighted standard deviation of repeatability, ��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦, also showed
a high variability of the measurement system. However, we could not take into account the
reproducibility of this Nested method in Minitab for this situation as operators did not do
measurements on identical motors.
18
Figure B.3 Summary of Nested Gauge R&R for 6μm
The R-chart in Figure B.3 shows that the level of variability within each motor group is very
low. In contrast, the range shows that identical motor groups seem to also apply for cleanliness
6μm, i.e., the homogeneous motor assumption worked well.
The X-chart in Figure B.3 shows that the motor group variability varied less than the control
limits. This result is, as previously mentioned, not good because the control limits are based on
the combined repeatability and within-group variability, where the area between the control
limits represents the measurement system variability. If more than half of those points lie within
these limits, it indicates that this type of instrument is unable to distinguish between the motors.
This result shows that the difference between the different motors is not likely to be discovered
across the repeatability error. The X-chart also shows along with “By Motor number”
(Operator) and “By Operator” that no major differences in the average value between the
operators’ measurements can be discerned, which proves that the randomization of motors to
each of the two operators worked well as no operator seemed to have motors with significantly
higher values.
The divided data in Table A.4 to Table A.5 was then used to compare an estimation of the
respective operator’s average and in the same way and as before calculate an average variability
that thus becomes an estimate of the reproducibility on the average of the operator’s variability.
This way the estimated reproducibility of parameter cleanliness 6μm was triangulated, which
gave more reliable results.
To calculate the reproducibility of cleanliness 6μm, sampling in pairs was first performed to
investigate whether there is any significant difference between the operators and the R-method
based on Montgomery (2013) was used to calculate the size of the same.
19
A setup of a confidence interval was made based on the method of sample in pairs with the help
of same previously used equation B.25 for the other parameters high pressure and external
leaks. The result of the confidence interval can be seen in equation B.66.
[−1,0771, 0,67706] (B.66)
Since this confidence interval contained zero, we could say with certainty 1 - α that the operators
did not have any significant impact on the outcome, i.e., the reproducibility was very low for
the parameter cleanliness 6μm. Now, it was time to calculate how large this variability of
contribution from the operators was. This was done using the same tabular procedure based on
Montgomery (2013) where the values were taken from Table A.5 as before.
��𝑚𝑎𝑥 = 𝑚𝑎𝑥(��1, ��2) = 12,0 (B.67)
��𝑚𝑖𝑛 = 𝑚𝑖𝑛(��1, ��2) = 11,8 (B.68)
𝑅�� = ��𝑚𝑎𝑥 − ��𝑚𝑖𝑛 (B.69)
= 12,0 − 11,8 = 0,20
��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦 = 𝑅��
𝑑2 =
0,20
1,128= 0,1773049645 (B.70)
Here the 𝑑2 factor was once again used from Appendix VI in Montgomery (2013) according to
that 𝑅�� where the range from a test with size two was calculated using equation B.70.
Now we had both components of the standard deviation from the measurement error, 𝜎𝑔𝑎𝑢𝑔𝑒,
and could calculate the total estimated variability for the measurement error, ��𝑔𝑎𝑢𝑔𝑒2 , through
equation B.71.
𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = ��𝑔𝑎𝑢𝑔𝑒
2 = ��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦2 + ��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦
2 (B.71)
= (1,5492)2 + (0,1773)2 = 2,431391013
20
Thus the standard deviation was obtained for the total measurement error using equation B.71
when both repeatability and reproducibility are taken into account.
��𝑔𝑎𝑢𝑔𝑒 = √𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = 1,559291831 (B.72)
From this, a P/T ratio was calculated for the standard deviation using the equation B.73 when
a one-sided specification limit exists where the process mean, 𝑋𝑃𝑟𝑜𝑐𝑒𝑠𝑠 𝑀𝑒𝑎𝑛 = 11.2, was
calculated for the 50 measurements from Table A.1.
𝑃
𝑇=
3��𝑔𝑎𝑢𝑔𝑒
|𝑈𝑆𝐿−𝑃𝑟𝑜𝑐𝑒𝑠𝑠 𝑀𝑒𝑎𝑛|=
3∗1,559291831
|𝑋−11,2|= 0,9745573944 (B.73)
This ratio of 97.46% was then compared with Table B.8 from Minitab at 71.31%, where both
indicate on an unacceptable measurement system. The reason that they became different may
have to do with the software’s method of calculation which calculates the value as the one-
sided variability divided by the one-sided tolerance and sees the one-sided process variability
as the “Study Var” divided by two. From these two results, the average value of the P/T ratio
(i.e., 84.38%) was calculated.
To relate to the proportion of the variability that exists and came from the products, equation
B.74 was used where the variability of the measurement system, 𝜎𝐺𝑎𝑢𝑔𝑒2 , had already been
calculated in equation B.72 and the total standard deviation, 𝜎𝑇𝑜𝑡𝑎𝑙 , was taken from Table A.1.
𝜎𝑇𝑜𝑡𝑎𝑙 2 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2 + 𝜎𝐺𝑎𝑢𝑔𝑒2 (B.74)
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡2 = 𝜎𝑇𝑜𝑡𝑎𝑙
2 − 𝜎𝐺𝑎𝑢𝑔𝑒2
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡2 = 1,60362 − 2,4314 = 0,1400375569
If the total variability of the process, 𝜎𝑇𝑜𝑡𝑎𝑙2 , is 100%, the variability of the products, 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2 ,
could be 90% of the total variance for a capable measurement system, and the variability of the
measurement system, 𝜎𝐺𝑎𝑢𝑔𝑒2 , could be 10% of the total variability for a capable measurement
system. To investigate this, the equations B.75 and B.76 were used.
𝜌𝑃 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.75)
21
𝜌𝑃 = 0,1400
2,5725= 0,0544590499
𝜌𝑀 = 𝜎𝐺𝑎𝑢𝑔𝑒
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.76)
𝜌𝑀 = 2,4314
2,5725= 0,94554095
The results show that the CA 50 motor as a product stands for 5.4% of the total variability in
the process and the measurement system for 94.6% of the total variability in the process.
Therefore, the measurement system stands for any variability of the total variance, i.e., 100%
of the variance since the variance of the reference motor could be seen as zero.
Cleanliness (14μm) - Through Nested Gauge R&R
First, the repeatability was calculated with help of the S-method for the average variability in
the measurement system (i.e., the TB).
Operator 1:
��1 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 1,4142 (B.77)
��3 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 3,5355 (B.78)
��5 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.79)
��7 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.80)
22
��9 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0 (B.81)
Operator 2:
��2 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 1,4142 (B.82)
��4 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,7071 (B.83)
��6 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 4,2426 (B.84)
��8 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 2,8284 (B.85)
��10 = 𝑠 = √1
(2−1)∑ (𝑥𝑖 − ��)2𝑛
𝑖=1 = 0,7071 (B.86)
The weighted standard deviation for the repeatability was calculated using equation B.87.
��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 = �� = √��12+ ��2
2+ ��32+ ��4
2+��52+ ��6
2+ ��72+ ��8
2+ ��92+ ��10
2
10= 2,08564536
(B.87)
This result was then compared with the results for the repeatability, σRepeatability, in Minitab’s
ANOVA (Table B.11) (2.08567), which is very close to the calculation made by hand in
equation B.88 (2.0856), which correlates with the method for Nested Gauge R&R in Minitab.
Table B.9
23
Table B.10
Table B.11
An individual analysis was also performed in Minitab for cleanliness 14μm. Based on the
results in Figure B.4, the Components of Variation had the same results as Table B.8, i.e., the
largest dimension is the source repeatability (% Study Var) at 87.07%. This result shows an
unacceptable measurement system because it is over 30%. If we look at the corresponding
analysis of variance in Table B.9, we see that the operators’ effect was not significant at the
level α = 0.05. This is because the p-value = 0.384 (> 0.05). Therefore, the null hypothesis –
Ho: ��2𝑂𝑝𝑒𝑟𝑎𝑡𝑜𝑟 = 0 – could not be rejected in this case either. Since the operators never
performed measurements on identical motors, this result was not considered critical without a
further analysis; this was also the same as for cleanliness 4 μm and 6μm. In the difference
between motor groups in the source motor number (Operator) in the same table, we see that
there exists a variability, but with a p-value on 0.229. Where the variability is 49.18 for part-to-
part according to Table B.1, it is much smaller than the variability that comes from repeatability.
These results can be interpreted as the measurement system needs to be improved, but it seems
to detect differences better at the various motor groups within each operator than the previous
levels of cleanliness. More measurements would be needed to confirm this.
The case that the measurement system is insufficient regarding the repeatability error could
highlight issues of the homogenous motor group assumption made for this destructive testing
in terms of cleanliness 14μm. But when the motor criteria were unable to differentiate models,
the earlier results for the weighted standard deviation for the repeatability, σRepeatability, also
showed a high variability (i.e., the measurement system was not suitable for cleanliness 14μm).
However, we cannot take into account the reproducibility of this method in the Nested Minitab
for this situation as the operators did not do measurements on identical engines.
24
Figure B.4 Summary of Nested Gauge R&R for 14μm
The R-chart in Figure B.4 shows that the level of variability within each motor group is very
low. In contrast, the range shows that identical motor groups seem to apply also for cleanliness
14μm, i.e. the homogeneous motor assumption worked well.
The X-chart (Figure B.4) shows that the motor group variability varied less than the control
limits. This result is, as previously mentioned, not good because the control limits are based on
the combined repeatability and within-group variability, where the area between the control
limits represents the measurement system variability. If more than half of those points lie within
these limits, it indicates that this type of instrument is unable to distinguish between the motors.
This result shows that the difference between the different motors is not likely to be discovered
across the repeatability error. The X-chart also shows along with “By Motor number”
(Operator) and “By Operator” that no major differences in the average value between the
operators’ measurements can be discerned, which proves that the randomization of motors to
each of the two operators worked well when no operator seemed to have motors with
significantly higher values.
The divided data in Table A.4 and Table A.5 was then used to compare an estimation of the
respective operator’s average and in the same way as before calculated an average variability
that thus becomes an estimate of the reproducibility on the average of the operator’s variability.
In this way, the estimated reproducibility of parameter cleanliness 14μm was triangulated,
giving more reliable results.
To calculate the reproducibility of cleanliness 14μm, sampling in pairs was first performed to
investigate whether there is any significant difference between the operators and the R-method
based on Montgomery (2013) and this was used to calculate the size.
25
A setup of a confidence interval was made based on the method of sample in pairs with the help
of same previously used equation B.25 for the other parameters high pressure and external
leaks. The result of the confidence interval can be seen in equation B.88.
[−0,87823, 0,97823] (B.88)
Since this confidence interval contained zero, we could say with certainty 1 - α that the operators
did not have any significant impact on the outcome, i.e., the reproducibility was very low for
the parameter cleanliness 14μm. Now it remained to calculate how large this variability of
contribution from the operators was. This was done using the same tabular procedure based on
Montgomery (2013) where the values were taken from Table A.5 as before.
��𝑚𝑎𝑥 = 𝑚𝑎𝑥(��1, ��2) = 6,85 (B.89)
��𝑚𝑖𝑛 = 𝑚𝑖𝑛(��1, ��2) = 6,80 (B.90)
𝑅�� = ��𝑚𝑎𝑥 − ��𝑚𝑖𝑛 (B.91)
= 6,85 − 6,80 = 0,05
��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦 = 𝑅��
𝑑2 =
0,05
1,128= 0,0443262411 (B.92)
Here the 𝑑2 factor was once again used from Appendix VI in Montgomery (2013) as 𝑅�� is the
range from a test with size two according to equation B.92.
Now we had both components of the standard deviation from the measurement error, 𝜎𝑔𝑎𝑢𝑔𝑒,
and could calculate the total estimated variability for the measurement error, ��𝑔𝑎𝑢𝑔𝑒2 , using
equation B.93.
𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = ��𝑔𝑎𝑢𝑔𝑒
2 = ��𝑅𝑒𝑝𝑒𝑎𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦2 + ��𝑅𝑒𝑝𝑟𝑜𝑑𝑢𝑐𝑖𝑏𝑖𝑙𝑖𝑡𝑦
2 (B.93)
= (2,0856)2 + (0,0443)2 = 4,351881383
26
Thus the standard deviation was obtained for the total measurement error using equation B.94
when both repeatability and reproducibility are taken into account.
��𝑔𝑎𝑢𝑔𝑒 = √𝜎𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟2 = 2,08611634 (B.94)
From this, a P/T ratio was calculated for the standard deviation by the equation B.95 when a
one-sided specification limit exists where the process means, 𝑋𝑃𝑟𝑜𝑐𝑒𝑠𝑠 𝑀𝑒𝑎𝑛 = 5.72, was
calculated for the 50 measurements from Table A.1.
𝑃
𝑇=
3��𝑔𝑎𝑢𝑔𝑒
|𝑈𝑆𝐿−𝑃𝑟𝑜𝑐𝑒𝑠𝑠 𝑀𝑒𝑎𝑛|=
3∗2,08611634
|𝑋−5,72|= 0,8596633269 (B.95)
This P/T ratio of 85.97% means that the measurement system was not so good. That the two
different estimates of the repeatability standard deviation agreed well was a good sign, where
Minitab’s result was 87.51%. It was interesting to speculate about because the results seemed
more credible when using Minitab analysis of variance with ANOVA where the repeatability
standard deviation is based on the basis of the method of Mean Square, i.e., the software takes
the sum of squares divided by the number of degrees of freedom (the number of observations
minus one).
To relate to the proportion of the variability that exists, the product’s equation B.96 was used
where the variability of the measurement system,𝜎𝐺𝑎𝑢𝑔𝑒2 , had already been calculated in
equation B.93 and the total standard deviation, 𝜎𝑇𝑜𝑡𝑎𝑙 , was taken from Table A.1.
𝜎𝑇𝑜𝑡𝑎𝑙 2 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2 + 𝜎𝐺𝑎𝑢𝑔𝑒2 (B.96)
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡2 = 𝜎𝑇𝑜𝑡𝑎𝑙
2 − 𝜎𝐺𝑎𝑢𝑔𝑒2
𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡2 = 2,1575
2 − 4,3519 = 0,3028124964
If the total variability of the process, 𝜎𝑇𝑜𝑡𝑎𝑙2 , is 100%, the variability of the products, 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2 ,
could be 90% of the total variance for a capable measurement system, and the variability of the
measurement system, 𝜎𝐺𝑎𝑢𝑔𝑒2 , could be 10% of the total variability for a capable measurement
system. To investigate this, equations B.97 and B.98 were used.
27
𝜌𝑃 = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.97)
𝜌𝑃 =0,3028
4,6547= 0,0650537273
𝜌𝑀 = 𝜎𝐺𝑎𝑢𝑔𝑒
2
𝜎𝑇𝑜𝑡𝑎𝑙2 (B.98)
𝜌𝑀 = 4,3519
4,6547= 0,9349221319
The results show that the CA 50 motor as a product stands for only 6.51% of the total variability
in the process and the measurement system for 93.49% of the total variability in the process.
1
Appendix C – Hypothesis tests from Minitab
Cleanliness 4μm
2
Cleanliness 6μm
3
4
Cleanliness 14μm
5
1
Appendix D – Control charts This appendix analyzes the three parameters high pressure, external leaks, and cleanliness. The
underlying data for the analysis are presented in Table A1.
High pressure P3 (bar)
First, a normal distribution test was retrieved from Minitab. In the software, the Anderson-
Darling was used for a normal distribution test. The test compared the empirical cumulative
distribution function (ECDF) of the sample data with the distribution expected if the data were
normal. If the observed difference were adequately large, we rejected the null hypothesis of
population normality. Figure D.1 displays a normal distribution for the parameter high pressure.
Figure D.1 Normal distribution for parameter high pressure
The normal distribution test can be seen in Figure D.2 where Minitab has calculated p-value of
the data to 0.735, which exceeds 0.05 and thus shows normal contributed data. The values lie
in a straight and fine line.
2
Figure D.2 Normality plot for parameter high pressure
In Figure D.3, an I-MR chart is presented for individual measurements where the process does
not show anything special, however, motor 27 in the MR chart constitutes an alarm where the
value is more than three standard deviations from the centerline. After a check of the
corresponding test protocol, it was concluded that there was no special reason for the alarm.
Two similar values assume an additional low pressure, but do not constitute any alarm in the
MR-chart. There are still only 50 measurements that this is based on, so it might as well be
random. Therefore, the process is considered stable.
Figure D.3 I-MR chart for parameter high pressure
3
External leaks QY (l/min)
The graph in Figure D.4 does not show a normal distribution.
Figure D.4 Normal distribution for parameter external leaks
In Figure D.5, a low p-value of 0.016 is obtained, which is less than 0.05, indicating that the
data cannot be regarded as normally distributed. The low p-value is the result of measurements
of equal size, with a relatively low resolution of one decimal accuracy on the measurement of
the external leaks.
Figure D.5 Normality plot for parameter external leaks
A transformation was tested, resulting in data becoming normally distributed with a new p-
value of 0.115 (Figure D.6).
4
Figure D.6 Normality plot after transformation for parameter external leaks
The new look of the normal distribution can be seen in the Figure D.7 below.
Figure D.7 Normal distribution for parameter external leaks after transformation
In Figure D.8 below, an I-MR chart for parameter external leaks is presented. The diagram
shows three alarms on measurement 31, 32, and 39 because the measured values are outside the
limit of three standard deviations. In addition, measurements 9, 10, 33, and 34 also made alarms
according to the Western Electric rule that 8 points fall on the same side of the centerline. The
chart also exhibits an appearance that arouses suspicion that the data contains autocorrelation.
From the first measurement to measurement 36, the look of the values appears to follow a
pattern and then becomes wobblier. It should be examined whether anything was done that
made the measurements rougher after motor 36.
The MR-chart shows a funnel pattern that indicates that the variability between the motors was
higher at around observation 31 and forward, which means that value 40 and 41 constitute
alarms. Based on these discoveries, the process cannot be considered stable or in statistical
control.
5
Figure D.8 I-MR chart for parameter external leaks
The first of the 36 motors also raises suspicion that the data could possibly contain a certain
autocorrelation. After discussion with the supervisor and the Master Black Belt for the project,
the conclusion was made that this could also be due to randomness as the analysis is based on
only 50 measurements. It was therefore concluded that the suspicion surrounding the
autocorrelation could be ignored.
Cleanliness (4μm)
In Figure D.9, the data are normally distributed.
6
Figure D.9 Normal distribution for parameter cleanliness 4μm
According to the normality plot in Figure D.10, the data seem to be normally distributed, but
according to the p-value < 0.005 in this case, it is not. The reason this p-value is so low is that
several measurements were supposed to have the same value. This has to do with the resolution
of the data when the parameter cleanliness is measured by only one decimal accuracy. Thus the
analysis was continued under the assumption that the data were normally distributed.
Figure D.10 Normality plot for parameter cleanliness 4μm
Figure D.11 reveals an alarm for measurement 16, where more than 8 points ended up on the
same side of the centerline. One more alarm occurred also for measurement 48, where the value
dropped below the lower control limit. The process can therefore not be considered to be stable
for cleanliness 4μm.
7
Figure D.11 I-MR chart for parameter cleanliness 4μm
Cleanliness (6μm)
In Figure D.12, it is possible to see that the data do not appear to be normally distributed, but
instead a look more like a Poisson distribution.
Figure D.12 Normal distribution for parameter cleanliness 6μm
8
Here it did not help with a transformation of the data. It is not fair to assume a normally
distributed data for this parameter because first values exist with the same size because of the
presented ISO classifications with it associated particle interval and not the specific number of
particles that occur. Therefore, the assumption of a normal distribution should not be taken too
seriously for parameter cleanliness 6μm. The p-value shows similar results as the particle size
4μm, i.e., on a non-normal distribution where the overlying points of the measurements are not
in a good line and not expected to come from a normal distribution, but instead have more of a
meandering appearance.
Figure D.13 Normality plot for parameter cleanliness 6μm
In Figure D.14, an alarm can be seen for measurement 13 where the point falls outside the
control limits. There were also two situations where at least 8 points ended up on the same side
of the centerline. The process can therefore not be considered stable for cleanliness 6μm.
9
Figure D.14 I-MR chart for parameter cleanliness 6μm
Cleanliness (14μm)
According to Figure D.15 the data show a normal distribution.
Figure D.15 Normal distribution for parameter cleanliness 14μm
According to the normal plot in Figure D.16, the data appear to be reasonably normally
distributed, but according to the p-value (< 0.005), data are not normally distributed. The reason
10
this p-value is so low is, as previously mentioned, that several measurements assume to have
the same value and because of the resolution of the data for parameter cleanliness. The analysis
continues with the premise that the data are normally distributed.
Figure D.16 Normality plot for parameter cleanliness 14μm
Figure D.17 shows an initial alarm for measurement 1 and measurement 3, where the points
themselves are outside the lower and upper control limits. Two similar situations also arise for
measurement 19 and measurement 38, which falls below the lower control limit in the MR-
chart, creating suspicion that this is a pattern. This should be checked. The process cannot, from
this result, be considered to be stable for cleanliness 14 μm.
11
Figure D.17 I-MR chart for parameter cleanliness 14μm
Total Cleanliness
A completion in the form of a normal plot for overall cleanliness, which can be seen in Figure
D.18, was chosen. The results show non-normal distributed data with a p-value of 0.025.
12
Figure D.18 Normality plot for total cleanliness
Figure D.19 shows an alarm outside the upper control limit for measurement 3 and alarm
according to Western Electric rules that 8 or more subsequent points end up on the same side
of the centerline. Therefore, the process cannot be considered stable in terms of overall
cleanliness.
Figure D.19 I-MR chart for total cleanliness
1
Appendix E – Capability study In this Appendix a capability study is done where the standard deviation coming from the
products, σ = 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛 = √𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛2 , have been taken from the equations B.14, B.30,
B.52, B.74, and B.96 in Appendix A, and the mean for the parameter, µ = ��, and its process
standard deviation, σ, are taken from the calculation made in Table A.1 in Appendix A.
High pressure P3 (bar)
𝐶𝑝 𝑎𝑛𝑑 𝐶𝑝𝑘 for the control system in TB 1 as a production parameter according to parameter
high pressure is calculated using equations 3.6 and 3.7.
𝐶𝑝 = 𝑇𝑢− 𝑇𝑙
6𝜎 (E.1)
𝐶𝑝 = 𝑋𝑋𝑋− 𝑋𝑋𝑋
6∗0,5238303513= 1,297783387
𝐶𝑝𝑘 = 𝑚𝑖𝑛 (𝑇𝑢− 𝜇
3𝜎,
𝜇− 𝑇𝑙
3𝜎 ) (E.2)
𝐶𝑝𝑘 = 𝑚𝑖𝑛 (𝑋𝑋𝑋−174,96
3∗0,51369641,
174,96− 𝑋𝑋𝑋
3∗0,51369641) (E.3)
𝐶𝑝𝑘 =1,323739054
This is complemented with a capability analysis over the 50 collected motor tests with their
total standard deviation in Table A.1 and can be seen as the process capability.
𝐶𝑝 = 𝑋𝑋𝑋− 𝑋𝑋𝑋
6∗0,558057857= 1,194619264 (E.4)
𝐶𝑝𝑘 = 𝑚𝑖𝑛 (𝑋𝑋𝑋−174,96
3∗0,558057857,
174,96− 𝑋𝑋𝑋
3∗0,558057857) (E.5)
𝐶𝑝𝑘 = 1,170726879
If a conclusion is made from the results of the capability index when the production standard
deviation is used, it could be concluded that the control system that processes around XXX bar
2
in pressure during motor tests is not capable when both its 𝐶𝑝 = 1.2978 𝑎𝑛𝑑 𝐶𝑝𝑘 = 1.3237 are
under the requirement of 1.33. The process capability does not show capable results either when
𝐶𝑝= 1.19 or 𝐶𝑝𝑘 = 1.17 (< 1.33). This was chosen to be compared with a capability analysis in
Minitab. Figure E.1 shows similar results with 𝐶𝑝 = 1.19 𝑎𝑛𝑑 𝐶𝑝𝑘 =
1.17 𝑤ℎ𝑒𝑟𝑒 𝑛𝑜𝑛𝑒 𝑜𝑓 𝑡ℎ𝑒𝑚 𝑓𝑢𝑙𝑙𝑓𝑖𝑙𝑙 𝑡ℎ𝑒 𝑟𝑒𝑞𝑢𝑖𝑟𝑒𝑚𝑒𝑛𝑡 𝑜𝑓 ≥ 1.33.
Figure E.1 Capability Analysis for parameter high pressure
The confidence interval for the TB control system for the parameter high pressure and the
process for the parameter high pressure is calculated using equation E.6 and E.7 where 𝑍𝛼/2 =
1.96 is retrieved from a table for α = 0.05 in Vännman (2002) and 𝐶𝑝𝑘 is the previous
calculated value in equation E.3 where the result was based on 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛 and E.5 where the
total standard deviation for the process was used.
𝐶𝑝𝑘 [1 − 𝑍𝛼/2√1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] < 𝐶𝑝𝑘 < 𝐶𝑝𝑘 [1 + 𝑍𝛼/2√
1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] (E.6)
1,32 [1 − 1,96√1
9 ∗ 50 ∗ 1,322+
1
2(50 − 1)] < 𝐶𝑝𝑘
3
< 1,32 [1 + 1,96√1
9 ∗ 50 ∗ 1,322+
1
2(50 − 1)]
[1,043] < 𝐶𝑝𝑘 < [1,597]
1,17 [1 − 1,96√1
9 ∗ 50 ∗ 1,172+
1
2(50 − 1)] < 𝐶𝑝𝑘
< 1,17 [1 + 1,96√1
9∗50∗1,172+
1
2(50−1)] (E.7)
[0,9211] < 𝐶𝑝𝑘 < [1,4203]
For the result to be considered excellent, the lower tolerance limits of these intervals should
preferably be ≥ 1.33, but in this case the results can still be considered decent for the control
system in which the lower limit is ≥ 1.0, which means that virtually the entire distribution will
have a distance from the tolerance limits and regarding to the point estimation of 1.32374 (≥
1.043).
External leaks QY (l/min)
This parameter has one-sided tolerance limit, which 𝐶𝑝 and 𝐶𝑝𝑘 for CA 50 as a product in
terms of the parameter external leaks is calculated using equations E.8 and E.9.
𝐶𝑝𝑘 = 𝐶𝑝𝑢 = 𝑇𝑢− µ
3𝜎 (E.8)
𝐶𝑝𝑢 = 𝑋−0,76
3∗0,5032647902 = 3,470671306 (E.9)
This is complemented with a capability analysis of the 50 collected motor tests and their total
standard deviation from Table A.1 and can thus be seen as the process capability.
4
𝐶𝑝𝑢 = 𝑋−1,366
3∗0,513733831 = 3,006745076 (E.10)
According to the results, both the CA 50 motor as a product and the process are capable when
both their values of 𝐶𝑝𝑢> 1.33. The complemented results in Minitab in Figure E.2 with 𝐶𝑝𝑘=
3.03 also meets the requirement of ≥ 1.33.
Figure E.2 Capability Analysis for parameter external leaks
The confidence interval for the product CA 50 and the capability index for the process, 𝐶𝑝𝑘,
according the parameter external leaks are calculated through equations E.11 and E.12, where
𝑍𝛼/2 = 1.96 is retrieved from a table for α = 0.05 in Vännman (2002) and 𝐶𝑝𝑘 are the previous
calculated values in equation E.9, where the result was based on 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛, and E.10, where
the total standard deviation for the process was used.
𝐶𝑝𝑘 [1 − 𝑍𝛼/2√1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] < 𝐶𝑝𝑘 < 𝐶𝑝𝑘 [1 + 𝑍𝛼/2√
1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] (E.11)
3,47 [1 − 1,96√1
9 ∗ 50 ∗ 3,472+
1
2(50 − 1)] < 𝐶𝑝𝑘
5
< 3,47 [1 + 1,96√1
9 ∗ 50 ∗ 3,472+
1
2(50 − 1)]
[2,777] < 𝐶𝑝𝑘 < [4,164]
3,01 [1 − 1,96√1
9 ∗ 50 ∗ 3,012+
1
2(50 − 1)] < 𝐶𝑝𝑘
< 3,01 [1 + 1,96√1
9∗50∗3,012+
1
2(50−1)] (E.12)
[2,4043] < 𝐶𝑝𝑘 < [3,6092]
The results show a very good capability for the parameter external leaks with today's tolerance
limit of X liters per minute in terms of both product variability and the whole process variability
when 𝐶𝑝𝑘 and the lower limit of the confidence intervals are ≥ 1.33.
Cleanliness (4μm)
This parameter has one-sided tolerance limit, which 𝐶𝑝𝑘 = 𝐶𝑝𝑢 for CA 50 as a product and
𝐶𝑝𝑘 = 𝐶𝑝𝑢 for the process in terms of parameter cleanliness 4μm are calculated using equations
E.13 and E.14.
𝐶𝑝𝑢 = 𝑋−13,9
3∗0,9058280689 = 1,508748419 (E.13)
𝐶𝑝𝑢 = 𝑋−13,9
3∗1,249489692 = 1,093779865 (E.14)
According to the results, the capability for the CA 50 as a product is good in terms of product
variability when 𝐶𝑝𝑢 = 1.51 (> 1.33), but if the capability is based on the process variability,
we get a result that shows a non-capable process when 𝐶𝑝𝑢 = 1.09 (< 1.33). The completed
calculation in Minitab, which can be seen in Figure E.3, also shows the process capability as
𝑃𝑝𝑘 = 1.09 (< 1.33). Here the value of Ppk is compared when it shows the long-term variability
and is thus the variability that the customer is going to experience.
6
Figure E.3 Capability Analysis for parameter cleanliness 4μm
The confidence interval for the product CA 50 and the capability index for the process, 𝐶𝑝𝑘,
according the parameter cleanliness 4μm are calculated using equations E.15 and E.16 where
𝑍𝛼/2 = 1.96 is retrieved from a table for α = 0.05 in Vännman (2002) and 𝐶𝑝𝑘 are the previous
calculated values in equation E.13, where the result was based on 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛, and E.14, where
the total standard deviation for the process was used.
𝐶𝑝𝑘 [1 − 𝑍𝛼/2√1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] < 𝐶𝑝𝑘 < 𝐶𝑝𝑘 [1 + 𝑍𝛼/2√
1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] (E.15)
1,51 [1 − 1,96√1
9 ∗ 50 ∗ 1,512+
1
2(50 − 1)] < 𝐶𝑝𝑘
< 1,51 [1 + 1,96√1
9 ∗ 50 ∗ 1,512+
1
2(50 − 1)]
[1,1960] < 𝐶𝑝𝑘 < [1,8215]
7
1,09 [1 − 1,96√1
9 ∗ 50 ∗ 1,092+
1
2(50 − 1)] < 𝐶𝑝𝑘
< 1,09 [1 + 1,96√1
9∗50∗1,092+
1
2(50−1)] (E.16)
[0,8583] < 𝐶𝑝𝑘 < [1,3292]
Cleanliness (6μm)
This parameter has one-sided tolerance limit, which 𝐶𝑝𝑘 = 𝐶𝑝𝑢 for CA 50 as a product and
𝐶𝑝𝑘 = 𝐶𝑝𝑢 for the process in terms of parameter cleanliness 6μm are calculated using equations
E.17 and E.18.
𝐶𝑝𝑢 = 𝑋−11,2
3∗0,3742159228 = 4,275606415 (E.17)
𝐶𝑝𝑢 = 𝑋−11,2
3∗1,603567451 = 0,9977753034 (E.18)
According to the results, the capability of the CA 50 as a product is very good in terms of the
product variability when 𝐶𝑝𝑢 = 4.26 (>1.33), but if the capability is based on the process
variability, we get a result that shows a non-capable process when Cpu = 0.998 (< 1.33). The
completed calculation in Minitab, which can be seen in Figure E.4, also shows similar results,
i.e., the process capability, 𝑃𝑝𝑘 = 1.00 (< 1.33). Here the value of Ppk is compared when it
shows the long-term variability and is thus the variability that the customer is going to
experience.
8
Figure E.4 Capability Analysis for parameter cleanliness 6μm
The confidence interval for the product CA 50 and the capability index for the process, 𝐶𝑝𝑘,
according the parameter cleanliness 6μm, are calculated using equations E.18 and E.19, where
𝑍𝛼/2 = 1.96 is retrieved from a table for α = 0.05 in Vännman (2002) and 𝐶𝑝𝑘 are the previous
calculated values in equation E.17, where the result was based on 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛, and E.18, where
the total standard deviation for the process was used.
𝐶𝑝𝑘 [1 − 𝑍𝛼/2√1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] < 𝐶𝑝𝑘 < 𝐶𝑝𝑘 [1 + 𝑍𝛼/2√
1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] (E.19)
4,28 [1 − 1,96√1
9 ∗ 50 ∗ 4,262+
1
2(50 − 1)] < 𝐶𝑝𝑘
< 4,28 [1 + 1,96√1
9 ∗ 50 ∗ 4,262+
1
2(50 − 1)]
[3,4241] < 𝐶𝑝𝑘 < [5,1272]
9
0,998 [1 − 1,96√1
9 ∗ 50 ∗ 0,9982+
1
2(50 − 1)] < 𝐶𝑝𝑘
< 0,998 [1 + 1,96√1
9∗50∗0,9982+
1
2(50−1)] (E.20)
[0,7797] < 𝐶𝑝𝑘 < [1,2159]
Cleanliness (14μm)
This parameter has one-sided tolerance limit, which 𝐶𝑝𝑘 = 𝐶𝑝𝑢 for CA 50 as a product and
𝐶𝑝𝑘 = 𝐶𝑝𝑢 for the process in terms of parameter cleanliness 14μm are calculated using
equations E.21 and E.22.
𝐶𝑝𝑢 = 𝑋−5,72
3∗0,5502726597 = 4,409935009 (E.21)
𝐶𝑝𝑢 = 𝑋−5,72
3∗2,157473958 = 1,124772171 (E.22)
According to the results, the capability of the CA 50 as a product is good in terms of the product
variability when 𝐶𝑝𝑢 = 1.40 (> 1.33), but if the capability is based on the process variability,
we get a result that shows a non-capable process when Cpu = 1.12 (< 1.33). The completed
calculation in Minitab, which can be seen in Figure E.5, also shows similar results, i.e., the
process capability, 𝑃𝑝𝑘 = 1.12 (< 1.33). Here the value of Ppk is compared when it shows
the long-term variability and is thus the variability that the customer is going to experience.
10
Figure E.5 Capability Analysis for parameter cleanliness 14μm
The confidence interval for the product CA 50 and the capability index for the process, 𝐶𝑝𝑘,
according the parameter cleanliness 14μm are calculated using equations E.22 and E.23, where
𝑍𝛼/2 = 1.96 is retrieved from a table for α = 0.05 in Vännman (2002) and 𝐶𝑝𝑘 is the previous
calculated value in equation E.21, where the result was based on 𝜎𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑖𝑜𝑛, and E.22, where
the total standard deviation for the process was used.
𝐶𝑝𝑘 [1 − 𝑍𝛼/2√1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] < 𝐶𝑝𝑘 < 𝐶𝑝𝑘 [1 + 𝑍𝛼/2√
1
9𝑛𝐶𝑝𝑘2
+ 1
2(𝑛−1)] (E.23)
4,41 [1 − 1,96√1
9 ∗ 50 ∗ 1,432+
1
2(50 − 1)] < 𝐶𝑝𝑘
< 4,41 [1 + 1,96√1
9 ∗ 50 ∗ 1,432+
1
2(50 − 1)]
[3,531] < 𝐶𝑝𝑘 < [5,288]
11
1,125 [1 − 1,96√1
9 ∗ 50 ∗ 1,1252+
1
2(50 − 1)] < 𝐶𝑝𝑘
< 1,125 [1 + 1,96√1
9∗50∗1,1252+
1
2(50−1)] (E.24)
[0,8836] < 𝐶𝑝𝑘 < [1,3659]
1
Appendix F Semi-structured interviews These semi-structured interviews with operators and a production engineer were carried out so
that random operators and a production engineer were interviewed on certain day and a
questionnaire was used during the interviews. The questions for the workers are listed below.
(1) If you could control, what would you have done concerning the improvement of the
TB?
(2) How can you as a worker affect the instructions according to the test TB?
(3) What can you as worker do wrong in the operation?
(4) What could be the consequences of a TB for the CA motor if the TB would not deliver
true results?
The answers to the questions that were asked were largely of a similar nature. The responses to
the questions asked are briefly described below.
(1) TB facilities for the CA motor are in the current situation in an opposite direction
which makes it easier for them to fail. There are currently three travers lifters where
the risk is that they can impact with each other. Better lift table for the CA motor.
(2) The same motor can produce inaccurate test values if the test protocols have
accidentally been wrong.
(3) If the instruction that exists is wrong, we will make mistakes. For example, if a cam
roller is forgotten under the assembly, it will lead to major breakdowns, where the
cam ring needs to be discarded, and thus pieces will come off from the piston and
further damaging the rest of the motor.
(4) The consequences could be that motors will be sent back as returns where a
demolition of the motors will be done to explore the "error" that does not actually
exist. Designers and other staff will in turn be involved, which will increase costs.
A possible disposal of the motor may occur. It could therefore depend on many
factors.
1
Appendix G – Integrated Contamination Monitoring System
(CMS)
1
Appendix G – ISO Cleanliness Code ISO4406-1999
The ISO cleanliness code is used to quantify particulate contamination levels per milliliter of
fluid for three sizes: 4µm, 6µm, and 14µm. The ISO code is expressed in three numbers (e.g.,
X/X/X). Each number represents a contaminant level code for the correlating particle size. The
code includes all particles of the specified size and larger. It is important to note that each time
a code increases the quantity range of particles doubles.
Below is an introduction of ISO 4406:1999. It shows the code number associated with the
contamination level.