statistics in validation sampling plans - · pdf file10 what is acceptance sampling? process...
TRANSCRIPT
Statistics in Validation
Sampling Plans that Result in Statistical Confidence
Raul Soto, MSc, CQEIVT Conference - October 2016
San Diego, CA
(c) 2016 Raul Soto 2
The contents of this presentation represent the opinion of the
speaker; and not necessarily that of his present or past employers.
IVT San Diego OCT 2016
About the Author• 20+ years of experience in the medical devices, pharmaceutical, biotechnology, and consumer electronics industries
• MS Biotechnology, emphasis in Biomedical Engineering
• BS Mechanical Engineering
• ASQ Certified Quality Engineer (CQE)
• Led validation / qualification efforts in multiple scenarios:
• High-speed, high-volume automated manufacturing and packaging equipment; machine vision systems
• Manufacturing Execution Systems (MES)
• Enterprise resource planning applications (i.e. SAP)
• IT network infrastructure, reporting systems, Enterprise applications
• Laboratory information systems and instruments
• Mobile apps
• Product improvements, material changes, vendor changes
• Contact information:
• Raul Soto [email protected]
3
4
• Present the fundamental concepts of Acceptance Sampling
• OC Curves• Quality Indices: AQL and LQ (LTPD)
• Sampling by Attributes• Single and Double plans
• Sampling by Variables
• Introduction to ASQ z1.4 and z1.9
What this talk is about
(c) 2016 Raul SotoIVT San Diego OCT 2016
Statistically Valid Sampling Plans
1. Select the appropriate AQL and LQ (LTPD)
• Based on these, select a sampling plan that provides the required protection to the patient
5(c) 2016 Raul SotoIVT San Diego OCT 2016
Statistically Valid Sampling Plans
2. KNOW what your sampling plan routinely accepts and rejects
• The protection provided by your sampling plan is defined by what it routinely accepts (defined by AQL) and what it routinely rejects (defined by LQ).
• KNOW the AQL and LQ for ALL your sampling plans
• YES, that includes the ones you get from ASQ z1.4 and z1.9
6(c) 2016 Raul SotoIVT San Diego OCT 2016
Statistically Valid Sampling Plans
3.RANDOMIZE
• Units selected for inspection must be selected at random.
• Every single unit in your lot must have the SAME probability to be selected as a sample.
• If this condition is not met, your sample will be biased and your sampling decisions will be WRONG.
7(c) 2016 Raul SotoIVT San Diego OCT 2016
PART 1
FUNDAMENTALS
WHAT IS SAMPLING /WHEN TO USE IT / OC CURVES / AQLS AND LQS
9
Fundamentals
• What is Acceptance Sampling?
• When to use it
• Risks of sampling
• OC Curves
• Quality Indices : what they really tell us
•Attributes vs Variables Sampling Plans
(c) 2016 Raul SotoIVT San Diego OCT 2016
10
What is Acceptance Sampling?
Process of evaluating a portion of the product in a lot/batch with the purpose of accepting or rejecting the entire lot/batch.
• Main advantage of acceptance sampling : economics.
• Despite the cost of designing and administering the plans, an overall cost reduction results from not having to inspect the whole lot.
• Sampling helps us to :
• Improve the quality level of the product going out to the customer• Protect us against the release of highly defective lots
(c) 2016 Raul SotoIVT San Diego OCT 2016
11
Alternatives for Product Inspection
The four (4) main alternatives for product inspection are:
1. No inspection• PAT, Parametric Release
2. Small samples• Stable processes, homogeneous product
3. Large samples• Validation, new products, processes with large variation
4. 100% Inspection• When there is a cost-effective alternative (i.e. automation), or when sampling
determines a very high % defective
(c) 2016 Raul SotoIVT San Diego OCT 2016
12
Limitations
• GOOD product may be REJECTED, and BAD product may be ACCEPTED.
• Acceptance sampling does NOT improve the quality of our manufacturing processes.
• Acceptance sampling does NOT provide an accurate estimate of lot quality. It determines an acceptance and rejection decision for each lot.
• The only way to provide 100% good product is to make 100% good product.
“You cannot inspect quality into a product, quality must be built into a product” - Deming
(c) 2016 Raul SotoIVT San Diego OCT 2016
13
Sampling Risks
• Neither statistical sampling, nor 100% inspection, can guarantee that you will find every defect on a lot.
• Sampling risks can be defined as:
• The risk of rejecting a good lot = alpha (a) risk / error (Producer’s Risk)• Represented by AQL
• The risk of accepting a bad lot = beta (b) risk / error (Consumer’s Risk)• Represented by LQ / RQ / LTPD
(c) 2016 Raul SotoIVT San Diego OCT 2016
AQL: Acceptable Quality Limit
• An attributes sampling plan (n = 82, a = 2) with AQL = 0.01 (1%) is used to inspect the following lots:
Accept or Reject?
• Lot A: 0.9 % defective _________________
• Lot B: 1.0 % defective _________________
• Lot C: 1.1 % defective _________________
• Lot D: 2.0 % defective _________________
• Lot E: 3.2 % defective _________________
• Lot F: 5.0 % defective _________________
• Which lots will be accepted, and which will be rejected?
14(c) 2016 Raul SotoIVT San Diego OCT 2016
AQL: Acceptable Quality Limit
LOT Probability of Acceptance (for n = 82, a = 2)
• Lot A: 0.9 % defective 96.2 %
• Lot B: 1.0 % defective 95.1 %
• Lot C: 1.1 % defective 93.8 %
• Lot D: 2.0 % defective 77.4 %
• Lot E: 3.2 % defective 51.0 %
• Lot F: 5.0 % defective 21.8%
15(c) 2016 Raul SotoIVT San Diego OCT 2016
AQL: Acceptable Quality Limit
An AQL of 1% means that a lot that is 1% defective will have a:
95% probability or better of being accepted… and
5% probability or lower of being rejected.
16(c) 2016 Raul SotoIVT San Diego OCT 2016
17
What the Quality Indices tell us
• Some people believe that lots with % defective < AQL will always be rejected. This is NOT correct.
• Lots with a % defective equal or lower than the AQL have a probability of acceptance of 95% or better.
• Lots with a % defective equal or higher than the LQ10 have a probability of acceptance of 10% or less.
• Lots with a % defective between the AQL and the LQ10 have a probability of acceptance between 10 – 95%.
• Recommendation: Generate sampling plans based on AQL and LQ
(c) 2016 Raul SotoIVT San Diego OCT 2016
18
Quality Indices for Acceptance Sampling Plans
• AQL (Acceptable Quality Limit): • Usually defined as the p (percent defective)
with a 95% Pa (Probability of acceptance).• Focus of AQL is to protect the PRODUCER
from rejecting GOOD lots
• LQ (Limiting Quality)• Usually defined as the p with a 5% Pa (LQ5) or
a 10% Pa (LQ10)• Focus of LQ is to protect the CONSUMER from
accepting BAD lots• Also called Lot Tolerance Percent Defective
(LTPD), Rejectable Quality Level (RQL).
• IQL (Indifference Quality Limit): • The p with a 50% Pa. • Lots with p = IQL have the same probability of
being accepted and being rejected.• This is your sampling plan’s blind spot.
• AOQL (Average Outgoing Quality Limit): • The maximum % defective possible in the
outgoing material for this sampling plan.
(c) 2016 Raul SotoIVT San Diego OCT 2016
19
The Operating Characteristic CurvePa
(p
rob
abili
ty o
f ac
cep
tan
ce)
p (% defective)0 1
1• The OC Curve for a sampling plan
quantifies the sampling risk graphically, by plotting the Probability of Acceptance (Pa, or P) of a lot as a function of the Lot % Defective (p).
• When the percent defective (p) is high, we want the probability of acceptance (Pa) to be low
• When the percent defective (p) is low, we want the probability of acceptance (Pa) to be high
0.95
0.10
(c) 2016 Raul SotoIVT San Diego OCT 2016
20(c) 2016 Raul SotoIVT San Diego OCT 2016
21
Two (2) Types of OC Curves
• Type A:• Assumes that samples are taken from an isolated lot of finite size, where the effect of removing a
sample is significant. (Lot size / sample size < 10)
• This sampling process is modeled by the Hypergeometric distribution.
• Type B: • Assumes that the sample came from a large lot, where the effect of removing a sample has negligible
effect. (Lot size / sample size > 10)
• This sampling process is modeled by the Binomial distribution (for attributes).
(c) 2016 Raul SotoIVT San Diego OCT 2016
Why is AQL not enough?
• In order to understand the level of protection provided by a sampling plan, you should know what the plan routinely accepts (AQL), and what it routinely rejects (LQ).
• AQL quantifies the producer’s risk for your sampling plan.
• A low AQL helps ensure you do not reject good product.
• LQ quantifies the consumer’s risk for your sampling plan.
• A low LQ helps ensure you do not accept bad product.
• To know this, you need to know the AQL, the LQ, and the OC Curve.
22(c) 2016 Raul SotoIVT San Diego OCT 2016
Why is AQL not enough?• These 4 sampling plans have
AQL ≈ 1% but different LQ10.
• They provide equivalent protection to the producer, but different levels of protection to the consumer.
23
Plan AQL IQ LQ10
n = 132, a = 3 1.0% 2.8% 5.01%
n = 65, a = 2 1.3% 4.2% 8.1%
n = 52, a = 2 1.6% 5.1% 9.9%
n = 25, a = 1 1.4% 6.6% 14.7%
(c) 2016 Raul SotoIVT San Diego OCT 2016
AQL and LQ for Validations
• Depends on the claim we want to be able to make:
• Claim: “At 95% confidence, the % defective for individual lots is less or equal than 1%”
=> design a plan with LQ5 ≤ 1%
• Claim: “At 90% confidence, the % defective for individual lots is less or equal than 5%”
=> design a plan with LQ10 ≤ 5%
• LQs for validating new processes or process changes should be smaller than for regular manufacturing with mature processes.
• Depends on severity of a failure: critical / major / minor / cosmetic
• Your company’s Quality Engineering standards should have target quality levels based on severity.
• Main driver should always be the protection of patient safety.
24(c) 2016 Raul SotoIVT San Diego OCT 2016
25
Attributes vs Variables Data
• Attributes data
• Quality characteristics that are expressed on a go / no-go basis• Qualitative and discrete• Pass / Fail• Examples: seal appearance, colors, presence or absence
• Variables data
• Quality characteristics that are measured in a numerical basis• Quantitative and continuous• Examples: weight, diameter, length, thickness, hardness, seal strength, concentration of solute
(c) 2016 Raul SotoIVT San Diego OCT 2016
26
Attributes vs Variables Sampling Plans
Attributes sampling plans
• Defined by:• a sample size n• an acceptance number a (or c)
• A random sample is taken from the lot• Each unit is inspected and classified as acceptable or defective• The number of defective units is then compared to the allowable number stated in the plan.• A decision is made of accepting or rejecting the lot• Modeled using the binomial or hypergeometric distribution
(c) 2016 Raul SotoIVT San Diego OCT 2016
27
Attributes vs Variables Sampling Plans
Variables sampling plans
• Defined by:• a sample size n• an critical distance constant k
• A sample is taken and a measurement of a specified characteristic is made on each unit.• The sample average, sample standard deviation, and k are calculated.• The appropriate formula is used to analyze the lot.• A decision is made of accepting or rejecting the lot.• Modeled using the normal distribution.
• Normality of the data must be confirmed before using a variables sampling plan
(c) 2016 Raul SotoIVT San Diego OCT 2016
PART 2
ATTRIBUTES SAMPLING PLANS
29
Attribute Sampling PlansSingle• lots inspected and decision to accept / reject is based on one sample• Simpler to design and implement• Larger sample sizes = less economically efficient
Double • lots inspected and decision to accept / reject is based on a maximum of two samples• More complex to design and implement• Smaller sample sizes in the long run = more economically efficient
Multiple• decision to accept / reject is based on maximum of seven samples (out of scope for this talk)• Most complex to design and implement• Fewer sampling, most economically efficient• Looks too much like “sampling until you pass”
(c) 2016 Raul SotoIVT San Diego OCT 2016
30
Attribute Sampling Plans
• Defined by sample size (n) and acceptance number (a)
• Example:
N = lot size = 10,000 unitsn = sample size = 89 unitsa = acceptance number = 2 units
• This means that we will take a random sample of 89 units from the lot, and inspect them; if the number of observed defectives is less than or equal to 2, the lot will be accepted.
• In attributes sampling plans, each inspected unit is judged as either conforming or nonconforming.
(c) 2016 Raul SotoIVT San Diego OCT 2016
31
Attribute Sampling Plans
• The OC Curve allows us to see the level of protection provided by a sampling plan
• In this case, since N/n >10, we can use the binomial distribution to construct the OC Curve
• For example, for p = 0.01 (1% defective), n = 89, a = 2
• In Excel: BINOM.DIST(2,89,0.01,TRUE) = 0.93969
• If we calculate Pa for multiple values of p, we obtain the OC Curve:
9397.0)99.0()01.0(!87!*2
!89)99.0()01.0(
!88!*1
!89
)99.0()01.0(!89!*0
!89}2{
872881
890
xPPa
(c) 2016 Raul SotoIVT San Diego OCT 2016
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 2 4 6 8 10 12 14 16 18
Pa
percent defective
OC Curve
32
% Defective po
Probability of Acceptance
Pa
0.005 0.9897
0.010 0.9397
0.020 0.7366
0.030 0.4985
0.040 0.3042
0.050 0.1721
0.060 0.0919
0.070 0.0468
0.080 0.0230
0.090 0.0109
AQL LQ10
0.95
0.10
0.5
(c) 2016 Raul SotoIVT San Diego OCT 2016
33
AQL and LQ
• AQL is the p (% defective) that corresponds to a Pa of 95%
• LQ10 is the p that corresponds to a Pa of 10%.
• For this attributes plan: AQL = 0.0092 ; LQ10 = 0.0598 ; IQ = 0.0300
• The protection of this plan (n = 89, a = 2) can be interpreted from the OC curve as follows:
• A lot with 0.92% defective has a 95% probability of acceptance (AQL)
• A lot with 3% defective has a 50% probability of acceptance (IQL)
• A lot with 5.98% defective has a 10% probability of acceptance (LQ10)
(c) 2016 Raul SotoIVT San Diego OCT 2016
34
Effects of N, n, a in the OC Curve
• Increasing the sample size n increases the precision of the plan, its ability to discriminate between good and bad lots. (Steeper OC curve)
• Decreasing the acceptance number a shifts the OC curve to the left, also increasing the protection of the plan.
• When a = 0, the OC curve is convex.
• The lot size does not have a significant effect on the OC curve unless the sample size is large (N/n < 10)
(c) 2016 Raul SotoIVT San Diego OCT 2016
35
Effect of sample size n
A lot that is 2% defective will have a probability of acceptance of (approximately):
85% with n = 3065% with n = 6040% with n = 100
(c) 2016 Raul SotoIVT San Diego OCT 2016
36
Effect of acceptance number
(c) 2016 Raul SotoIVT San Diego OCT 2016
37
Effect of lot size N
Assuming lots are homogeneous
(c) 2016 Raul SotoIVT San Diego OCT 2016
38
Effects of N, n, a in the OC Curve
• Sometimes the sample size is set as a fixed percentage of the lot size (for example, sampling 10% of every lot).
• Another common practice is to set the sample size as the square root of the lot size.
• The problem with these method is that they do not provide a consistent protection, since the sample size will vary as the lot size varies.
• More protection to large lots, less protection to small lots.
• You still need to know your plan’s AQLs and LTPDs. Remember, you should ALWAYS know what your sampling plans routinely accept and routinely reject.
(c) 2016 Raul SotoIVT San Diego OCT 2016
What if we use n = 10% of N?
39
N = 1000
N = 5000
N = 2000
Inconsistent protectionLarger lots get better protection than
smaller lots
(c) 2016 Raul SotoIVT San Diego OCT 2016
What if we use n = √N?
40
Inconsistent protectionLarger lots get better protection than
smaller lots
(c) 2016 Raul SotoIVT San Diego OCT 2016
41
AOQ and AOQL
• Average Outgoing Quality (AOQ) is the average quality level of all accepted lots under a specific sampling plan
• AOQ is what the client will receive in the long term
• AOQ calculation assumptions:• Rejected lots are 100% inspected• 100% inspection finds all defective units• Defective units are replaced or repaired• Lot size is large compared to sample size
• AOQ is defined as AOQ(p) = p * Pa(p)
• The maximum value of the AOQ is the Average Outgoing Quality Limit, AOQL
(c) 2016 Raul SotoIVT San Diego OCT 2016
42
AOQ and AOQL
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
0 1 2 3 4 5 6 7
Ave
rage
Ou
tgo
ing
Qu
alit
y A
OQ
percent defective p
AOQ Curve for n = 132, a = 3
AOQL = 1.47%• Assumptions
• 100% inspection of rejected lots
• All defective units found and discarded / repaired
• Large lot size (N > 10n)
(c) 2016 Raul SotoIVT San Diego OCT 2016
43
“Zero Defects” Sampling Plans
• Using an acceptance number (a = 0) on a sampling plan does NOT guarantee zero defects.
• An a = 0 sampling plan is NOT a tool to drive a process to zero defects; all you achieve is to reject more product.
• Zero defects can only be achieved by eliminating the problems that cause defects, NOT by sampling.
(c) 2016 Raul SotoIVT San Diego OCT 2016
44
“Zero Defects” Sampling Plans
• Alternatives:
• For critical defects with major impact to patient safety it is necessary to keep the lowest AQL possible. In this case, a large sample with an accept number of 0 is the most appropriate alternative.
• In these cases, we must be aware of the cost of implementing such a plan, vs the cost of releasing a critical defect to the market. For medical devices/ biopharma, the safety of the customer must be the most important consideration.
• For non-critical defects: Non-zero defects plan with equivalent LTPD, slightly higher AQL
(c) 2016 Raul SotoIVT San Diego OCT 2016
Zero vs Non-Zero Defects Sampling Plansn = 50 a = 0
Pa% Index p95 AQL 0.102534990 0.210496750 IQ 1.376728010 LTPD10 4.50074205 LTPD5 5.8155100
NEARLY EQUIVALENT TO
n = 85 a = 1
Pa% Index p95 AQL 0.419673190 0.627401650 IQ 1.966671010 LTPD10 4.49905805 LTPD5 5.4596020
45(c) 2016 Raul SotoIVT San Diego OCT 2016
Double Sampling Plans
• Lots inspected and decision to accept / reject is based on a maximum of twosamples
• For mature processes with low % defective, double sampling plans provide equivalent protection with smaller sample sizes.
PROS:
• Very good or very bad lots will be rejected with less sampling overall.
• More economically favorable if cost of sampling is an issue
• Equivalent levels of producer and consumer risks with significantly less samples in the long term
CONS:
• More complex to design and administer
46(c) 2016 Raul SotoIVT San Diego OCT 2016
47
Double Sampling Plans
Take n1 = 60
# defects
Accept Reject
Take n2 = 82
Total # defects
Accept Reject
0 ≥ 4
1 - 3
≥ 4≤ 3
n1 = 60, a1 = 0, r = 4n2 = 82, a2 = 3
Total # defects: add up defects found in both rounds of inspection
(c) 2016 Raul SotoIVT San Diego OCT 2016
48
Double Sampling Plans
• Example: Target AQL = 1%, Target LQ10 = 5%
• Single sampling plann = 132, a = 3
Pa (%) Index p (%)95 AQL 1.0490 1.3250 IQ 2.7710 LQ10 4.995 LQ5 5.77
AOQL = 1.47%
(c) 2016 Raul SotoIVT San Diego OCT 2016
49
Double Sampling Plans
• Equivalent: Target AQL = 1%, Target LQ10 = 5%
• Double sampling plann1 = 60, a1 = 0, r = 4n2 = 82, a2 = 3 a2 → total cumulative # defects
Pa (%) Index p (%)95 AQL 1.0090 1.2850 IQ 2.7010 LQ10 4.995 LQ5 5.85
AOQL = 1.42%
(c) 2016 Raul SotoIVT San Diego OCT 2016
Single vs Double Sampling Plan Comparison
50
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 1 2 3 4 5 6 7
Pro
bab
ility
of
Acc
ep
tan
ce (
Pa)
percent defective p
Single vs Double Sampling Plan
Pa (double)
Pa (single)
p Pa (double) Pa (single)
0 1 1
0.65 0.9871363 0.9888251
1.3 0.8952717 0.905678
1.95 0.7228923 0.7425666
2.6 0.5279285 0.5501373
3.25 0.3570842 0.3751051
3.9 0.2284081 0.2392301
4.55 0.1405285 0.1445002
5.2 0.08433088 0.08343898
5.85 0.04993715 0.04638324
6.5 0.02946017 0.02495489
Target AQL = 1%, LQ10 = 5%
• Single sampling plan• n = 132, a = 3
• Double sampling plan• n1 = 60, a1 = 0, r = 4• n2 = 82, a2 = 3
AQL
LQ10
(c) 2016 Raul SotoIVT San Diego OCT 2016
Using Minitab
• Stats / Quality Tools/ Acceptance Sampling by Attributes
51(c) 2016 Raul SotoIVT San Diego OCT 2016
52
Acceptance Sampling by Attributes
Measurement type: Go/no goLot quality in percent defectiveUse binomial distribution to calculate probability of acceptance
Acceptable Quality Level (AQL) 1Producer's Risk (Alpha) 0.05
Rejectable Quality Level (RQL or LTPD) 10Consumer's Risk (Beta) 0.1
Generated Plan(s)
Sample Size 52Acceptance Number 2
Accept lot if defective items in 52 sampled <= 2; Otherwise reject.
Percent Probability ProbabilityDefective Accepting Rejecting
1 0.985 0.01510 0.097 0.90350 0.000 1.000
Using Minitab
(c) 2016 Raul SotoIVT San Diego OCT 2016
53
MS EXCEL Functions for Attributes Sampling
BINOM.DIST (acceptance number, sample size, %defective, true) = returns the probability of acceptance PaTrue = cumulative functionFalse = probability mass function
BINOM.INV (sample size, AQL, alpha) = returns the minimum acceptance number for given AQL
BINOM.INV (sample size, LTPD, beta) = returns the minimum rejection number for given LTPD
POISSON.DIST (acceptance number, n * AQL, true) = Returns the probability of acceptance Pa
HYPGEOM.DIST (acceptance number, sample size, % defective, lot size) = Returns the probability of acceptance Pa
(c) 2016 Raul SotoIVT San Diego OCT 2016
PART 3
VARIABLES SAMPLING PLANS
55
Variables Sampling Plans
• Variables sampling plans are defined by two quantities: • the sample size n• the critical distance constant k
• Generally based on sample mean and sample standard deviation of the quality characteristic.
(c) 2016 Raul SotoIVT San Diego OCT 2016
56
Advantages and Disadvantages
• Advantages
• The same OC Curve can be obtained with a smaller sample size than with an attributes sampling plan. (The same protection with smaller sample sizes).
• Measurements data usually provide more information about the process than attributes data.
(c) 2016 Raul SotoIVT San Diego OCT 2016
57
Advantages and Disadvantages
• Disadvantages
• The distribution of the quality characteristic should be normal.
• Separate sampling plans must be used for each quality characteristic of interest.
• It is possible to reject a lot even though the actual sample does not contain any defective items.
(c) 2016 Raul SotoIVT San Diego OCT 2016
58
Types of Variable Sampling Plans
• Known vs Unknown Standard Deviation• Larger sample sizes are needed when the standard deviation is unknown.
• One or two specification limits• Some quality characteristics only have USL or LSL, not both• Example: sealing strength in some packaging
• All variables sampling plan are characterized by• Sampling size n• Critical distance constant k
(c) 2016 Raul SotoIVT San Diego OCT 2016
59
Process
Standard
Deviation
Spec Limits Acceptance Criteria
Known LSL LSL + k * s ≤ x̄
Known USL x̄ ≤ USL – k * s
Known USL and LSL LSL + k * s ≤ x̄ ≤ USL – k * s
Unknown LSL LSL + k * s ≤ x̄
Unknown USL x̄ ≤ USL – k * s
Unknown USL and LSL LSL + k * s ≤ x̄ ≤ USL – k * s
Types of Variable Sampling Plans
(c) 2016 Raul SotoIVT San Diego OCT 2016
60
Example:
• Sampling plan for variables: n = 20 and k = 2.0
• Known process standard deviation: s = 1.2 mg
• One-sided Specification: LSL = 12 mg
• Take a sample, calculate sample mean: x ̄ = 16.32 mg
• Do we accept or reject this lot?
(c) 2016 Raul SotoIVT San Diego OCT 2016
61
Example:
For this example:
• Known process standard deviation, LSL only
• Accept if : LSL + k * s ≤ x ̄ acceptance criteria
• (12 + 2.0 * 1.2 ) ≤ x ̄
• 14.4 ≤ 16.32 YES
• Since the sample mean is larger than the acceptance criteria, the lot is accepted.
(c) 2016 Raul SotoIVT San Diego OCT 2016
62
Another Example:
• Sampling plan for variables: n = 20 and k = 2.0
• Two-Sided Specs : LSL = 120 mg, USL = 140 mg
• Process standard deviation is unknown
• Take a sample, sample mean x ̄= 124.07 mgsample stdev s = 2.15 mg
• Do we accept or reject this lot?
(c) 2016 Raul SotoIVT San Diego OCT 2016
63
Another Example:
For this example:
• LSL and USL ; Unknown process standard deviation
• Accept if: LSL + k * s ≤ x̄ ≤ USL – k*s Acceptance Criteria
• 120 + 2 * 2.15 ≤ 124.07 ≤ 140 – 2 * 2.15• 124.30 ≤ 124.07 ≤ 135.70 ??? NO
• Since the sample mean does not satisfy the acceptance criteria, the lot is rejected
(c) 2016 Raul SotoIVT San Diego OCT 2016
64
Comparing Sampling Plan Types: Attributes vs Variables
• Compare plans: AQL = 1%, LTPD = 5% target
• Attributes:• n = 132, a = 3
• Variables, unknown process standard deviation:• n = 55, k = 1.95
• Variables, known process standard deviation:• n = 19, k = 1.95
• Using a variables sampling plan can provide equivalent levels of protection with a much smaller sample size.
(c) 2016 Raul SotoIVT San Diego OCT 2016
65
Normality Assumption• The normality of the quality characteristic of interest must be tested before using a variables
sampling plan.
• Typical tests for normality:
• Visual methods: • Histograms, P-P plots (probability-probability), Q-Q plots (quartile-quartile)
• Normality tests:• Kolmogorov-Smirnov (KS), Lilliefors-corrected KS, Shapiro-Wilk test, Anderson-Darling,
Cramer-von Mises, D’Agostino skewness test, Anscombe-Glynn kurtosis test, D’Agostino-Pearson omnibus test, Jarque-Bera test.
• KS is frequently used, highly sensitive to extreme values• Some researches consider the Shapiro-Wilk (SW) test as the best choice• Minitab includes AD, KS, and Ryan-Joiner which is similar to SW.
Ghasemi A, Zahediasl S. Normality Tests for Statistical Analysis:
A Guide for Non-Statisticians.
Int J Endocrinol Metab. 2012;10(2):486-9. DOI: 10.5812/ijem.3505
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3693611/
(c) 2016 Raul SotoIVT San Diego OCT 2016
66
Normality Assumption
• For some processes the quality characteristic we want to measure does not follow a normal distribution
• Alternatives: transform the data (for example, take a natural logarithm) and check if the transformed data is normally distributed
• ALWAYS consult with your company’s statisticians before you do any data transformations
(c) 2016 Raul SotoIVT San Diego OCT 2016
67
Effect of lot – to – lot variability
• If the mean of the quality characteristic varies from lot to lot, a variables sampling plan will overestimate the variability and will probably reject good lots.
• This type of variability will also affect your Cpk
• Variables sampling plans do not react well when a process has outliers or short periods of highly defective products.
• Use automated 100% inspection instead
(c) 2016 Raul SotoIVT San Diego OCT 2016
Using Minitab
• Stats / Quality Tools/ Acceptance Sampling by Variables/ [Create / Compare]
68(c) 2016 Raul SotoIVT San Diego OCT 2016
Using Minitab
69
Acceptance Sampling by Variables - Create/Compare
Lot quality in percent defective
Lower Specification Limit (LSL) 7.5Upper Specification Limit (USL) 8.5
Acceptable Quality Level (AQL) 1Producer's Risk (Alpha) 0.05
Rejectable Quality Level (RQL or LTPD) 10Consumer's Risk (Beta) 0.1
Generated Plan(s)
Sample Size 20Critical Distance (k Value) 1.73910Maximum Standard Deviation (MSD) 0.244685
Z.LSL = (mean - lower spec)/standard deviationZ.USL = (upper spec - mean)/standard deviationAccept lot if standard deviation <= MSD, Z.LSL >= k and Z.USL >= k; otherwise reject.
Percent Probability ProbabilityDefective Accepting Rejecting
1 0.953 0.04710 0.111 0.88950 0.000 1.000
(c) 2016 Raul SotoIVT San Diego OCT 2016
70
MS EXCEL Functions for Variables Sampling
Z.TEST (array, x, sigma)Returns the one-tailed P-value of a z-test
NORM.INV (probability, mean, standard_dev)Returns the inverse of the normal cumulative distribution for the specified mean and standard deviation
NORM.S.DIST (z)Returns the standard normal cumulative distribution (has a mean of zero and a standard deviation of one)
NORM.DIST (x, mean, standard_dev, cumulative)Returns the normal cumulative distribution for the specified mean and standard deviation
NORM.S.INV (probability)Returns the inverse of the standard normal cumulative distribution (has a mean of zero and a standard deviation of one)
(c) 2016 Raul SotoIVT San Diego OCT 2016
PART 4
ASQ 1.4 AND 1.9
72
ASQ Z1.4 (2003)
• Most commonly used sampling system in the world… and a lot of people are using it WRONG.
• It is NOT intended to be a table of sampling plans from where you can pick the one you want… but that’s how a lot of people use it.
• ASQ z1.4 is available at www.asq.org
(c) 2016 Raul SotoIVT San Diego OCT 2016
73
ASQ Z1.4 – Use of Switching Rules
• Z1.4 contains tables of matches single, double, and multiple sampling plans.
• It also contains OC Curves, percentiles, ASN curves, and AOQLs for these plans.
• Z1.4 includes reduced, normal, and tightened sampling plans, and a set of switchingrules
• ASQ z1.4 page 7, section 11.6:
• If we are NOT using the switching rules, we can’t state we are inspecting as per ASQ z1.4
(c) 2016 Raul SotoIVT San Diego OCT 2016
74
ASQ
z1
.4 (
20
03
) p
age
7, s
ecti
on
11
.6
(c) 2016 Raul SotoIVT San Diego OCT 2016
75
ASQ Z1.4 – for Individual Plans
• Section 11.6.2
“When employed in this way, this document simply represents a repository for a collection of individual plans indexed by AQL. The operational characteristics and other measures of a plan so chosen must be assessed individually for that plan from the tables provided.”
(c) 2016 Raul SotoIVT San Diego OCT 2016
76
ASQ Z1.4 – How to Use It
• Specify a sampling plan by level of inspection, AQL, and lot size
• get the sample size letter code from Table I
• get the single sampling plan from Table II-A
(c) 2016 Raul SotoIVT San Diego OCT 2016
77
ASQ Z1.4 – Inspection Level
“Three inspection levels: I, II, III, are given in Table I for general use. Normally, Inspection Level II is used. However, Inspection Level I may be used when less discrimination is needed, or Level III may be used for greater discrimination”
“Four additional Special levels, S-1, S-2, S-3, and S-4, are given in the same table, and may be used where relatively small sample sizes are necessary, and large sampling risks can or must be tolerated.”
• Level I has largest LTPD, Level III has smaller LTPDs
• Increasing the Inspection Level basically increases the sampling size letter code.
(c) 2016 Raul SotoIVT San Diego OCT 2016
78
Why Increase the Inspection Level?
• Economics: to provide increased protection for larger lots
• Increase probability of acceptance at the AQL
• Decrease LTPD
(c) 2016 Raul SotoIVT San Diego OCT 2016
79
z1.4 Single Sampling Plans for AQL = 1%
Letter
Code
Sampling
Plan
Pa @ AQL Actual
AQL (%)
LTPD (%)
E n = 13, a = 0 0.877 0.394 16.2
H n = 50, a = 1 0.911 0.715 7.56
J n = 80, a = 2 0.953 1.03 6.52
K n = 125, a = 3 0.963 1.10 5.27
L n = 200, a = 5 0.984 1.31 4.59
M n = 315, a = 7 0.985 1.27 3.71
N n = 500, a = 10 0.987 1.24 3.06
P n = 800, a = 14 0.983 1.16 2.51
Q n = 1250
a = 21
0.991 1.19 2.25
(c) 2016 Raul SotoIVT San Diego OCT 2016
80
ASQ Z1.4
• Probability of acceptance (Pa) is NOT constant in z1.4
• Z1.4 changes the plan as the lot size increases, NOT to maintain a constant level of protection, but instead to give MORE protection to larger lots.
• Changes in lot size mean changing the sample size
• Alternatives:
• Use the average lot size to select a plan OR
• Use the largest lot size to select a plan, then inspect all lots using the same plan
(c) 2016 Raul SotoIVT San Diego OCT 2016
81
ASQ Z1.4 Example
• Select a single sampling plan with AQL = 1%, Level 2 inspection. Lot sizes range from 4,000 to 8,000 units:
• From Table I (pg. 10), code letter is L
• From Table IIA (pg. 11), select plan n = 200, a = 5
• This plan provides the following protection (Table X-L) (pg. 51):
• AQL = 1.31% , slightly higher than the target AQL• LTPD10 = 4.64%• LTPD5 = 5.26%
(c) 2016 Raul SotoIVT San Diego OCT 2016
82(c) 2016 Raul SotoIVT San Diego OCT 2016
83(c) 2016 Raul SotoIVT San Diego OCT 2016
84(c) 2016 Raul SotoIVT San Diego OCT 2016
85
Switching Rules
• z1.4 provides 3 inspection types: Normal, Tightened, and Reduced
• Normal: Default starting point
• Move to reduced when an improvement in lot quality is detected
• Move to tightened when a decrease in lot quality is detected
(c) 2016 Raul SotoIVT San Diego OCT 2016
86
Switching Rules
• Reduced
•Smaller sample sizes than Normal, assumes quality is good
•If ten (10) consecutive lots pass the Normal inspection, move to Reduced
•If one (1) lot fails reduced inspection, move back to Normal
(c) 2016 Raul SotoIVT San Diego OCT 2016
87
Switching Rules
• Tightened
•Lower acceptance numbers than Normal, therefore steeper OC curves
•If two (2) out of five (5) lots fail Normal, move to Tightened
•If five (5) consecutive lots pass Tightened, move back to Normal
(c) 2016 Raul SotoIVT San Diego OCT 2016
88
Example:
• Find a plan with L-2 inspection, AQL = 1.0%• Lot sizes between 4,000 and 8,000 units
• From the Table:• Normal inspection : n = 200, a = 5• Tightened inspection: n = 200, a = 3• Reduced inspection: n = 80, a = 2, r = 5
• What does r mean in the Reduced inspection plan?
• For 0, 1, or 2 defects found in the sample, accept the lot and remain in Reduced inspection
• For 3, 4, or 5 defects fond in the sample, accept the lot but switch to Normal inspection.
• For 6 or more defects, reject the lot and switch to Normal inspection
(c) 2016 Raul SotoIVT San Diego OCT 2016
89
Advantages and Disadvantages of Switching Rules
• Advantages• Improved protection during stationary conditions• Reduced sample size during periods of good quality• Work best in a well established process in good control.
• Disadvantages• Bad protection during periods of changing quality. • No protection against isolated bad quality batches or first batch in a run of bad
ones.• More complex to administer
(c) 2016 Raul SotoIVT San Diego OCT 2016
90
ASQ Z1.9 (2003)
• Equivalent z1.4 but for variables
• Also has normal, reduced and tightened plans, with similar switching rules
• Plans are organized by letter code and AQL
• AQLs in z1.9 go from 0.04% to 15%
• z1.9 has 5 inspection levels : General I, II, III; Special S3, S4• (A7.1): “Normal” is General Level II• We can use Level I for less discrimination or Level III for more• S3 and S4 are used for small sample sizes and large sampling risks
(c) 2016 Raul SotoIVT San Diego OCT 2016
91
ASQ z1.9 vs z1.4
Table on p.101 provides a comparison between:
z1.9 (2003)z1.4 (1972) z1.9 (2003)
(c) 2016 Raul SotoIVT San Diego OCT 2016
92
ASQ Z1.9 : Methods
• Z1.9 includes the following methods
• Unknown variability, standard deviation method•Individual spec (LSL or USL) p.32•Double spec (LSL and USL) p. 38
• Unknown variability, range method•Individual spec (LSL or USL) p.55•Double spec (LSL and USL) p. 62
• Known variability•Individual spec (LSL or USL) p.80•Double spec (LSL and USL) p. 88
(c) 2016 Raul SotoIVT San Diego OCT 2016
RS 93
Z1.9 Tables
IVT San Diego OCT 2016
IVT San Diego OCT 2016 94
95
Example:
• Our quality characteristic has LSL = 225. • AQL target = 1 %• Lot sizes = 100,000 units• Process standard deviation is unknown
• Lot size = 100,000 units, Normal sampling (Gen Lvl 2)• Sample size code letter = N
• From Table B-1 in p.36 for unknown variability• for N the sample size is n = 150• for an AQL = 1%, k = 2.03
(c) 2016 Raul SotoIVT San Diego OCT 2016
Key Take-Aways:• The protection offered by a sampling plan is defined by what it routinely accepts and rejects.
• AQL represents your Producer’s Risk, quantifies what your plan routinely accepts.
• LQ represents your Consumer’s Risk, quantifies what your plan routinely rejects.
• Sample size has the most significant effect on the protection offered by a sampling plan.
• To select a valid sampling plan:• Establish the purpose of the inspection (validation, regular production, etc.)• Pick AQL and LTPD.• Select or calculate a plan that provides the desired level of protection.
• Knowing the AQL and LTPD of all our sampling plans allows us to know the level of protection they offer.
96(c) 2016 Raul SotoIVT San Diego OCT 2016
Selecting Sampling Plans with Statistical Confidence
Select AQLs and LTPDs that provide appropriate protection
KNOW the AQLs and LTPDs for ALL your sampling plans
RANDOMIZE your samples97(c) 2016 Raul SotoIVT San Diego OCT 2016
98
Q & A
(c) 2016 Raul SotoIVT San Diego OCT 2016
References1. Ferryanto, Liem. "Statistical Sampling Plan for Design Verification and Validation of Medical Devices." Journal of Validation Technology
(2015): n. pag. Web. http://www.ivtnetwork.com/article/statistical-sampling-plan-design-verification-and-validation-medical-devices
2. Flaig, John. “Does c = 0 Sampling Really Save Money?” Quality Digest (2013): n. pag. Web. https://www.qualitydigest.com/print/23830
3. Ghasemi A, Zahedias S. Normality Tests for Statistical Analysis: A Guide for Non-Statisticians. Int J Endocrinol Metab. 2012;10(2):486-9. DOI: 10.5812/ijem.3505 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3693611/
4. Gojanovic, Tony. "Back to Basics: Zero Defect Sampling." ASQ Quality Progress (2007): n. pag. Web. http://asq.org/quality-progress/2007/11/basic-quality/zero-defect-sampling.html
5. Niles, Kim. "Sample Wise: Settling on a Suitable Sample Size for Your Project Is Half the Battle." ASQ Quality Progress (2009): n. pag. Web. http://asq.org/quality-progress/2009/05/back-to-basics/sample-wise.html
6. Schilling, Edward G., and Dean V. Neubauer. Acceptance Sampling in Quality Control. Boca Raton: CRC, 2009. Print. https://www.crcpress.com/Acceptance-Sampling-in-Quality-Control-Second-Edition/Schilling-Neubauer/p/book/9781584889526
7. Taylor, Wayne A. Guide to Acceptance Sampling. Lake Villa, IL: Taylor Enterprises, 1992. Print.
8. Taylor, Wayne. "Article - "Acceptance Sampling Update" N.p., n.d. Web. http://www.variation.com/techlib/as-9.html
9. Taylor, Wayne. "Article - "The Effect of Lot Size" N.p., n.d. Web. http://www.variation.com/techlib/as-3.html
10. Taylor, Wayne. "Article - "Statistically Valid Sampling Plans" N.p., n.d. Web. http://www.variation.com/techlib/as-2.html
11. "What Kinds of Lot Acceptance Sampling Plans (LASPs) Are There?" NIST Engineering Standards Handbook Section 6.2.2, n.d. Web. http://itl.nist.gov/div898/handbook/pmc/section2/pmc22.htm
99(c) 2016 Raul SotoIVT San Diego OCT 2016