stat 3120 statistical methods i lecture 4 hypothesis testing

STAT 3120Statistical Methods I

Lecture 4Hypothesis Testing

STAT3120 – Hypothesis Testing Some Limitations of Confidence Interval Analysis

– Descriptive– Provides limited analysis of potential errors (Impact/likelihood)

Hypothesis Testing is Decision Oriented– Is a Population Parameter (or Proportion) less than, Equal to, or Greater than a specific

Value (With Decision Making Implications)– Guides user to a particular choice – Provides User with quantitative information regarding probabilities of different

outcomes (both good and bad).

Highlights that Two Different Decision Making Errors Possible– TYPE I or -Error– TYPE II or -Error

-Level of significance predefines standard of proof (risk)

1- – indicating the “power” of the test (the probability of finding a significant effect)

p-Value (Prob-Value) Aids in Interpreting Results– Strength of the Evidence

STAT3120 – Hypothesis Testing

The first step in Hypothesis Testing is to develop a claim to be tested. For example:

• Drug A will lower an individual’s cholesterol level by a minimum of 10 points.<Drug A will not lower an individual’s cholesterol level by a minimum of 10 points>• Trashbag A has greater tensile strength than Trashbag B.< Trashbag A does not have greater tensile strength than Trashbag B>•Automobile A will average at least 35 miles to the gallon.< Automobile A will not average at least 35 miles to the gallon>

Note that every possible claim has an opposite statement – the two statements together must be mutually exclusive and collectively exhaustive.


When conducting Hypothesis Testing, the process of testing includes five distinct parts:

1. Statement of the Claim – This is referred to as the Alternative Hypothesis and is designated as H1 or Ha;

2. Statement of the Opposite of the Claim – This is referred to as the Null Hypothesis and is designated as H0.

You can think of this as the “base case” or “do nothing” scenario;

3. Calculation of the appropriate Test Statistic;

4. Identification of the Rejection Region (H0);

5. Assess Results and Draw Conclusions.


Note that Hypothesis Statements can take one of three forms (these are the Null Statements):

Two Sided Test: ≠ H0

One Sided Test: < H0

One Sided Test: >H0

Develop the appropriate Hypothesis Statements to test the claims (try to develop the statements using mathematical operators):

1.The Coca Cola Marketing Department wants to run a TV ad – “Diet Coke tastes better than Diet Pepsi”. Assume some kind of taste scale of 1-10 (10 is the best). 2.A Pharmaceutical company wants to claim in their marketing materials that a particular drug will lower cholesterol by at least 30%.3.A manufacturer of PVC pipes wants to become a supplier to a large civil engineering firm. The manufacturer claims that they can manufacture pipes to within 1mm of the engineering firms specifications. 4.The Milemaster Tire company has a new tire that they claim will go 100,000 before the treads wear out.



True State of nature

Decision

Ho is trueHo is true Ho is false Ho is false

Reject HoReject Ho

Do not reject Do not reject HoHo

When conducting a hypothesis test, you should always develop the 2x2 matrix below – which compares the statistically-supported decision to the “true” state of nature:

Descriptions of errors: Type I Error: Reject Null Hypothesis When Null is actually true (-

Error) Type II Error: Accept Null Hypothesis which is false (-Error)

Significance level, , is Maximum Risk of making Type I error that we are prepared to “Live With”. In other words, Probability of Type I Error = (usually set at .05 or Less)

A Type II Error = (not typically controlled) Type I Error and Type II error cannot be controlled simultaneously

Type of Error DECISION CONSEQUENCES/COSTS

TYPE I Reject Null; Null is true

Market tire, but should not have done so; may cause customer dissatisfaction, loss, claims for refund

TYPE II Accept Null; Null is false;

Fail to market a good product; opportunity cost


Tire is no goodHo is true

Tire meets expectationsHo is false

Market TireReject Ho

Don’t Market TireDo not reject Ho

customer dissatisfaction, lose market shareType 1 Error -

Gain market shareValid Decision*

No changeValid Decision

Fail to capitalize on good product, opportunity costType II Error -


* The calculated probability of this outcome is 1- . This is known as the “Power” of the test.


Ho is true Ho is false

Reject Ho Typically the Worst Possible Mistake*…this decision asserts that an effect is present when it is not; a false positive.

The “Power” of the test. There was an effect present and it was detected.

Do not reject Ho

A “Push”. There was no effect present and this was correctly determined.

Lost opportunity. There was an effect present and it was not detected; a false negative*

Descriptions of each outcome “in English”:

* In certain contexts, the False Negative is may be the worse mistake.


After the Hypothesis Statements have been developed, and the Type I and Type II errors have been evaluated, we establish the alpha level – the highest probability we are willing to assume of committing a Type I error. This alpha value corresponds to a “Critical” or cut off Z-score*:

One tailed testsalpha/Z-score

.01/2.33 .05/1.645 .10/1.28

Two tailed testsalpha/Z-score

.01/2.575 .05/1.96 .10/1.645

*Note that if the sample sizes are less than 100, a t-stat should be used over a Z-stat.

At this point, we perform the calculation of the ACTUAL Z-score (or t-stat) and compare this to the CRITICAL Z-score (or t-stat):


For Example…A corporation manages a fleet of company cars. A random sample of 40 cars is examined. The mean and std for the sample are 2,752 and 350 miles, respectively. Records for previous years indicated that the average miles driven was 2,600. Use the sample data to test the claim that the current mean is different from the previous mean. Use alpha = .05.

t = (2752-2600)/(350/SQRT(40)) = 2.75

Now, consider these questions…

1.How does this compare to a Critical t of 2.02? 2.What is your decision? 3.What is the implication if you are wrong? 4.What if the claim was testing if “the current mean is greater than the previous mean”? How would your answer change (if at all)?


Sample Conditions p-Value Significance level

Z-Score/t-stat

Associated with the Actual or Calculated Z-score/t-stat

Associated with the Critical Z-score/t-stat

Range 0 < p < 1 0 < < 1

DefinitionActual Probability of Making a Type I Error

Maximum Probability of Making a Type I Error

How Determined and When Known

From Sample Data After Analysis

Set Before Analysis


The Z-scores can be compared directly. However, we typically translate these Z-scores into probabilities:


Decision Rule:

If the p-value is less than the established alpha value, REJECT the null hypothesis and proceed with the claim. The potential error is a Type I (Type II is not possible and therefore not relevant).

If the p-value is greater than the established alpha value, we FAIL TO REJECT the null hypothesis and maintain the status quo. The potential error is a Type II (Type I is not possible and therefore not relevant).


Fun Exercise to do by Hand (yeah!):

A dealer in recycled paper places empty trailers behind Whole Foods locations. The trailers are gradually filled by customers stopping by with their old newspapers and cardboard. The dealer currently picks up the trailers every other week. This schedule works as long as the average amount of recycled paper is more than 1600 cubic feet (the amount needed to justify operating costs of the trailer). The dealer’s records for 18 2-week periods show the following volumes for the Whole Foods on Briarcliff Road and La Vista:

1660 1820 1590 1440 1730 1680 1750 1720 19001570 1700 1900 1800 1770 2010 1580 1620 1690


1. What are the Hypothesis Statements?

2. Complete the testing Matrix.

3. Calculate the Appropriate test statistic (since there are less than 30 observations, lets use a t-stat rather than a z-stat).a) For fun, build the 95% CI – is the target value included?

4. Identify the rejection region for the testing distribution.

5. Draw your conclusion using alpha = .05.

6. What are the implications of making the wrong choice?

stat 3120 statistical methods i lecture 4 hypothesis testing

Documents