1

Econ 240A

Power 6

2

Interval Estimation and Hypothesis Testing

3

Outline

Interval Estimation Hypothesis Testing Decision Theory

4

How good was last week’s LA Times Poll?

Oct 1, 2003 LA Times

5

Power 4

6

The Los Angeles Times Poll In a sample of approximately 2000

people, 56% indicate they will vote to recall Governor Davis

If the poll is an accurate reflection or subset of the population of voters next Tuesday, what is the expected proportion that will vote for the recall?

How much uncertainty is in that expectation?

Power 4

LA Times Poll The estimated proportion, from the

sample, that will vote for recall is:

where is 0.56 or 56% k is the number of “successes”,

the number of people sampled who are for recall, approximately 1,120

n is the size of the sample, 2000

nkp /ˆ

p̂

Power 4

LA Times Poll What is the expected proportion of

voters next Tuesday that will vote for recall?

= E(k)/n = np/n = p, where from the binomial distribution, E(k) = np

So if the sample is representative of voters and their preferences, 56% should vote for recall next Tuesday

)ˆ( pE

Power 4

LA Times Poll How much dispersion is in this estimate,

i.e. as reported in newspapers, what is the margin of sampling error?

The margin of sampling error is calculated as the standard deviation or square root of the variance in

= VAR(k)/n2 = np(1-p)/n2 =p(1-p)/n

and using 0.56 as an estimate of p, = 0.56*0.44/2000 =0.0001

p̂)ˆ( pVAR

)ˆ( pVAR

Power 4

10

Interval Estimation

Based on the Poll of 56% for recall, what was the probability that the fraction, p, voting for recall would exceed 50%, i.e. lie between 0.5 and 1.0?

The standardized normal variate, z =

npppp

ppEp

/)ˆ1(*ˆ/)ˆ(

)ˆ(/)ˆˆ(

0.15.0 p

11

Interval estimation

Why can we use the normal distribution?

Where does the formula for z come from?

?)15.0( pp

01./)56.0(/ˆ1(ˆ/)ˆ( pnppppz

Solving for p:.01*z = 0.56 - p

p = 0.56 -.01*z

and substituting for p:

and subtracting 0.56from each of the 3 partsof this inequality:

?)1)*01.56.0(5.0( zp

?)56.01*01.56.05.0( zp

And multiplying by -100,which changes the signs of the inequality:

?)44.0*01.006.0(

?)56.01*01.056.05.0(

zp

zp

?)446( zp

And using the standardized normal distribution, this probabilityequals ….

Density Function for the Standardized Normal Variate

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

-5 -4 -3 -2 -1 0 1 2 3 4 5

Standard Deviations

Den

sity

2]1/)0[(2/1*]2/1[)( zezf

6-44

Cumulative Distribution Function for a Standardized Normal Variate

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-5 -4 -3 -2 -1 0 1 2 3 4 5

Standard Deviations

Pro

ba

bilt

y

6-44

16

Interval Estimation

So by last Wednesday, Arnold’s camp knew that if the “big mo’” did not shift, they were in fat city…

Rather than using values a=0.5, and b=1 for the unknown parameter p, the fraction that would vote for Swarzenegger, the conventional approach is to choose a probability for the interval such as 95% or 99%

17

So z values of -1.96 and1.96 leave2.5% in eachtail

Density Function for the Standardized Normal Variate

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

-5 -4 -3 -2 -1 0 1 2 3 4 5

Standard Deviations

Den

sity

2]1/)0[(2/1*]2/1[)( zezf

-1.96

2.5% 2.5%

1.96

95.0)96.196.1( zp

Substituting for z

01./)56.(/)ˆ1(ˆ/)ˆ( pnppppz

95.)96.101./)56.0(96.1( pp

And multiplying all three parts of the inequality by 0.01

95.)0196.56.00196.( pp

And subtracting 0.56 from all three parts of the inequality

95.)5404.5796.(

95.)56.0196.56.0196.(

pp

pp

And multiplying by -1, which changes the signs of the inequality:

95.0)54.058.0( pp

So a 95% confidence interval based on the poll, predicted a recall vote between 54% and 58%, an inference about the unknown parameter p.

Z values of -2.575 and 2.575 leave 1/2% in each tail.You might calculate a 99% confidence interval for the poll.

21

Based on the Santa Barbara News-Press, with about 52% of the vote counted, recall was 55% yes

22

http://www.sfgate.com/election/races/2003/10/07/map.shtml

23

Interval Estimation

Sample mean example: Monthly Rate of Return, UC Stock Index Fund, Sept. 1995 - Aug. 2003• number of observations: 96• sample mean: 0.816• sample standard deviation: 4.46• Student’s t-statistic• degrees of freedom: 95

)//()( nsxt

Monthly Rate of Return on the UC Stock Index Fund

-15

-10

-5

0

5

10

Jun-94 Oct-95 Mar-97 Jul-98 Dec-99 Apr-01 Sep-02 Jan-04

Date

Ra

te

Samplemean0.816

25

Appendix BTable 4p. B-9

2.5 % in the upper tail

26

Interval Estimation

95% confidence interval

substituting for t

)96/46.4/()816(.)//()(

95.0)985.1985.1(

nsxt

tp

95.)985.1155./)816(.985.1( p

27

Interval Estimation

Multiplying all 3 parts of the inequality by 0.155

subtracting .816 from all 3 parts of the inequality,

95.)308.816.308.( p

95.)508.124.1( p

28

Interval EstimationAn Inference about E(r)

And multiplying all 3 parts of the inequality by -1, which changes the sign of the inequality

So, the population annual rate of return on the UC Stock index lies between 13.4% and 6.1% with probability 0.95, assuming this rate is not time varying

95.0)51.12.1( p

29

Hypothesis Testing

30

Hypothesis Testing: 4 Steps

Formulate all the hypotheses Identify a test statistic If the null hypothesis were true, what is

the probability of getting a test statistic this large?

Compare this probability to a chosen critical level of significance, e.g. 5%

31

Hypothesis Test Example

Last Week’s LA Times Poll on Recall Step #1: null, i.e. the maintained,

hypothesis: true proportion for recall is 50% H0 : p = 0.5; the alternative hypothesis is that the true population proportion supporting recall is greater than 50%, Ha a : p>0.5

32

Hypothesis Test Example

Step #2: test statistic: standardized normal variate z

Step #3: Critical level for rejecting the null hypothesis: e.g. 5% in upper tail; alternative 1% in upper tail

601./)50.056.0(

/)1(*/)ˆ()ˆ(/)ˆˆ(

z

npppppVARpEpz

33

Density Function for the Standardized Normal Variate

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

-5 -4 -3 -2 -1 0 1 2 3 4 5

Standard Deviations

Den

sity

2]1/)0[(2/1*]2/1[)( zezf

6

1.645

5 % upper tail

Samplestatistic

Step #4: compare the probability for the teststatistic(z=6) to the chosen critical level(z=1.645)

35

Hypothesis Test Example

So reject the null hypothesis

36

Decision Theory

Decision Theory Inference about unknown population

parameters from calculated sample statistics are informed guesses. So it is possible to make mistakes. The objective is to follow a process that minimizes the expected cost of those mistakes.

Types of errors involved in accepting or rejecting the null hypothesis depends on the true state of nature which we do not know at the time we are making guesses about it.

Decision Theory For example, consider the LA Times

poll of Oct. 1, and the null hypothesis that the proportion that would vote for recall the following week was 0.5, i.e. p = 0.5. The alternative hypothesis was that this proportion was greater than 0.5, p > 0.5. Last week, no one knew which was right, but guesses could be made based on the poll.

Decision Theory If we accept the null hypothesis

when it is true, there is no error. If we reject the null hypothesis when it is false there is no error.

If we reject the null hypothesis when it is true, we commit a type I error. If we accept the null when it is false, we commit a type II error.

Decision

Accept null

Reject null

True State of Nature

p = 0.5 P > 0.5

No Error

Type I error No Error

Type II error

Decision Theory The size of the type I error is the

significance level or probability of making a type I error,

The size of the type II error is the probability of making a type II error,

We could choose to make the size of the type I error smaller by reducing for example from 5 % to 1 %. But, then what would that do to the type II error?

Decision

Accept null

Reject null

True State of Nature

p = 0.5 P > 0.5

No Error 1 -

Type I error

No Error 1 -

Type II error

Decision Theory There is a tradeoff between the

size of the type I error and the type II error.

This tradeoff depends on the true state of nature, the value of the population parameter we are guessing about. To demonstrate this tradeoff, we need to play what if games about this unknown population parameter.

44

What is at stake?

Suppose last Wednesday you were in Arnold’s camp.

What does the Arnold camp want to believe about the true population proportion p?• they want to reject the null

hypothesis, p=0.5• they want to accept the alternative

hypothesis, p>0.5

Cost of Type I and Type II Errors

The best thing for the Arnold camp is to lean the other way from what they want

The cost to them of a type I error, rejecting the null when it is true is high. They might relax at the wrong time.

Expected Cost E(C) = Chigh(type I error)*P(type I error) + Clow(type II error)*P(type II error)

46

Costs in Arnold’s Camp

Expected Cost E(C) = Chigh(type I error)*P(type I error) + Clow(type II error)*P(type II error)

E(C) = Chigh(type I error)* Clow(type II error)*

Recommended Action: make probability of type I error small, i.e. don’t be eager to reject the null

Decision

Accept null

Reject null

True State of Nature

p = 0.5 P > 0.5

No Error 1 -

Type I error C(I)

No Error 1 -

Type II error C(II)

E[C] = C(I)* + C(II)*

Arnold: C(I) is large so make small

48

How About Costs to the Davis Camp on Oct 1?

What do they want? They do not want to reject the null,

p=0.5 The Davis camp should lean against

what they want The cost of accepting the null when

it is false is high to them, so C(II) is high

49

Costs in the Davis Camp

Expected Cost E(C) = Clow(type I error)*P(type I error) + Chigh(type II error)*P(type II error)

E(C) = Clow(type I error)* Chigh(type II error)*

Recommended Action: make probability of type II error small, i.e. make the probability of accepting the null when it is false small

Decision

Accept null

Reject null

True State of Nature

p = 0.5 P > 0.5

No Error 1 -

Type I error C(I)

No Error 1 -

Type II error C(II)

E[C] = C(I)* + C(II)*

Davis: C(II) is large so make small

Decision Theory Example If we set the type I error, to 1%,

then from the normal distribution (Table 3), the standardized normal variate z will equal 2.33 for 1% in the upper tail.

011.0/)5.0ˆ(2000/5.*5./)5.0ˆ(33.2

/)1(*/)ˆ()ˆ(/)]ˆ(ˆ[

ppz

nppppppEpz

So for this poll size of 2000, with So for this poll size of 2000, with p=0.5 under the null hypothesis, p=0.5 under the null hypothesis, given our choice of the type I given our choice of the type I error of size 1%, which error of size 1%, which determines the value of z of 2.33, determines the value of z of 2.33, we can solve for awe can solve for a

526.0ˆ p

Density Function for the Standardized Normal Variate

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

-5 -4 -3 -2 -1 0 1 2 3 4 5

Standard Deviations

Den

sity

2]1/)0[(2/1*]2/1[)( zezf

2.33

1 %

Decision Theory Example So if 52.6% of the polling sample, or

0.526*2000=1052 say they will recall, then we reject the null of p=0.5.

But suppose the true value of p is 0.54, and we use this decision rule to reject the null if 1052 voters are for recall, but accept the null (assumed false if p=0.54) if this number is less than 1052. What is the size of the type II error?

The Probabilty of a Type II Error

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

940 960 980 1000 1020 1040 1060 1080 1100 1120Polled People For the Recall

Fre

qu

ency

p=0.5

p=0.54

Reject Null

Accept Null

1052

alpha =1 %

?

Decision Theory Example What is the value of the type II error, if

the true population proportion is p = 0.54?

Recall our decision rule is based on a poll proportion of 0.526 or 1052 for recall

z(beta) = (0.526 – p)/[p*(1-p)/n]1/2

Z(beta) = (0.526 – 0.54)/[.54*.46/2000]1/2

Z(beta) = -1.256

nppppbetaz /)1(*/)ˆ()(

true p z beta 1-beta0.51 1.43137 0.923838 0.0761620.52 0.537086 0.704396 0.2956040.53 -0.358417 0.360016 0.6399840.54 -1.256224 0.104517 0.8954830.55 -2.15744 0.015486 0.9845140.56 -3.063187 0.001095 0.9989050.57 -3.974624 3.53E-05 0.9999650.58 -4.892943 4.97E-07 10.59 -5.819384 2.96E-09 1

Decision Theory Example

Operating Characteristic Curve

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.5 0.52 0.54 0.56 0.58 0.6

Presumed Population Proprtion, p

Bet

a

Power Function of the Test

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.5 0.52 0.54 0.56 0.58 0.6

Assumed Population Proportion, p

1-b

eta

Idealpowerfunction