i l l i n o i scourses.education.illinois.edu/edpsy580/lectures/correlation-ha.pdf · i l l i n o i...

Correlation Slide 1 of 38

CorrelationEdpsy 580

Carolyn J. AndersonDepartment of Educational Psychology

I L L I N O I SUNIVERSITY OF ILLINOIS AT URBRANA-CHAMPAIGN

● Outline: Pearson Correlation

Coefficient

Definition & Properties

Inference & the Correlation

Coefficient

Fisher’s Z-Transformation


Overview: Correlation & Regression

■ Pearson correlation coefficient

■ Simple Linear Regression.

◆ What and why?

◆ How (interpretation, estimation & diagnostics).

◆ Statistical Inference.

◆ Comments regarding interpretation.

■ Bi-variate regression

■ Multiple regression

■ General Linear Model


Coefficient



Coefficient



Outline: Pearson Correlation Coefficient

■ Definition & Properties.

■ Statistical Inference

◆ t-test that correlation equals 0.

◆ Fisher’s Z-Transformation.

◆ Confidence intervals for ρ.

◆ Test of Ho : ρ = K.

◆ Test of Ho : ρ1 = ρ2 (2 independent populations).


Coefficient


● Correlation: Definition &

Properties

● Scatter Diagram & Summary

Statistics● Definition: Correlation

coefficient

● How r Works

● Examples of Different r’s


● Non-Linear Relationships

● Properties: Correlation

Coefficient● Properties: Correlation

Coefficient


Coefficient



Correlation: Definition & Properties

■ “Pearson Product Moment Correlation”

■ Two numerical variables measured on same individual,

(Xi, Yi) for i = 1, . . . , n. e.g.,

◆ Height and weight.

◆ Math and science scores.

◆ Salary and merit.

◆ High school GPA and college GPA.

◆ Cost of wine and annual rainfall.

◆ Conservative Party donors and people who buy gardenbulbs by mail.


Coefficient



Properties



coefficient

● How r Works






Coefficient


Coefficient



Scatter Diagram & Summary Statistics


Coefficient



Properties



coefficient

● How r Works






Coefficient


Coefficient



Definition: Correlation coefficient

■ ρ(Greek “rho”) = population correlation.

■ r = sample correlation.

■ Formal definition

r =cov(X, Y )

sxsy=

sxy

sxsy

=1

n−1

∑ni=1(Xi − X)(Yi − Y )

√

1n−1

∑ni=1(Xi − X)2

√

1n−1

∑ni=1(Yi − Y )2

=

∑ni=1(Xi − X)(Yi − Y )

√

∑ni=1(Xi − X)2

√

∑ni=1(Yi − Y )2

■ It measures the extent to which two random variables arelinearly related.


Coefficient



Properties



coefficient

● How r Works






Coefficient


Coefficient



How r Works

r =n

i=1(Xi − X)(Yi − Y )

n

i=1(Xi − X)2 n

i=1(Yi − Y )2=

n

i=1

zxizyi


Coefficient



Properties



coefficient

● How r Works






Coefficient


Coefficient



Examples of Different r’s


Coefficient



Properties



coefficient

● How r Works






Coefficient


Coefficient



Non-Linear Relationships

r = 0


Coefficient



Properties



coefficient

● How r Works






Coefficient


Coefficient



Properties: Correlation Coefficient

■ −1 ≤ r ≤ +1◆ −1 ≤ r < 0 −→ small values of X go with large values of Y and

large values of X go with small values of Y .◆ 0 < r ≤ +1 −→ large values of X go with large values of Y and

small values of X go with small values of Y .◆ r = 0 −→ No linear relationship.

■ r measures the strength of the relationship (magnitude)between two variables and the direction of the relationship(sign).


Coefficient



Properties



coefficient

● How r Works






Coefficient


Coefficient



Properties: Correlation Coefficient

■ r measures linear relationship.■ Linear transformations of X and/or Y do not change the size

(magnitude) of r. Linear transformations do not change thedirection (sign) as long as

X∗ = aX + b

where a > 0 (e.g., z scores).■ In a scatter plot, a linear transformation(s) (where a > 0)

simply corresponds to relabelling axis (axes).


Coefficient



Coefficient● Inference & the Correlation

Coefficient● The Bivariate Normal

Distribution● Example: ρ = 0 &

dependent

● Hypothesis Testing

● Hypothesis Testing● Example Hypothesis Testing

for ρ

● Alternative Method




● Computing Correlations: SAS



Inference & the Correlation Coefficient

■ Preliminaries: bivariate normal distribution.■ This is a generalization of the normal distribution for two

random variables(say X and Y ).

■ The parameters of the bivariate normal distribution are:

µx, σ2x, µy, σ2

y, and ρxy

■ It looks like a bell or a little hill.■ MatLab program.


Coefficient






dependent



for ρ








The Bivariate Normal Distribution

■ If X and Y have a bivariate normal distribution, then◆ X ∼ N (µx, σ2

x)

◆ Y ∼ N (µy, σ2y)

◆ ρxy measures how related X and Y are.

■ If X and Y are bivariate normal and ρxy = 0,then X and Y are statistically independent.

■ If X and Y are statistically independent,

then ρxy = 0.

■ The case where ρxy = 0 and the (joint) distribution of X and Y isnot bivariate normal does not imply that X and Y are statistically

independent.


Coefficient






dependent



for ρ








Example: ρ = 0 & dependent

r = 0

Marginal distributions of X and Y are not normal:


Coefficient






dependent



for ρ








Hypothesis Testing

■ Statistical Hypotheses: The most common case,Ho : ρ = 0 versus Ha : ρ 6= 0

■ Assumptions:◆ X and Y are random variables whose joint distribution is

bivariate normal.*** qualification.◆ Observations are independent.


Coefficient






dependent



for ρ








Hypothesis Testing

■ Test Statistic: Given the assumptions above and Ho : ρ = 0,t =

r√

(1−r2)n−2

■ Sampling Distribution of the test statistic is Student’s t withν = n − 2.

■ Note: the test statistic depends on both r and the samplesize n. So for a given α-level, you do not have to computethe test statistic. . . just find the “critical” value for r.


Coefficient






dependent



for ρ








Example Hypothesis Testing for ρ

■ High School & Beyond: Reading scores and Motivation■ Ho : ρread,mot = 0 vs Ha : ρread,mot 6= 0.■ Test statistic

t =.21061

√

(1−.210612)600−2

=.21061

√

.9556/598= 5.269

■ For ν = 600 = 2 = 598, p value = P (|t| ≥ 5.269) < .001;therefore, Reject Ho.

■ Conclusion: The data provide evidence that there is a linearrelationship between reading and motivation.


Coefficient






dependent



for ρ








Alternative Method

■ Find the critical r and compare to the observed r.■ Will reject Ho : ρ = 0 vs Ha : ρ 6= 0 whenever

observed tn−2 ≤.025 tn−2 or observed tn−2 ≥.975 tn−2

■ Take

t =r

(1−r2)n−2

= r(n − 2)

(1 − r2)

and r as a function of t.


Coefficient






dependent



for ρ








Alternative Method

t =r

√

(1−r2)(n−2)

= r

√

(n − 2)√

(1 − r)2

■ Square both sides and solve for r:t2 = r2 (n − 2)

1 − r2

t2(1 − r2)

(n − 2)= r2

t2

(n − 2)= r2(1 + t2/(n − 2))

r2 =t2

(n − 2)(1 + t2/(n − 2))


Coefficient






dependent



for ρ








Alternative Method

■ So rcrit =tcrit

√

(n − 2)(1 + t2crit/(n − 2))

■ For our HSB example:rcrit =

1.9639√

598(

1 + (1.9639)2

598

)

=1.9639√601.85

= .08

■ Any correlation > .08 (or < −.08) would be “significant” forn = 600.

■ Note: “Statistical significance” does not imply “importance”.


Coefficient






dependent



for ρ








Alternative Method

■ More correlations from the HSB data where N = 600,α = .05, and rcrit = .0877:

Locus of Selfcontrol concept Motivation

Reading .38 .06 .21(p-value) (< .01) (.14) (< .01)

Science .32 .07 .12(p-value) (< .01) (.09) (< .01)

■ Note p-value= Prob(|r| ≥ r given ρ = 0), i.e., Ho : ρ = 0.


Coefficient






dependent



for ρ








Computing Correlations: SAS

■ SAS program command:PROC CORR;

VAR rdg sci;

With locus concpt mot;

■ or

PROC CORR;

VAR rdg sci locus concpt mot;

■ ASSIST

■ ANALYST

■ Interactive data analysis


Coefficient



Coefficient


● Fisher’s Z-Transformation


● Fisher’s Z-Transformation● How Fisher’s

Z-Transformation Works● . . . and for Smaller

Samples. . .

● Sampling Distribution of

Fisher’s Z

● Using Fisher’s Z

● Example Using Fisher’s Z

● Confidence Interval for ρ


● Fisher’s Z in SAS

● Two Independent Group Test

● Two Independent Group Test● Example Two Independent

Group Test● Example Two Independent

Group Test



■ Why? When ρ 6= 0, the sampling distribution for r is skewed.

■ Fisher’s Z-Transformation is a function of r whose samplingdistribution of the transformed value is close to normal.

■ Can compute confidence intervals and a variety of testsusing Fisher’s Z.

■ Requirement: For the distribution of Z to be approximatelynormal,

◆ Variables from a bivariate normal distribution.

◆ Sample size should be n ≥ 10 (and larger if question thebivariate normal assumption).


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test



Fisher’s Z-Transformation:

Z =1

2ln

(

1 + r

1 − r

)

,

where

■ r is the sample correlation

■ Z is the transformed value of r

■ ln is the natural logarithm.

■ Note: the natural logarithm has base equal toexp = e = 2.718281828; that is,

if expa = x then ln(x) = a


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test



■ Taking the logarithm of numbers has the effect of“compressing” the differences or space between the largervalues and “stretching” the space between smaller values.

■ If a distribution is positively skewed, the taking the logarithmhas the effect of making the distribution more symmetric.

■ How Fisher’s Z-Transformation works. . .


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


How Fisher’s Z-Transformation Works


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


. . . and for Smaller Samples. . .


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Sampling Distribution of Fisher’s Z

■ IF Observations◆ Are from a bivariate normal distribution.◆ Are independent across individuals.◆ n ≥ 10

■ THEN the sampling distribution of Z is ≈ N (µZ , σ2Z) where

E(Z) = µZ = Zρ =1

2ln

(

1 + ρ

1 − ρ

)

+ρ

2(n − 1)

σ2Z =

1

n − 3

■ The value ρ2(n−1) is the bias factor, which in SAS you can

request that a bias adjustment be used (in confidenceintervals).

■ µZ and σ2Z are independent of each other.

■ The transformation of r is known as the “inverse of thehyperbolic tangent of r”.


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Using Fisher’s Z

■ HSB data: Are there relationships between psychologicalvariables and achievement: motivation and reading?

■ Observed correlation, r = .21061.

■ If the true population correlation coefficient is ρ > 0, then thesampling distribution of r will be skewed.

■ Use Fisher’s Z transformation,

Z =1

2ln

(

1 + .21061

1 − .21061

)

=1

2ln(1.53360) =

1

2(.4276) = .2138

■ The standard deviation,σZ =

1√600 − 3

= .04093


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Example Using Fisher’s Z

■ Suppose want to test Ho : ρ = .25 vs Ha : ρ 6= .25.

■ Need the value of Z for ρ = .25,

Z.25 =1

2ln

(

1 + .25

1 − .25= .2554

)

■ Test statistic is

z =Zobs − Znull

σZ=

.2138 − .2554

.04093=

−.0416

.04093= −1.016

■ Retain Ho. . . .

■ Note: a lower case z is used for the test statistic and uppercase Z is denotes Fisher’s Z-transformed value of r.


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Confidence Interval for ρ

■ Another use for Fisher’s Z-transformation.

■ Suppose we want a 95% CI for correlation betweenmotivation and reading scores.

■ Steps:

1. Transform the sample correlation: Zobs = .2138.2. Compute the (1 − α)% CI for Zρ

Zobs ± zα/2σZ

.2138 ± 1.96(.04093) =⇒ (.13, .29)

3. Un-transform the end points of the CI above.


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Confidence Interval for ρ

■ Reversing the Fisher Z transformation. . . a little algebra gives

r =e2Z − 1

e2Z + 1

■ Our example

rlower =e2(.1336) − 1

e2(.1336) + 1=

2.71828.2672 − 1

2.71828.2672 + 1=

.3063

2.3063= .1328

rupper =e2(.2940) − 1

e2(.2940) + 1=

2.71828.5880 − 1

2.71828.5880 + 1=

.8004

2.8004= .2858

■ The 95% confidence interval for ρ between motivation andreading scores is (.13, .29).


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Fisher’s Z in SAS

TITLE ’Testing Ho: rho=0 using Fisher-Z transformation’;proc corr data=hsb fisher;

var mot rdg;RUN;

TITLE ’Ho: rho= .25 , No bias adjustment’;proc corr data=hsb fisher(rho0=.25 biasadj=no alpha=.05);var mot rdg;RUN;

TITLE ’Ho: rho= .25 , With bias adjustment’;proc corr data=hsb fisher(rho0=.25 biasadj=yes alpha=.05);var mot rdg;RUN;

——————————————————————- slide -

34-1


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Two Independent Group Test

■ Test whether the correlation from 2 independent groups arethe same or different.

■ The same procedure that we used for testing differencebetween mean for large samples.

■ Statistical hypotheses:

Ho : ρ1 = ρ2 vs Ha : ρ1 6= ρ2

■ Assumptions:

◆ Observations are independent within and betweenpopulations

◆ The joint distribution of the two variables in eachpopulation is bivariate normal.


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Two Independent Group Test

■ Test Statistic:

z =Z1 − Z2

σZ1−Z2

where◆ Z1 and Z2 are Fisher Z-transformations of the sample

correlations, r1 and r2, from the two groups.

◆ Standard deviation,

σZ1−Z2=

√

σ2Z1

+ σ2Z2

=

√

1

n1 − 3+

1

n2 − 3

Why?

◆ Sampling distribution of the test statistic is N (0, 1).


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Example Two Independent Group Test

■ Is the relationship between writing scores and locus ofcontrol the same or different for male and female high schoolstudents?

■ The data: nmale = 327 and rmale = .40196nfemale = 273 and rfemale = .28250

■ Statistical hypotheses:Ho : ρmale = ρfemale vs Ha : ρmale 6= ρfemale

■ Assumptions:

◆ Scores come from bivariate normal populations.

◆ Independence within and between groups.

■ . . . so what’s dependent?


Coefficient



Coefficient






Samples. . .


Fisher’s Z









Group Test


Example Two Independent Group Test

Test Statistic:

Zfemale =1

2ln

(

1 + .28250

1 − .28250

)

=1

2(.5808) = .29040

Zmale =1

2ln

(

1 + .40196

1 − .40196

)

=1

2(.85197) = .425980

σ(Zm−Zf ) =

√

1

nmale − 3+

1

nfemale − 3=

√

1

324+

1

270= .08240

z =.42598 − .29040

.08240= 1.645

Conclusion: Retain Ho for α = .05. The difference between thecorrelations more likely to be due to chance than reflect real adifference.

i l l i n o i scourses.education.illinois.edu/edpsy580/lectures/correlation-ha.pdf · i l l i n o i...

Documents