class 3 relationship between variables

Class 3Relationship Between

Variables

CERAM February-March-April 2008

Lionel NestaObservatoire Français des Conjonctures Economiques

[email protected]

Introduction Typically, the social scientist is less interested in

describing one variable than in describing the association between two or more variables. This class is devoted to the study of whether (yes or no) two variables are related (associated).

By relationship between variables, we mean any association between two dimensions, qualitative or quantitative or both, which appears to be systematic in some ways.

Structure of the Class One qualitative (multinomial) and one

quantitative (continuous/discrete) variables Analysis of variance

Two qualitative (multinomial) variables Chi-square (χ²) independence test

Two quantitative (continuous/discrete) variables Correlation coefficient

ANOVA: ANalysis Of VAriance ANOVA is a generalization of Student t test

Student test applies to two categories only:

H0: μ1 = μ2

H1: μ1 ≠ μ2

ANOVA is a method to test whether group means are equal or not.

H0: μ1 = μ2 = μ3 = ... = μn

H1: At least one mean differs significantly

ANOVA

This method is called after the fact that it is based on measures of variance. The F-statistics is a ratio comparing the variance due to group differences (explained variance) with the variance due to other phenomena (unexplained variance).

explained varianceunexplained variance

F Higher F means more explanatory power, thus more significance of groups.

Revenues (in million of US $ )

Sector 1 Sector 2 Sector 3

Firm 1 18.0 21.5 34.8

Firm 2 18.0 21.5 34.8

Firm 3 18.0 21.5 34.8

Firm 4 18.0 21.5 34.8

Firm 5 18.0 21.5 34.8



Firm 1 18.0 18.0 18.0

Firm 2 21.5 21.5 21.5

Firm 3 25.0 25.0 25.0

Firm 4 28.7 28.7 28.7

Firm 5 34.8 34.8 34.8



Firm 1 19.6 23.7 30.8

Firm 2 19.4 28.4 32.9

Firm 3 21.9 28.5 35.3

Firm 4 21.2 31.7 31.8

Firm 5 24.6 37.0 35.7

Do sectors differ significantly in their revenues? H0 : μ1 = μ2 = μ3 = ... = μn

H1: At least one mean differs significantly.

ANOVA 2 22

Total Variance Within-group variance Between-group variance(Total Sum of Square) (Within sum of Square) (between sum of Square)

SS SS SStotal within between

k kn nk k k

ij ij j k jj i j i j

x x x x n x x

df = (k – 1)df = n – kdf = n – 1

residual

This decomposition produces Fisher’s Statistics as follows:

__

1 explained variance1,unexplained variance

betweendf num

df denomwithin

SS kF k N k F

SS N k

Origin of variation SS d.f. MSS F-Stat Prob>F

SS-between 379.1 2 189.6

SS-within (residual) 132.5 12 11.0

SS-total 511.6 14 36.54 17.7 0.0003

The result tells me that I can reject the null Hypothesis H0 with 0.03% chances

of rejecting the null Hypothesis H0 while H0 holds true (being wrong).

I WILL TAKE THE CHANCE!!!

The ANOVA decomposition on Revenues

Verify that US companies are larger than those from the rest of the world with an ANOVA

Are there systematic Sectoral differences in terms of labour; R&D, sales

Write out H0 and H1for each variables Analyse Comparer les moyennes ANOVA à un fateur What do you conclude at 5% level? What do you conclude at 1% level?

SPSS Application: ANOVA

SPSS Application: t test comparing meansDescriptives

35 447.4501 182.4318 30.83661 384.78256 510.117613 182.0091 817.925332 462.3145 310.5638 54.90044 350.34433 574.284688 19.5265 946.5801

281 32416.80 157435.7 9391.827 13929.247 50904.3542 16.0008 119381096 409.9650 453.3413 46.26895 318.10950 501.820453 11.1539 1665.716

100 193.4619 97.58658 9.7586578 174.09856 212.825145 49.3978 558.6539153 8004.322 30796.25 2489.729 3085.3790 12923.2649 14.1116 184461.8173 1387.709 1264.239 96.11829 1197.9855 1577.432087 141.0070 5852.729208 17733.77 124017.6 8599.072 780.78382 34686.7595 123.0168 166454074 77161.50 222879.1 25909.17 25524.608 128798.396 281.2427 851216.245 1089.904 1240.178 184.8749 717.31279 1462.494371 1.0716 3790.107

155 251.1483 167.9513 13.49017 224.49859 277.797952 27.8838 1432.0721352 14903.52 103262.3 2808.364 9394.2945 20412.7510 1.0716 1664540

55 50230.05 26169.055 3528.635 43155.57 57304.54 13588 10400064 133708.02 96812.548 12101.569 109524.96 157891.07 20000 308000

306 55764.62 43392.780 2480.600 50883.36 60645.87 3619 18117699 63445.73 45073.200 4530.027 54456.04 72435.42 2662 145787

120 36001.37 36324.601 3315.967 29435.42 42567.31 2998 149644161 101231.85 95716.749 7543.537 86334.11 116129.59 1508 403508177 128311.31 102126.3 7676.286 113161.90 143460.72 18200 417800280 140859.11 153239.3 9157.799 122831.96 158886.27 647 87600076 75601.54 42905.729 4921.625 65797.16 85405.92 11305 16500065 185022.20 81524.803 10111.907 164821.34 205223.06 30964 317100

231 60497.76 42138.389 2772.502 55035.01 65960.51 1153 1730001634 91298.87 96400.957 2384.818 86621.25 95976.50 647 876000

55 41423.22 35721.57 4816.696 31766.325 51080.11179 5627.646 121962.665 21827.52 15167.33 1881.276 18069.238 25585.80114 2590.539 52380.74

309 565218.4 2146365 122102.5 324957.84 805478.883 2158.768 1240000099 29890.76 15579.40 1565.789 26783.498 32998.01180 9015.374 69895.68

120 12803.84 6396.795 583.9448 11647.575 13960.11274 2814.375 31224.46161 821966.6 3180044 250622.6 327011.59 1316921.53 467.169 16600000178 22379.21 18921.53 1418.229 19580.397 25178.02485 1679.668 79085.95288 291520.4 1310460 77219.60 139531.82 443508.950 52.365 807140477 1522011 3744994 426781.6 672001.30 2372019.91 4679.127 1240000067 23450.50 18731.51 2288.419 18881.521 28019.47136 38.080 81152.94

231 14908.32 11406.94 750.5212 13429.539 16387.09100 262.905 56015.211650 318383.6 1713117 42174.03 235663.33 401103.930 38.080 16600000

1320282933353637384899Total1320282933353637384899Total1320282933353637384899Total

rd

labour

sales

N Moyenne Ecart-typeErreur

standardBorne

inférieureBorne

supérieure

Intervalle de confiance à95% pour la moyenne

Minimum Maximum

SPSS Application: t test comparing means

ANOVA

5.11E+011 10 5.11E+010 4.934 .0001.39E+013 1341 1.04E+0101.44E+013 13512.79E+012 10 2.79E+011 36.607 .0001.24E+013 1623 7.63E+0091.52E+013 16332.43E+014 10 2.43E+013 8.683 .0004.60E+015 1639 2.80E+0124.84E+015 1649

Inter-groupesIntra-groupesTotalInter-groupesIntra-groupesTotalInter-groupesIntra-groupesTotal

rd

labour

sales

Sommedes carrés ddl

Moyennedes carrés F Signification

Chi-Square Independence Test

Introduction to Chi-Square

This part devoted to the study of whether two qualitative (categorical) variables are independent:

H0: Independent: the two qualitative variables do not

exhibit any systematic association.

H1: Dependent: the category of one qualitative

variable is associated with the category of another qualitative variable in some systematic way which departs significantly from randomness.

The Four Steps Towards The Test1. Build the cross tabulation to compute observed joint

frequencies

2. Compute expected joint frequencies under the assumption of independence

3. Compute the Chi-square (χ²) distance between observed and expected joint frequencies

4. Compute the significance of the χ² distance and conclude on H0 and H1

1. Cross Tabulation A cross tabulation displays the joint distribution of two

or more variables. They are usually referred to as a contingency tables.

A contingency table describes the distribution of two (or more) variables simultaneously. Each cell shows the number of respondents that gave a specific combination of responses, that is, each cell contains a single cross tabulation.

1. Cross Tabulation We have data on two qualitative and

categorical dimensions and we wish to know whether they are related

Region (AM, ASIA, EUR)

Type of company (DBF, LDF)




Type of company (DBF, LDF)

continent

263 61.0 61.0 61.051 11.8 11.8 72.9

117 27.1 27.1 100.0431 100.0 100.0

AMEREURJPTotal

ValideEffectifs Pourcentage

Pourcentagevalide

Pourcentagecumulé

AnalyseStatistiques descriptivesEffectifs




Type of company (DBF, LDF)AnalyseStatistiques descriptivesEffectifs

type

167 38.7 38.7 38.7264 61.3 61.3 100.0431 100.0 100.0

DBFLDFTotal

ValideEffectifs Pourcentage

Pourcentagevalide

Pourcentagecumulé

1. Cross Tabulation Crossing Region (AM, ASIA, EUR) × Type of

company (DBF, LDF) AnalyseStatistiques descriptivesTableaux CroisésCelluleObservé

Tableau croisé continent * type

Effectif

156 107 26311 40 510 117 117

167 264 431

AMEREURJP

continent

Total

DBF LDFtype

Total

2. Expected Joint Frequencies In order to say something on the relationship between

two categorical variables, it would be nice to produce expected, also called theoretical, frequencies under the assumption of independence between the two variables.

Total line Total ColumnOverall Sample SizeijE

Crossing Region (AM, ASIA, EUR) × Type of company (DBF, LDF)

AnalyseStatistiques descriptivesTableaux CroisésCelluleThéorique

2. Expected Joint Frequencies


Effectif théorique

101.9 161.1 263.019.8 31.2 51.045.3 71.7 117.0

167.0 264.0 431.0

AMEREURJP

continent

Total

DBF LDFtype

Total

AnalyseStatistiques descriptivesTableaux CroisésCelluleObservé & Théorique

2. Expected Joint Frequencies


156 107 263101.9 161.1 263.0

59.3% 40.7% 100.0%93.4% 40.5% 61.0%36.2% 24.8% 61.0%

11 40 5119.8 31.2 51.0

21.6% 78.4% 100.0%6.6% 15.2% 11.8%2.6% 9.3% 11.8%

0 117 11745.3 71.7 117.0.0% 100.0% 100.0%.0% 44.3% 27.1%.0% 27.1% 27.1%167 264 431

167.0 264.0 431.038.7% 61.3% 100.0%

100.0% 100.0% 100.0%38.7% 61.3% 100.0%

EffectifEffectif théorique% dans continent% dans type% du totalEffectifEffectif théorique% dans continent% dans type% du totalEffectifEffectif théorique% dans continent% dans type% du totalEffectifEffectif théorique% dans continent% dans type% du total

AMER

EUR

JP

continent

Total

DBF LDFtype

Total

3. Computing the χ² statistics We can now compare what we observe with what we

should observe, would the two variables be independent. The larger the difference, the less independent the two variables. This difference is termed a Chi-Square distance.

2

2 ij ij

i j ij

O E

E

With a contingency table of n lines and m columns, the statistics follows a χ² distribution with (n-1)×(m-1) degree of freedom, with the lowest expected frequency being at least 5.

AnalyseStatistiques descriptivesTableaux CroisésStatistiqueChi-deux

Tests du Khi-deux

127.233a 2 .000

166.879 2 .000

431

Khi-deux de PearsonRapport devraisemblanceNombre d'observationsvalides

Valeur ddl

Significationasymptotique

(bilatérale)

0 cellules (.0%) ont un effectif théorique inférieur à 5.L'effectif théorique minimum est de 19.76.

a.

3. Computing the χ² statistics

4. Conclusion on H0 versus H1 We reject H0 with 0.00% chances of being wrong I will take the chance, and I tentatively conclude

that the type of companies and the regional origins are not independent.

Using our appreciative knowledge on biotechnology, it makes sense: biotechnology was first born in the USA, with European companies following and Asian (i.e. Japanese) companies being mainly large pharmaceutical companies.

Most DBFs are found in the US, then in Europe. This is less true now.

Correlations

Introduction to Correlations

This part is devoted to the study of whether – and the extent to which – two or more quantitative variables are related:

Positively correlated: the values of one variable “varying somewhat in step” with the values of another variable

Negatively correlated: the values of one continuous variable “varying somewhat in opposite step” with the values of another variable

Not correlated: the values of one continuous variable “varying randomly” when the values of another variable vary.

Scatter Plot of Fertilizer and Production

Scatter Plot of R&D and Patents (log)

The Pearson product-moment correlation coefficient is a measure of the co-relation between two variables x and y.

Pearson's r reflects the intensity of linear relationship between two variables. It ranges from +1 to -1.

r near 1 : Positive Correlation r near -1 : Positive Correlation r near 0 : No or poor correlation

,1 1 x yr

Pearson’s Linear Correlation Coefficient r

1

,2 2

1 1

,

n

i ii

x y n nx y

i ii i

x x y yCov x yr

x x y y

Cov(x,y) : Covariance between x and y

x et y : Standard deviation of x and Standard deviation of y

n : Number of observations


Is significantly different from 0 ?

H0 : rx,y= 0

H1 : rx,y 0

,*

2,1

2

x y

x y

rt

r

n

t* : if t* > t with (n – 2) degree of freedom and critical

probability α (5%), we reject H0 and conclude that r

significantly different from 0.


Analyse Corrélation Bivariée Click on Pearson

Corrélations

1 .217** .146** .389** .326**.000 .002 .000 .000

457 457 457 457 457.217** 1 -.588** -.815** .929**.000 .000 .000 .000457 457 457 457 457

.146** -.588** 1 .642** -.248**

.002 .000 .000 .000457 457 457 457 457

.389** -.815** .642** 1 -.684**

.000 .000 .000 .000457 457 457 457 457

.326** .929** -.248** -.684** 1

.000 .000 .000 .000457 457 457 457 457

Corrélation de PearsonSig. (bilatérale)NCorrélation de PearsonSig. (bilatérale)NCorrélation de PearsonSig. (bilatérale)NCorrélation de PearsonSig. (bilatérale)NCorrélation de PearsonSig. (bilatérale)N

lnpatent

lnassets

lnrd_assets

lnpat_assets

lnrd

lnpatent lnassets lnrd_assets lnpat_assets lnrd

La corrélation est significative au niveau 0.01 (bilatéral).**.


Assumptions of Pearson’s r There is a linear relationships between x and y Both x and y are continuous random variables Both variables are normally distributed Equal differences between measurements represent

equivalent intervals.

We may want to relax (one of) these assumptions


Spearman’s Rank Correlation Coefficient ρ Spearman's rank correlation is a non parametric

measure of the intensity of a correlation between two variables, without making any assumptions about the distribution of the variables, i.e. about the linearity, normality or scale of the relationship.

near 1 : Positive Correlation near -1 : Positive Correlation near 0 : No or poor correlation

x,y1 1

n2

i 1x,y x,y 2

6 dRho 1

n n 1

d² : the difference between ranks of paired values of x and y

n : Number of observations

ρ is simply a special case of the Pearson product-moment coefficient in which the data are converted to rankings before calculating the coefficient.

Spearman’s Rank Correlation Coefficient ρ

Analyse Corrélation Bivariée Click on “Spearman”

Spearman’s Rank Correlation Coefficient ρ

Corrélations

1.000 .243** .130** .385** .335**. .000 .005 .000 .000

457 457 457 457 457.243** 1.000 -.536** -.774** .941**.000 . .000 .000 .000457 457 457 457 457

.130** -.536** 1.000 .604** -.282**

.005 .000 . .000 .000457 457 457 457 457

.385** -.774** .604** 1.000 -.669**

.000 .000 .000 . .000457 457 457 457 457

.335** .941** -.282** -.669** 1.000

.000 .000 .000 .000 .457 457 457 457 457

Coefficient de corrélationSig. (bilatérale)NCoefficient de corrélationSig. (bilatérale)NCoefficient de corrélationSig. (bilatérale)NCoefficient de corrélationSig. (bilatérale)NCoefficient de corrélationSig. (bilatérale)N

lnpatent

lnassets

lnrd_assets

lnpat_assets

lnrd

lnpatent lnassets lnrd_assets lnpat_assets lnrd

La corrélation est significative au niveau 0,01 (bilatéral).**.

Pearson’s r or Spearman’s ρ?

Relationship between tastes and levels of consumption on a large sample? (ρ)

Relationship between income and Consumption on a large sample? (r)

Relationship between income and Consumption on a small sample? Both (ρ) and (r)

Assignments on CERAM_LMC Produce descriptive statistics on R&D, sales and

number of employees, by sector

Perform an ANOVA to test whether there are significant differences between sectors in these three variables

Perform an ANOVA using the log of these three variables. What do you observe

Is the sector composition of the LMCs region specific?

class 3 relationship between variables

Documents