class 3 relationship between variables ceram february-march-april 2008 lionel nesta observatoire...
TRANSCRIPT
Class 3
Relationship Between Variables
CERAM February-March-April 2008
Lionel Nesta
Observatoire Français des Conjonctures Economiques
Introduction Typically, the social scientist is less interested in
describing one variable than in describing the
association between two or more variables. This
class is devoted to the study of whether (yes or no)
two variables are related (associated).
By relationship between variables, we mean any
association between two dimensions, qualitative or
quantitative or both, which appears to be systematic
in some ways.
Structure of the Class
One qualitative (multinomial) and one
quantitative (continuous/discrete) variables
Analysis of variance
Two qualitative (multinomial) variables
Chi-square (χ²) independence test
Two quantitative (continuous/discrete) variables
Correlation coefficient
ANOVA
ANOVA: ANalysis Of VAriance
ANOVA is a generalization of Student t test
Student test applies to two categories only:
H0: μ1 = μ2
H1: μ1 ≠ μ2
ANOVA is a method to test whether group means are equal or not.
H0: μ1 = μ2 = μ3 = ... = μn
H1: At least one mean differs significantly
ANOVA
This method is called after the fact that it is based on
measures of variance. The F-statistics is a ratio
comparing the variance due to group differences
(explained variance) with the variance due to other
phenomena (unexplained variance).
explained variance
unexplained varianceF Higher F means more explanatory power,
thus more significance of groups.
Revenues (in million of US $ )
Sector 1 Sector 2 Sector 3
Firm 1 18.0 21.5 34.8
Firm 2 18.0 21.5 34.8
Firm 3 18.0 21.5 34.8
Firm 4 18.0 21.5 34.8
Firm 5 18.0 21.5 34.8
Revenues (in million of US $ )
Sector 1 Sector 2 Sector 3
Firm 1 18.0 18.0 18.0
Firm 2 21.5 21.5 21.5
Firm 3 25.0 25.0 25.0
Firm 4 28.7 28.7 28.7
Firm 5 34.8 34.8 34.8
Revenues (in million of US $ )
Sector 1 Sector 2 Sector 3
Firm 1 19.6 23.7 30.8
Firm 2 19.4 28.4 32.9
Firm 3 21.9 28.5 35.3
Firm 4 21.2 31.7 31.8
Firm 5 24.6 37.0 35.7
Do sectors differ significantly in their revenues?
H0 : μ1 = μ2 = μ3 = ... = μn
H1: At least one mean differs significantly.
ANOVA
2 22
Total Variance Within-group variance Between-group variance(Total Sum of Square) (Within sum of Square) (between sum of Square)
SS SS SStotal within between
k kn nk k k
ij ij j k jj i j i j
x x x x n x x
df = (k – 1)df = n – kdf = n – 1
residual
This decomposition produces Fisher’s Statistics as follows:
__
1 explained variance1,
unexplained variance
betweendf num
df denomwithin
SS kF k N k F
SS N k
Origin of variation SS d.f. MSS F-Stat Prob>F
SS-between 379.1 2 189.6
SS-within (residual) 132.5 12 11.0
SS-total 511.6 14 36.54 17.7 0.0003
The result tells me that I can reject the null Hypothesis H0 with 0.03% chances
of rejecting the null Hypothesis H0 while H0 holds true (being wrong).
I WILL TAKE THE CHANCE!!!
The ANOVA decomposition on Revenues
Verify that US companies are larger than those from the rest of the world
with an ANOVA
Are there systematic Sectoral differences in terms of labour; R&D, sales
Write out H0 and H1for each variables
Analyse Comparer les moyennes ANOVA à un fateur What do you conclude at 5% level? What do you conclude at 1% level?
SPSS Application: ANOVA
SPSS Application: t test comparing meansDescriptives
35 447.4501 182.4318 30.83661 384.78256 510.117613 182.0091 817.9253
32 462.3145 310.5638 54.90044 350.34433 574.284688 19.5265 946.5801
281 32416.80 157435.7 9391.827 13929.247 50904.3542 16.0008 1193810
96 409.9650 453.3413 46.26895 318.10950 501.820453 11.1539 1665.716
100 193.4619 97.58658 9.7586578 174.09856 212.825145 49.3978 558.6539
153 8004.322 30796.25 2489.729 3085.3790 12923.2649 14.1116 184461.8
173 1387.709 1264.239 96.11829 1197.9855 1577.432087 141.0070 5852.729
208 17733.77 124017.6 8599.072 780.78382 34686.7595 123.0168 1664540
74 77161.50 222879.1 25909.17 25524.608 128798.396 281.2427 851216.2
45 1089.904 1240.178 184.8749 717.31279 1462.494371 1.0716 3790.107
155 251.1483 167.9513 13.49017 224.49859 277.797952 27.8838 1432.072
1352 14903.52 103262.3 2808.364 9394.2945 20412.7510 1.0716 1664540
55 50230.05 26169.055 3528.635 43155.57 57304.54 13588 104000
64 133708.02 96812.548 12101.569 109524.96 157891.07 20000 308000
306 55764.62 43392.780 2480.600 50883.36 60645.87 3619 181176
99 63445.73 45073.200 4530.027 54456.04 72435.42 2662 145787
120 36001.37 36324.601 3315.967 29435.42 42567.31 2998 149644
161 101231.85 95716.749 7543.537 86334.11 116129.59 1508 403508
177 128311.31 102126.3 7676.286 113161.90 143460.72 18200 417800
280 140859.11 153239.3 9157.799 122831.96 158886.27 647 876000
76 75601.54 42905.729 4921.625 65797.16 85405.92 11305 165000
65 185022.20 81524.803 10111.907 164821.34 205223.06 30964 317100
231 60497.76 42138.389 2772.502 55035.01 65960.51 1153 173000
1634 91298.87 96400.957 2384.818 86621.25 95976.50 647 876000
55 41423.22 35721.57 4816.696 31766.325 51080.11179 5627.646 121962.6
65 21827.52 15167.33 1881.276 18069.238 25585.80114 2590.539 52380.74
309 565218.4 2146365 122102.5 324957.84 805478.883 2158.768 12400000
99 29890.76 15579.40 1565.789 26783.498 32998.01180 9015.374 69895.68
120 12803.84 6396.795 583.9448 11647.575 13960.11274 2814.375 31224.46
161 821966.6 3180044 250622.6 327011.59 1316921.53 467.169 16600000
178 22379.21 18921.53 1418.229 19580.397 25178.02485 1679.668 79085.95
288 291520.4 1310460 77219.60 139531.82 443508.950 52.365 8071404
77 1522011 3744994 426781.6 672001.30 2372019.91 4679.127 12400000
67 23450.50 18731.51 2288.419 18881.521 28019.47136 38.080 81152.94
231 14908.32 11406.94 750.5212 13429.539 16387.09100 262.905 56015.21
1650 318383.6 1713117 42174.03 235663.33 401103.930 38.080 16600000
13
20
28
29
33
35
36
37
38
48
99
Total
13
20
28
29
33
35
36
37
38
48
99
Total
13
20
28
29
33
35
36
37
38
48
99
Total
rd
labour
sales
N Moyenne Ecart-typeErreur
standardBorne
inférieureBorne
supérieure
Intervalle de confiance à95% pour la moyenne
Minimum Maximum
SPSS Application: t test comparing means
ANOVA
5.11E+011 10 5.11E+010 4.934 .000
1.39E+013 1341 1.04E+010
1.44E+013 1351
2.79E+012 10 2.79E+011 36.607 .000
1.24E+013 1623 7.63E+009
1.52E+013 1633
2.43E+014 10 2.43E+013 8.683 .000
4.60E+015 1639 2.80E+012
4.84E+015 1649
Inter-groupes
Intra-groupes
Total
Inter-groupes
Intra-groupes
Total
Inter-groupes
Intra-groupes
Total
rd
labour
sales
Sommedes carrés ddl
Moyennedes carrés F Signification
Chi-Square Independence Test
Introduction to Chi-Square
This part devoted to the study of whether two
qualitative (categorical) variables are independent:
H0: Independent: the two qualitative variables do not
exhibit any systematic association.
H1: Dependent: the category of one qualitative
variable is associated with the category of another
qualitative variable in some systematic way which
departs significantly from randomness.
The Four Steps Towards The Test1. Build the cross tabulation to compute observed joint
frequencies
2. Compute expected joint frequencies under the
assumption of independence
3. Compute the Chi-square (χ²) distance between
observed and expected joint frequencies
4. Compute the significance of the χ² distance and
conclude on H0 and H1
1. Cross Tabulation A cross tabulation displays the joint distribution of two
or more variables. They are usually referred to as a
contingency tables.
A contingency table describes the distribution of two (or
more) variables simultaneously. Each cell shows the
number of respondents that gave a specific
combination of responses, that is, each cell contains a
single cross tabulation.
1. Cross Tabulation We have data on two qualitative and
categorical dimensions and we wish to know
whether they are related
Region (AM, ASIA, EUR)
Type of company (DBF, LDF)
1. Cross Tabulation We have data on two qualitative and
categorical dimensions and we wish to know
whether they are related
Region (AM, ASIA, EUR)
Type of company (DBF, LDF)
continent
263 61.0 61.0 61.0
51 11.8 11.8 72.9
117 27.1 27.1 100.0
431 100.0 100.0
AMER
EUR
JP
Total
ValideEffectifs Pourcentage
Pourcentagevalide
Pourcentagecumulé
AnalyseStatistiques descriptivesEffectifs
1. Cross Tabulation We have data on two qualitative and
categorical dimensions and we wish to know
whether they are related
Region (AM, ASIA, EUR)
Type of company (DBF, LDF)AnalyseStatistiques descriptivesEffectifs
type
167 38.7 38.7 38.7
264 61.3 61.3 100.0
431 100.0 100.0
DBF
LDF
Total
ValideEffectifs Pourcentage
Pourcentagevalide
Pourcentagecumulé
1. Cross Tabulation
Crossing Region (AM, ASIA, EUR) × Type of
company (DBF, LDF) AnalyseStatistiques descriptivesTableaux CroisésCelluleObservé
Tableau croisé continent * type
Effectif
156 107 263
11 40 51
0 117 117
167 264 431
AMER
EUR
JP
continent
Total
DBF LDF
type
Total
2. Expected Joint Frequencies In order to say something on the relationship between
two categorical variables, it would be nice to produce
expected, also called theoretical, frequencies under the
assumption of independence between the two
variables.
Total line Total Column
Overall Sample SizeijE
Crossing Region (AM, ASIA, EUR) × Type of
company (DBF, LDF) AnalyseStatistiques descriptivesTableaux CroisésCelluleThéorique
2. Expected Joint Frequencies
Tableau croisé continent * type
Effectif théorique
101.9 161.1 263.0
19.8 31.2 51.0
45.3 71.7 117.0
167.0 264.0 431.0
AMER
EUR
JP
continent
Total
DBF LDF
type
Total
AnalyseStatistiques descriptivesTableaux
CroisésCelluleObservé & Théorique
2. Expected Joint Frequencies
Tableau croisé continent * type
156 107 263
101.9 161.1 263.0
59.3% 40.7% 100.0%
93.4% 40.5% 61.0%
36.2% 24.8% 61.0%
11 40 51
19.8 31.2 51.0
21.6% 78.4% 100.0%
6.6% 15.2% 11.8%
2.6% 9.3% 11.8%
0 117 117
45.3 71.7 117.0
.0% 100.0% 100.0%
.0% 44.3% 27.1%
.0% 27.1% 27.1%
167 264 431
167.0 264.0 431.0
38.7% 61.3% 100.0%
100.0% 100.0% 100.0%
38.7% 61.3% 100.0%
Effectif
Effectif théorique
% dans continent
% dans type
% du total
Effectif
Effectif théorique
% dans continent
% dans type
% du total
Effectif
Effectif théorique
% dans continent
% dans type
% du total
Effectif
Effectif théorique
% dans continent
% dans type
% du total
AMER
EUR
JP
continent
Total
DBF LDF
type
Total
3. Computing the χ² statistics We can now compare what we observe with what we
should observe, would the two variables be
independent. The larger the difference, the less
independent the two variables. This difference is
termed a Chi-Square distance.
2
2 ij ij
i j ij
O E
E
With a contingency table of n lines and m columns, the statistics follows a χ² distribution with (n-1)×(m-1) degree of freedom, with the lowest expected frequency being at least 5.
AnalyseStatistiques descriptivesTableaux
CroisésStatistiqueChi-deux
Tests du Khi-deux
127.233a 2 .000
166.879 2 .000
431
Khi-deux de Pearson
Rapport devraisemblance
Nombre d'observationsvalides
Valeur ddl
Significationasymptotique
(bilatérale)
0 cellules (.0%) ont un effectif théorique inférieur à 5.L'effectif théorique minimum est de 19.76.
a.
3. Computing the χ² statistics
4. Conclusion on H0 versus H1 We reject H0 with 0.00% chances of being wrong I will take the chance, and I tentatively conclude
that the type of companies and the regional origins are not independent.
Using our appreciative knowledge on biotechnology, it makes sense: biotechnology was first born in the USA, with European companies following and Asian (i.e. Japanese) companies being mainly large pharmaceutical companies.
Most DBFs are found in the US, then in Europe. This is less true now.
Correlations
Introduction to Correlations
This part is devoted to the study of whether – and the
extent to which – two or more quantitative variables are
related:
Positively correlated: the values of one variable “varying somewhat
in step” with the values of another variable
Negatively correlated: the values of one continuous variable
“varying somewhat in opposite step” with the values of another
variable
Not correlated: the values of one continuous variable “varying
randomly” when the values of another variable vary.
Scatter Plot of Fertilizer and Production
Scatter Plot of R&D and Patents (log)
Scatter Plot of R&D and Patents (log)
The Pearson product-moment correlation coefficient is a measure of the co-relation between two variables x and y.
Pearson's r reflects the intensity of linear relationship between two variables. It ranges from +1 to -1.
r near 1 : Positive Correlation r near -1 : Positive Correlation r near 0 : No or poor correlation
,1 1 x yr
Pearson’s Linear Correlation Coefficient r
1
,2 2
1 1
,
n
i ii
x y n nx y
i ii i
x x y yCov x y
r
x x y y
Cov(x,y) : Covariance between x and y
x et y : Standard deviation of x and Standard deviation of y
n : Number of observations
Pearson’s Linear Correlation Coefficient r
Is significantly different from 0 ?
H0 : rx,y= 0
H1 : rx,y 0
,*
2,1
2
x y
x y
rt
r
n
t* : if t* > t with (n – 2) degree of freedom and critical
probability α (5%), we reject H0 and conclude that r
significantly different from 0.
Pearson’s Linear Correlation Coefficient r
Analyse Corrélation Bivariée Click on Pearson
Corrélations
1 .217** .146** .389** .326**
.000 .002 .000 .000
457 457 457 457 457
.217** 1 -.588** -.815** .929**
.000 .000 .000 .000
457 457 457 457 457
.146** -.588** 1 .642** -.248**
.002 .000 .000 .000
457 457 457 457 457
.389** -.815** .642** 1 -.684**
.000 .000 .000 .000
457 457 457 457 457
.326** .929** -.248** -.684** 1
.000 .000 .000 .000
457 457 457 457 457
Corrélation de Pearson
Sig. (bilatérale)
N
Corrélation de Pearson
Sig. (bilatérale)
N
Corrélation de Pearson
Sig. (bilatérale)
N
Corrélation de Pearson
Sig. (bilatérale)
N
Corrélation de Pearson
Sig. (bilatérale)
N
lnpatent
lnassets
lnrd_assets
lnpat_assets
lnrd
lnpatent lnassets lnrd_assets lnpat_assets lnrd
La corrélation est significative au niveau 0.01 (bilatéral).**.
Pearson’s Linear Correlation Coefficient r
Assumptions of Pearson’s r
There is a linear relationships between x and y
Both x and y are continuous random variables
Both variables are normally distributed
Equal differences between measurements represent
equivalent intervals.
We may want to relax (one of) these assumptions
Pearson’s Linear Correlation Coefficient r
Spearman’s Rank Correlation Coefficient ρ Spearman's rank correlation is a non parametric
measure of the intensity of a correlation between two variables, without making any assumptions about the distribution of the variables, i.e. about the linearity, normality or scale of the relationship.
near 1 : Positive Correlation near -1 : Positive Correlation near 0 : No or poor correlation
x,y1 1
n2
i 1x,y x,y 2
6 dRho 1
n n 1
d² : the difference between ranks of paired values of x and y
n : Number of observations
ρ is simply a special case of the Pearson product-moment coefficient in which the data are converted
to rankings before calculating the coefficient.
Spearman’s Rank Correlation Coefficient ρ
Analyse Corrélation Bivariée Click on “Spearman”
Spearman’s Rank Correlation Coefficient ρ
Corrélations
1.000 .243** .130** .385** .335**
. .000 .005 .000 .000
457 457 457 457 457
.243** 1.000 -.536** -.774** .941**
.000 . .000 .000 .000
457 457 457 457 457
.130** -.536** 1.000 .604** -.282**
.005 .000 . .000 .000
457 457 457 457 457
.385** -.774** .604** 1.000 -.669**
.000 .000 .000 . .000
457 457 457 457 457
.335** .941** -.282** -.669** 1.000
.000 .000 .000 .000 .
457 457 457 457 457
Coefficient de corrélation
Sig. (bilatérale)
N
Coefficient de corrélation
Sig. (bilatérale)
N
Coefficient de corrélation
Sig. (bilatérale)
N
Coefficient de corrélation
Sig. (bilatérale)
N
Coefficient de corrélation
Sig. (bilatérale)
N
lnpatent
lnassets
lnrd_assets
lnpat_assets
lnrd
lnpatent lnassets lnrd_assets lnpat_assets lnrd
La corrélation est significative au niveau 0,01 (bilatéral).**.
Pearson’s r or Spearman’s ρ?
Relationship between tastes and levels of
consumption on a large sample? (ρ)
Relationship between income and
Consumption on a large sample? (r)
Relationship between income and
Consumption on a small sample? Both (ρ)
and (r)
Assignments on CERAM_LMC Produce descriptive statistics on R&D, sales and
number of employees, by sector
Perform an ANOVA to test whether there are
significant differences between sectors in these three
variables
Perform an ANOVA using the log of these three
variables. What do you observe
Is the sector composition of the LMCs region specific?