Lecturer : Prof. Dr Nor Aishah Hamzah Date : 23rd May 2014

WHAT IS RANDOM EFFECTS MODEL ???

a treatments

Factor that has a large number of possible treatments

OBJECTIVE: To draw conclusions about the entire population of levels, not just that were used in the experiment.

Random Effects Model

Single factor

Two-factor mixed

Two-factor

factorial

SINGLE FACTOR MODEL

• yij = + i + ij i = 1, 2, …, a j = 1, 2, …, n

Statistical Model

• i N ( 0, 2 )

• ij N ( 0, 2 )

• i and ij are independent.

• V(yij) = 2 + 2

Assumptions

• H0 : 2 = 0

• H0 : i = 0 Hypothesis

Testing

Statistical Analysis

Sum of Squares

• SStotal = (𝑦𝑖𝑗 − 𝑦 ..)2𝑛

𝑗=1𝑎𝑖=1

• SStreat = n (𝑦 𝑖. − 𝑦 ..)2𝑎

𝑖=1

• 𝑆𝑆𝐸 = (𝑦𝑖𝑗 − 𝑦 𝑖.)2𝑛

𝑗=1𝑎𝑖=1

Under 𝐻0 ∶ 𝜎𝜏2 = 0

•SS

total

2 𝑁−12

•SStreat2 𝑎−1

2

•𝑆𝑆𝐸

2 𝑁−𝑎2

𝐍𝐨𝐭𝐞 𝐭𝐡𝐚𝐭 : 𝑦𝑖𝑗 N ( , 2

+ 2T )

Expected Mean Squares

• E(𝑀𝑆𝑡𝑟𝑒𝑎𝑡) = 2 + n𝜎𝜏2

• E(𝑀𝑆𝐸) = 2

Test Statistic

𝐹0 = 𝑆𝑆𝑡𝑟𝑒𝑎𝑡𝑎−1𝑆𝑆𝐸𝑁−𝑎

= 𝑀𝑆𝑡𝑟𝑒𝑎𝑡

𝑀𝑆𝐸

Reject 𝑯𝟎 if 𝐹0 𝐹,𝑎−1,𝑁−𝑎

( 𝐻0 ∶ 𝜎𝜏2= 0 )

ANOVA Table Source of

variability

Sum of squares Degrees of

freedom

Mean squares 𝐹0

Between

treatments SStreat =

1

𝑛 𝑦𝑖.

2𝑛𝑖=1 -

𝑦..2

𝑁

a-1 𝑀𝑆𝑡𝑟𝑒𝑎𝑡

=𝑆𝑆𝑡𝑟𝑒𝑎𝑡𝑎 − 1

𝐹0 = 𝑀𝑆𝑡𝑟𝑒𝑎𝑡

𝑀𝑆𝐸

Error(within

treatments)

𝑆𝑆𝐸 =SStotal - SStreat N-a 𝑀𝑆𝐸 =

𝑆𝑆𝐸𝑁 − 𝑎

Total SStotal = 𝑦𝑖𝑗2𝑛

𝑗=1𝑎𝑖=1 -

𝑦..2

𝑁

N-1

Estimation of Variance Components (𝜎2 and 𝜎𝜏2)

Analysis of variance method: 𝜎 2 = 𝑀𝑆𝐸

𝜎 𝜏2 =

𝑀𝑆𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 −𝑀𝑆𝐸𝑛

Note that: Analysis of variance method produces a negative estimate of variance component.

Analysis Using Confidence Limit: Applying the formula below:

L = 1

𝑛(𝑀𝑆𝑡𝑟𝑒𝑎𝑡

𝑀𝑆𝐸

1

𝐹𝛼2,𝑎−1,𝑁−𝑎

− 1)

U = 1

𝑛(𝑀𝑆𝑡𝑟𝑒𝑎𝑡

𝑀𝑆𝐸

1

𝐹1−

𝛼2,𝑎−1,𝑁−𝑎

− 1)

Therefore, a 100(1-) percent confidence interval for 𝜎𝜏2

𝜎2+ 𝜎𝜏2 is :

𝐿

1+𝐿

𝜎𝜏2

𝜎2+ 𝜎𝜏2

𝑈

1+𝑈

TWO-FACTOR FACTORIAL MODEL • yijk = 𝜇+ 𝑖 + 𝑗 + ()𝑖𝑗 + ∈ijk

Statistical Model

• 𝜏𝑖, 𝛽𝑗, (𝜏𝛽)𝑖𝑗 and 𝜖𝑖𝑗𝑘 are all independent random variables.

• 𝑖 N ( 0 , 𝜎𝜏2 )

• 𝑗 N ( 0 , 𝜎2 )

• ()𝑖𝑗 N ( 0 , 𝜎2 )

• ∈𝑖𝑗𝑘 N ( 0 , 𝜎2)

• V(𝑦𝑖𝑗𝑘) = 𝜎𝜏2 + 𝜎

2 + 𝜎2 + 𝜎2

Assumption

• H0: 𝜎𝜏2 = 0

• H0: 𝜎2 = 0

• H0: 𝜎2 = 0

Hypothesis Testing

Statistical Analysis

Sum of Squares

Expected Mean Squares

• E(𝑀𝑆𝐴) = 𝑏𝑛𝜎𝜏2 + 𝑛𝜎

2 + 𝜎2

• E(𝑀𝑆𝐵) = 𝑎𝑛𝜎2 + 𝑛𝜎

2 + 𝜎2

• E(𝑀𝑆𝐴𝐵) = 𝑛𝜎2 + 𝜎2

• E(𝑀𝑆𝐸) = 𝜎2

Test Statistics

𝐻0: 𝜎𝜏𝛽2= 0

• 𝐹0 = 𝑀𝑆𝐴𝐵

𝑀𝑆𝐸

H0: 𝜎𝜏2= 0

• 𝐹0 = 𝑀𝑆𝐴

𝑀𝑆𝐴𝐵

H0: 𝜎𝛽2= 0

• 𝐹0 = 𝑀𝑆𝐵

𝑀𝑆𝐴𝐵

ANOVA Table Source of

variability

Sum of

Squares

Degrees of

freedom

Mean squares 𝐹0

Treatment A 𝑆𝑆𝐴 a-1 𝑀𝑆𝐴 =

𝑆𝑆𝐴𝑎 − 1

𝑀𝑆𝐴𝑀𝑆𝐴𝐵

Treatment B 𝑆𝑆𝐵 b-1 𝑀𝑆𝐵 =

𝑆𝑆𝐵𝑏 − 1

𝑀𝑆𝐵𝑀𝑆𝐴𝐵

Interaction 𝑆𝑆𝐴𝐵 (a-1)(b-1) 𝑀𝑆𝐴𝐵

=𝑆𝑆𝐴𝐵

(𝑎 − 1)(𝑏 − 1)

𝑀𝑆𝐴𝐵𝑀𝑆𝐸

Error 𝑆𝑆𝐸 ab(n-1) 𝑀𝑆𝐸 =

𝑆𝑆𝐸𝑎𝑏(𝑛 − 1)

Total 𝑆𝑆𝑇𝑜𝑡𝑎𝑙 abn-1

Estimation of Variance Components

Analysis of variance method:

𝜎 2 = 𝑀𝑆𝐸

𝜎 𝜏𝛽2 =

𝑀𝑆𝐴𝐵 −MS𝐸𝑛

𝜎 𝛽2 =

𝑀𝑆𝐵 −MS𝐴𝐵𝑎𝑛

𝜎 𝜏2 =

𝑀𝑆𝐴 −MS𝐴𝐵𝑏𝑛

The situation where one of the factors a is fixed and the other factor b is random. This is called the mixed model analysis of variance.

ASSUMPTION

Linear model

i = 1, 2, …, a

𝒚𝒊𝒋𝒌 = + 𝒊 + 𝒋 + ()𝒊𝒋 + ∈𝒊𝒋𝒌 j = 1, 2, …, b

k = 1, 2, …, n

• 𝑖 is a fixed effect

• 𝑗 is a random effect

• the interaction ()𝑖𝑗 is assumed to be a random effect

• ∈𝑖𝑗𝑘 is a random error

We assumed that:

i) 𝑖 𝑎𝑖=1 = 0

ii) 𝑗 N(0, 𝜎𝛽2)

iii) ()𝑖𝑗 N(0, [(a-1)/a]𝜎𝜏𝛽2 )

iv) ()𝑖𝑗𝑎𝑖=1 = ().𝑗 = 0 , j= 1,2,…,b

v) ∈𝑖𝑗𝑘 N(0,2)

HYPOTHESIS TESTING

i) Test for no difference in means of the fixed factor effects:

𝐻0: 𝜏𝑖 = 0

ii) Test for no difference in the means of the random factor effects:

𝐻0: 𝜎𝛽2 = 0

iii) Test for no interaction effects:

𝐻0: 𝜎𝜏𝛽2 = 0

Expected Mean Squares

• E(𝑀𝑆𝐴) = 𝑏𝑛 𝜏𝑖

2𝑎𝑖=1

𝑎−1 + 𝑛𝜎

2 + 𝜎2

• E(𝑀𝑆𝐵) = 𝑎𝑛𝜎2 + 𝜎2

• E(𝑀𝑆𝐴𝐵) = 𝑛𝜎2 + 𝜎2

• E(𝑀𝑆𝐸) = 𝜎2

Test Statistics

𝐻0: 𝜎𝜏𝛽2= 0

• 𝐹0 = 𝑀𝑆𝐴𝐵

𝑀𝑆𝐸

H0: 𝜎𝜏 2= 0

• 𝐹0 = 𝑀𝑆𝐴

𝑀𝑆𝐴𝐵

H0: 𝜎𝛽2= 0

• 𝐹0 = 𝑀𝑆𝐵

𝑀𝑆𝐸

ANOVA Table Source of

variability

Sum of

Squares

Degrees of

freedom

Mean squares 𝑭𝟎

Treatment

A

𝑆𝑆𝐴 a-1 𝑀𝑆𝐴 =

𝑆𝑆𝐴𝑎 − 1

𝑀𝑆𝐴𝑀𝑆𝐴𝐵

Treatment

B

𝑆𝑆𝐵 b-1 𝑀𝑆𝐵 =

𝑆𝑆𝐵𝑏 − 1

𝑀𝑆𝐵𝑀𝑆𝐸

Interaction 𝑆𝑆𝐴𝐵 (a-1)(b-1) 𝑀𝑆𝐴𝐵

=𝑆𝑆𝐴𝑏

(𝑎 − 1)(𝑏 − 1)

𝑀𝑆𝐴𝐵𝑀𝑆𝐸

Error 𝑆𝑆𝐸 ab(n-1) 𝑀𝑆𝐸 =

𝑆𝑆𝐸𝑎𝑏(𝑛 − 1)

Total 𝑆𝑆𝑇𝑜𝑡𝑎𝑙 abn-1

ESTIMATION OF MODEL PARAMETERS

Estimate fixed factor effect as :

𝜇 = 𝑦 …

𝜏𝑖 = 𝑦 𝑖.. - 𝑦 …

ESTIMATION OF VARIANCE COMPONENTS

E(𝑀𝑆𝐴) = 𝑏𝑛 𝜏𝑖

2𝑎𝑖=1

𝑎−1 + 𝑛𝜎

2 + 𝜎2 eliminate

E(𝑀𝑆𝐵) = 𝑎𝑛𝜎2 + 𝜎2

E(𝑀𝑆𝐴𝐵) = 𝑛𝜎2 + 𝜎2

E(𝑀𝑆𝐸) = 𝜎2

𝜎 2 = 𝑀𝑆𝐸

Solve above equations, we obtained 𝜎 𝜏𝛽2 =

𝑀𝑆𝐴𝐵−MS𝐸

𝑛

𝜎 𝛽2 =

𝑀𝑆𝐵−MS𝐸

𝑎𝑛

Article 1

Growth and renewable energy in Europe:

A random effect model with evidence for neutrality hypothesis

By: Angeliki N. Menegaki

The study on the causal relationship between economic growth and renewable energy for 27 European countries over period 1997-2007 using a random effect model.

• Annual data from 1997 to 2007

• 27 european countries

Dependent variable:

GDP = real gross domestic product per capita in PPP terms

Independent variable:

• RES = percentage of renewable energy resources in gross inland energy consumption

• CON = final energy consumption

• GRE = greenhouse gas emission

• EMP = employment rate

Unit Root Test

• Levin, Lin and Chu test (LLC)

Breush Pagan LM

Haussmann chi square

Why choose random effects model?

• Country is perceived as a random variable being part of a larger population of countries

• In fixed models the interest lays in the individual means across the levels of the fixed factor while in random effect model the interest lays in the variance of means across the levels of a random factor

• Fixed effect model would rather inconvenient due to dummy variable requirement and the large number of countries included in the data set.

One- way random effects model

GDP = 5.71 +0.44RES – 0.00 CON + 0.60 GRE + 0.49 EMP

• The RES, GRE, EMP were significant at 5 %.

The result indicates that:

• 1% increase in RES, 4.4% increase in GDP

• 1% increase in GRE, 6.0% increase in GDP

• 1% increase in EMP, 4.9% increase in GDP

• 1% increase in greenhouse emission give more positive impact compared to an increase in renewable energy sources.

• This is due to high cost faced in renewable energy investment.

• So, regulators must build a more efficient regulatory framework to reward investment in RES

Article 2

Comparison of models of fixed and mixed effects on the analysis of an

experiment with mutant strains of cellulotic fungus Trichoderma viride

By : Sarai Gómez, Verena Torres,

Yoleisy García, L.M. Fraga, Lucía Sarduy and Lourdes L. Savón

Objectives

To compare the fixed and mixed effects model on the analysis of repeated measurements in time through an experiment with mutant strains of cellulolytic fungi Trichoderma viride.( For our study, we decided to focus on mixed effects model only)

Fixed effects: The strains and the sampling times

Random effects: The experimental units (Erlenmeyer flasks)

The data

The results of an experiment with mutant strains of cellulolytic fungi

Trichoderma viride M5 and MCX1371 of the strains bank from the Department of Bio-physiological Sciences of the Institute of Animal Science were taken

They were assessed through solid state fermentation and

5 g of the solid material were taken at 0, 48, 72 and 96 h of fermentation.

Four repetitions per treatment were carried out for the analysis of the physic chemical properties and the variables NDF and lignin were used (Savón et al. 2005).

The estimation of the parameters is based on the methods of maximum likelihood or restricted maximum likelihood

Result and discussion

The repeated measurements in the same experimental unit

through time are more efficient than using one different experimental unit for each measuring in time. It not only requires fewer units, it reduces the costs too. The measurements increase in precision because it tends to vary less compared to it does in different units. The repeated measurements may complicating the measurement process and analysis of the experiments. These inconvenients may be solved with a mixed model.

The analysis with mixed models solved the failing of the basis hypothesis, solves the limitation of the multivariate analysis of variance and gives higher flexibility and information when selecting the model of better fit. It also allows analyzing data bases in unbalanced designs

The verosimilarity and the information criteria of Akaike and Bayesian were considered in the first place as fit criteria of the model. They are based on estimations of the lack of information; they are unbiased, of minimum variance and adaptable to experiments with unbalanced data. The statistics with lower value for both variables were selected for achieving a better fit of the model.

Table 5 shows the results of the analysis of variance for the mixed model. The interaction between sampling times and treatments (strains) did influence significantly (P<0.001) in both measurements. The differences between the sampling times also influenced the result significantly (P<0.001) in both measurements. However, the effects of strains are not significant in both measurements.

In the mixed model, it does not combine several methods, as in the previous case. Besides, it offers criteria of the goodness of fit of the model

The structure is better seen and the interference space is more widen because the precision is the same with correlated and incomplete data, and it is more flexible with the failing of the traditional hypothesis

It offers a wider information as it has information criteria for selecting the model of best fit. The mixed model responds, first, to one of the limitations of the multivariate analysis of variance,referred to the amount of individuals and variables

Table 4 The second analysis were obtained adjustment or statistical information criteria for each of the variables under study

References Angeliki N. Menegaki (2011 March). Growth and renewable energy in Europe: A random effect model with evidence for neutrality hypothesis. Energy Economics, 33(2), 257–263. Retrieved from http://www.sciencedirect.com/science/article/pii/S0140988310001829 Montgomery, D.C.: Design and Analysis of Experiments. 7th edition, John Wiley, 2004. Richard Ayamah. (2011). Factorial Design and Model For The Effect Of National Road Safety Strategy Two (NRSS II). A Case Study In The Northern And Ashanti Regions. Kwame Nkrumah University of Science and Technology, Ghana. Sarai Gómez, Verena Torres, Yoleisy García, L.M. Fraga, Lucía Sarduy and Lourdes L. Savón (2012). Comparison of models of fixed and mixed effects on the analysis of an experiment with mutant strains of cellulotic fungus Trichoderma viride. Agricultural Science, 46 (2), 127-131. Retrieved from http://www.ciencia-animal.org/cuban-journal-of-agricultural- science/articles/V46-N2-Y2012-P127-Sarai-Gomez.pdf