lecture 5. linear models for correlated data: inference
DESCRIPTION
Inference Estimation Methods –Weighted Least Squares (WLS) (V i known) –Maximum Likelihood (V i unknown) –Restricted Maximum Likelihood (V i unknown) –Robust Estimation (V i unknown) Hypothesis Testing Example: Growth of Sitka TreesTRANSCRIPT
Lecture 5
Linear Models for Correlated Data: Inference
Inference
• Estimation Methods– Weighted Least Squares (WLS)
(Vi known)– Maximum Likelihood (Vi unknown)– Restricted Maximum Likelihood
(Vi unknown)– Robust Estimation (Vi unknown)
• Hypothesis Testing• Example: Growth of Sitka Trees
Weighted-Least Squares Estimation
Weighted-Least Squares Estimation (cont’d)
Weighted-Least Squares Estimation (cont’d)
Weighted-Least Squares Estimation (cont’d)
Weighted-Least Squares Estimation (cont’d)
Weighted-Least Squares Estimation (cont’d)
Weighted-Least Squares Estimation (cont’d)
Weighted-Least Squares Estimation (cont’d)
Estimation of Mean Model: Weighted Least Squares
Estimation of Mean Model: Weighted Least Squares (cont’d)
Estimation of Mean Model: Weighted Least Squares (cont’d)
Note that we can re-write the WRRS as:
What does this equation say?Examples…
Examples: V diagonal
Examples: V diagonal (cont’d)
Examples: V not diagonal
Examples: AR-1 (V not diagonal)
Examples: AR-1 (V not diagonal) (cont’d)
Weighted Least Squares Estimation:Summary
Pigs – “WLS” Fit
“WLS” Model results
Pigs – “WLS” Fit
Pigs – “WLS” Fit
Pigs – “WLS” Fit
Pigs – “WLS” Fit
Pigs – OLS fit. regress weight time
Source | SS df MS Number of obs = 432-------------+------------------------------ F( 1, 430) = 5757.41 Model | 111060.882 1 111060.882 Prob > F = 0.0000 Residual | 8294.72677 430 19.2900622 R-squared = 0.9305-------------+------------------------------ Adj R-squared = 0.9303 Total | 119355.609 431 276.927167 Root MSE = 4.392
------------------------------------------------------------------------------ weight | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- time | 6.209896 .0818409 75.88 0.000 6.049038 6.370754 _cons | 19.35561 .4605447 42.03 0.000 18.45041 20.26081------------------------------------------------------------------------------
OLS results
Pigs – “WLS” Fit
Pigs – “WLS” Fit
“WLS” Model results
Pigs – OLS fit. regress weight time
Source | SS df MS Number of obs = 432-------------+------------------------------ F( 1, 430) = 5757.41 Model | 111060.882 1 111060.882 Prob > F = 0.0000 Residual | 8294.72677 430 19.2900622 R-squared = 0.9305-------------+------------------------------ Adj R-squared = 0.9303 Total | 119355.609 431 276.927167 Root MSE = 4.392
------------------------------------------------------------------------------ weight | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- time | 6.209896 .0818409 75.88 0.000 6.049038 6.370754 _cons | 19.35561 .4605447 42.03 0.000 18.45041 20.26081------------------------------------------------------------------------------
OLS results
Efficiency
Efficiency (cont’d)
Example
Example (cont’d)
When can we use OLS and ignore V?
1. Uniform Correlation Model2. Balanced Data
When can we use OLS and ignore V? (cont’d)
1. (Uniform Correlation) With a common correlation between any two equally-spaced measurements on the same unit, there is no reason to weight measurements differently.
2. (Balanced Data) This would not be true if the number of measurements varied between units because, with >0, units with more measurements would then convey more information per unit than units with fewer measurements.
When can we use OLS and ignore V? (cont’d)
In many circumstances where there is a balanced design, the OLS estimator is perfectly satisfactory for point estimation.
Example: Two-treatment crossover design
Example: Two-treatment crossover design (cont’d)
Example: Two-treatment crossover design (cont’d)
Example: Two-treatment crossover design (cont’d)
(Recall slide) Inference
• Estimation Methods– Weighted Least Squares (WLS)
(Vi known)– Maximum Likelihood (Vi unknown)– Restricted Maximum Likelihood
(Vi unknown)– Robust Estimation (Vi unknown)
• Hypothesis Testing• Example: Growth of Sitka Trees
Maximum Likelihood Estimation under a Gaussian Assumption
Maximum Likelihood Estimation under a Gaussian Assumption (cont’d)
Maximum Likelihood Estimation under a Gaussian Assumption (cont’d)
Maximum Likelihood Estimation under a Gaussian Assumption (cont’d)
(Recall slide) Inference
• Estimation Methods– Weighted Least Squares (WLS)
(Vi known)– Maximum Likelihood (Vi unknown)– Restricted Maximum Likelihood
(Vi unknown)– Robust Estimation (Vi unknown)
• Hypothesis Testing• Example: Growth of Sitka Trees
Restricted Maximum Likelihood Estimation
(Recall slide) Inference
• Estimation Methods– Weighted Least Squares (WLS)
(Vi known)– Maximum Likelihood (Vi unknown)– Restricted Maximum Likelihood
(Vi unknown)– Robust Estimation (Vi unknown)
• Hypothesis Testing• Example: Growth of Sitka Trees
Generalized Least Square EstimatorRobust Estimation
(unstructured covariance matrix)
Robust Estimation of V under a saturated model
Robust Estimation of V“restricted ML” – makes estimates unbiased
Example
Robust Estimation vs.
A Parametric Approach
Maximum Likelihood Estimation of V
Example: Growth of sitka trees
Figure 1. Observed data and mean response profiles in each of the four growth chambers for the treatment and control.
Figure 2. Observed mean response in each of the four
chambers.
Season 1
Season 2
Example: Growth of sitka trees (cont’d)
Example: Growth of sitka trees (cont’d)
We first consider the 1998 data.
Example: Growth of sitka trees (cont’d)
• Unstructured covariance matrix
Example: Growth of sitka trees (cont’d)
Example: Growth of sitka trees (cont’d)
Example: Growth of sitka trees (cont’d)
Example: Growth of sitka trees (cont’d)
Sitka spruce data: Estimated covariance matrix for 1988
Sitka spruce data: Estimated covariance matrix for 1989
Summary: Unstructured Covariance Matrix
Summary: Parametric Models for Covariance
Reasons for parametric modelling:
Summary: Parametric Models for Covariance
(cont’d)Reasons for parametric modelling (cont’d):
Summary: Unstructured vs. Parametric Covariance
Overall Summary
Overall Summary