1

Introduction* to Mixed Effects and Repeated Measures Models

Presented by Peter Westfall, Professor of Statistics, Texas Tech

Warning: There will be a quiz

*Some material is adapted from Brown and Prescott, Applied Mixed Models in Medicine, Wiley New York (1999); Some from Littell et al. The SAS SystemFor Mixed Models, SAS Institute Inc.1996

2

Mixed Models are Useful For…

• Crossover trials• Multicenter trials• Unbalanced data, missing data• Comparing means when sample sizes vary• Repeated observations on a patient

Over timeConcurrently, but in different body locations

(e.g. left and right eye)

3

Introductory Example (EX1)Treatment A eye 1, Treatment B in eye 2 (randomized) Treatment Difference PatientPatient A B A-B Mean 1 20 12 8 16.0 2 26 24 2 25.0 3 16 17 -1 16.5 4 29 21 8 25.0 5 22 21 1 21.5 6 24 17 7 20.5Mean 22.83 18.67 4.17 20.75

4

Model A (bad): Independence

yij = + tj + eij , where

yij = observation j on patient ij=A or B (treatment) = overall meantj = “effect” of treatment jeij = error term

Assumptions:• , ti are fixed • the eij are random, mean zero, and (gasp) independent, with common variance 2.

5

SAS File for model A (ind.)Title "model A: bad";Data ex1; input sub y trt$ @@; cards;1 20 A 1 12 B2 26 A 2 24 B3 16 A 3 17 B4 29 A 4 21 B5 22 A 5 21 B6 24 A 6 17 B;proc glm; class trt; model y = trt; estimate "Mean A" intercept 1 trt 1 0; estimate "Mean B" intercept 1 trt 0 1; run; quit;

6

Results from Model A

StandardParameter Estimate Error t Value Pr > |t|

Mean A 22.8333333 1.79891943 12.69 <.0001Mean B 18.6666667 1.79891943 10.38 <.0001

Dependent Variable: y

Sum of Source DF Squares Mean Square F Value Pr > F Model 1 52.0833333 52.0833333 2.68 0.1325 Error 10 194.1666667 19.4166667 Corrected Total 11 246.2500000

7

Model B: Fixed Patient Effects

yij = + pi + tj + eij , where

yij = observation j on patient ij=A or B (treatment) = overall meanpj = “effect” of patient itj = “effect” of treatment jeij = error term

Assumptions:• , pi, and ti are fixed• the eij are independent mean zero random variables with common variance 2.

8

SAS File for model B (fixed effects.)

Title "model B: fixed";proc glm data=ex1; class trt sub; model y = trt sub;

estimate "Mean A" intercept 1 trt 1 0 ; estimate "Mean B" intercept 1 trt 0 1 ;

estimate "Mean A, sub 2" intercept 1 trt 1 0 sub 0 1 0 0 0 0 ; estimate "Mean B, sub 6" intercept 1 trt 0 1 sub 0 0 0 0 0 1 ;

run; quit;

9

Results from Model B

StandardParameter Estimate Error t Value Pr > |t|

Mean A 22.8333333 1.14624992 19.92 <.0001Mean B 18.6666667 1.14624992 16.28 <.0001Mean A, sub 2 27.0833333 2.14443725 12.63 <.0001Mean B, sub 6 18.4166667 2.14443725 8.59 0.0004

Sum of Source DF Squares Mean Square F Value Pr > F

Model 6 206.8333333 34.4722222 4.37 0.0634 Error 5 39.4166667 7.8833333 Corrected Total 11 246.2500000

Source DF Type III SS Mean Square F Value Pr > F

trt 1 52.0833333 52.0833333 6.61 0.0500 sub 5 154.7500000 30.9500000 3.93 0.0798

10

Model C-1: Random effectsyij = + pi + tj + eij , where

yij = observation j on patient ij=A or B (treatment) = overall meanpj = “effect” of patient itj = “effect” of treatment jeij = error term

Assumptions:• and the ti are fixed • the pj and the eij are random, mean zero, and independent, with mean 0, and with Var(pj)= and Var(eij) = 2.

2p

11

SAS File for model C-1 (random effects)

Title "model C-1: random";proc mixed data=ex1; class trt sub; model y = trt; random sub;

estimate "Mean A" intercept 1 trt 1 0 ; estimate "Mean B" intercept 1 trt 0 1 ;

estimate "Mean A, sub 2" intercept 1 trt 1 0 | sub 0 1 0 0 0 0 ; estimate "Mean B, sub 6" intercept 1 trt 0 1 | sub 0 0 0 0 0 1 ;

run; quit;

12

Results from Model C-1

Standard Label Estimate Error DF t Value Pr > |t|

Mean A 22.8333 1.7989 5 12.69 <.0001 Mean B 18.6667 1.7989 5 10.38 0.0001 Mean A, sub 2 26.0008 1.9396 5 13.41 <.0001 Mean B, sub 6 18.4803 1.9396 5 9.53 0.0002

Covariance Parameter Estimates

Cov Parm Estimate sub 11.5333 Residual 7.8833

Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F trt 1 5 6.61 0.0500

-2 Res Log Likelihood 59.4

13

Model C-2: Mean – Covariance Form of

C-1yij = + tj + eij , where

yij = observation j on patient ij=A or B (treatment) = overall meantj = “effect” of treatment jeij = error term

Assumptions:• and the ti are fixed • the (eiA,eiB) are random, mean zero, and independent vectors, with Cov(eiA,eiB) =

2 2 2

2 2 2p p

p p

14

SAS File for model C-2

Title "model C-2: mean-cov form";proc mixed data=ex1; class trt sub; model y = trt; repeated trt / subject = sub type=cs;

estimate "Mean A" intercept 1 trt 1 0 ; estimate "Mean B" intercept 1 trt 0 1 ;

run; quit;

15

Results from Model C-2

Estimates

Standard Label Estimate Error DF t Value Pr > |t|

Mean A 22.8333 1.7989 5 12.69 <.0001 Mean B 18.6667 1.7989 5 10.38 0.0001

Covariance Parameter Estimates

Cov Parm Subject Estimate CS sub 11.5333 Residual 7.8833

Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F trt 1 5 6.61 0.0500

-2 Res Log Likelihood 59.4

16

Variance, Covariance notes:

Var(X) = E{(X-E(X))2} Var(aX+b) = a2Var(X), if a and b are constantsVar(X+Y) = Var(X) + Var(Y) + 2Cov(X,Y)

Cov(X,Y) = E{(X-E(X))(Y-E(Y))}Cov(aX+b, cY+d) = acCov(X,Y), if a,b,c,d are constants

Covariance matrix: = Cov(X1,…,Xk) = 1 1 2 1 k

2 1 2 2 k

k 1 k 2 k

Var(X ) Cov(X ,X ) Cov(X ,X )

Cov(X ,X ) Var(X ) Cov(X ,X )

Cov(X ,X ) Cov(X ,X ) Var(X )

Covariance matrix Of a vector of linear combinations: Cov(AX) = AA’

17

Exercise 1: Show that model C-1 implies model C-2

Exercise 2: Find the variance of the difference between

treatment averages using A, C-1, and C-2.

18

Model D: Unstructured Covariance Matrix

Identical to model C-2 except that Cov(eiA,eiB) =

11 12

12 22

Exercise 3: Show that Model C-2 Implies Model D, but not vice versa.

19

SAS File for model D

Title "model D: unstructured";proc mixed data=ex1; class trt sub; model y = trt; repeated trt / subject = sub type=un;

estimate "Mean A" intercept 1 trt 1 0 ; estimate "Mean B" intercept 1 trt 0 1 ;

run; quit;

20

Results from Model D

Estimates Standard Label Estimate Error DF t Value Pr > |t|

Mean A 22.8333 1.8693 5 12.21 <.0001 Mean B 18.6667 1.7256 5 10.82 0.0001

Covariance Parameter Estimates

Cov Parm Subject Estimate

UN(1,1) sub 20.9667 UN(2,1) sub 11.5333 UN(2,2) sub 17.8667

Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F trt 1 5 6.61 0.0500

-2 Res Log Likelihood 59.4

21

More exercises

Exercise 4: Show how to get the standard error for the mean of treatment A in the unstructured model

Exercise 5: Find the standard error for the difference between sample means using the unstructured model.

22

Model E: Multivariate Analysis representation of

model DTitle "standard multivariate";Data ex1_mv; input yA yB; diff = yA-yB; cards; 20 12 26 24 16 17 29 21 22 21 24 17 ;proc univariate; var diff; run;proc corr cov; var ya yb; run;

23

Output from multivariate analysis

Tests for Location: Mu0=0 Test -Statistic- -----p Value------ Student's t t 2.570363 Pr > |t| 0.0500

Covariance Matrix, DF = 5 yA yB yA 20.96666667 11.53333333 yB 11.53333333 17.86666667

Pearson Correlation Coefficients, N = 6 Prob > |r| under H0: Rho=0 yA yB yA 1.00000 0.59589 0.2120

yB 0.59589 1.00000 0.2120

24

Linear models Univariate: Y = X + (nx1) (nxp) (px1) (nx1)

Multivariate: Y = X + (nxm) (nxp) (pxm) (nxm)

Comment: With mixed linear models, we always use the “univariate”form. But it is helpful to know the correspondence.

Exercise 6: Write model D in the univariate form. Write model E in the multivariate form.Exercise 7: Suppose that there is a covariate Xi, for each patient. Write model D in univariate form, allowing (covariate x treatment) interaction. Repeat for model E.

25

Example 2: Randomized placebo/control trial

Patients are randomized to treatment goups. Measurements are taken from both eyes. Data is in

http://members.tripod.com/PWestfall/ex2.txt

Patient Trt Eye Response

1 A L 20.1

1 A R 17.6

2 A L 17.1

2 A R 22.3

… … … …

51 P L 12.4

51 P R 13.5

52 P L 12.3

52 P R 13.5

… … … …

26

Random Effects Model for Example 2

yijk = + pi + tk + eijk , where

yijk = observation j on patient i in treatment group kj=L or R (eye)k = A or P (Treatment or Placebo) = overall meanpj = “effect” of patient itk = “effect” of treatment keijk = error term

Assumptions:• and the tk are fixed • the pj and the eijk are random, mean zero, and independent, with mean 0, and with Var(pj)= and Var(eijk) = 2.

2p

27

SAS File for Example 2: Random Effects

data ex2; infile "c:\research\ex2.txt"; input patid trt$ eye$ resp; run;

proc mixed data=ex2; class trt eye patid; model resp = trt/s; random patid;run; quit;

28

Output from random effects model, Example 2

Covariance Parameter Estimates Cov Parm Estimate

patid 5.1586 Residual 7.4941

-2 Res Log Likelihood 1054.7

Solution for Fixed Effects StandardEffect trt Estimate Error DF t Value Pr > |t|Intercept 15.0030 0.4220 98 35.55 <.0001trt A 1.1340 0.5968 100 1.90 0.0603trt P 0 . . . .

Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F

trt 1 100 3.61 0.0603

29

Mean-Covariance form for Ex. 2

yijk = + tk + eijk , where

yijk = observation j on patient i in treatment group kj=L or R (eye)k = A or P (Treatment or Placebo) = overall meantk = “effect” of treatment keijk = error term

Assumptions:• and the tk are fixed • the (eiLk, eiRk) are random, mean zero, and independent vectors, with Cov(eiLk, eiRk) =

2 2 2

2 2 2p p

p p

30

SAS File for Mean-Covariance form of Example 2

proc mixed data=ex2; class trt eye patid; model resp = trt/s; repeated eye/subject=patid type=cs;run; quit;

Exercise 8: Would the “type=un” covariance matrix make any sense here?

Exercise 9: Write the data for this study as Y = X + (use “…” notation as needed)

Exercise 10: Find the standard error of the difference between Sample treatment means

31

Fitting heteroscedastic covariance matrices

Model: Same as the mean-covariance form for ex. 2, but

with Cov(eiLA, eiRA) =

And Cov(eiLP, eiRP) =

2 2 2

2 2 2Ap A Ap

Ap Ap A

2 2 2

2 2 2Pp P Pp

Pp Pp P

32

SAS File for heteroscedastic covariances in Example 2

proc mixed data=ex2; class trt eye patid; model resp = trt/s; repeated eye/subject=patid type=cs group=trt;run; quit;

Exercise 11: Write the model as Y = X + . What isdifferent from Exercise 9?

33

Comparing Models

Nested Models: Chi-square test2 = -2LnL0 - (-2LnL1) with df = difference in number of parameters

Any models: Penalized Likelihood AIC = -2LnL +2q (smaller-is-better)BIC = -2LnL + 2qLog(N*) (smaller-is-better)

q = number of cov parameters N* = approx. number of independent sampling units

34

Output from heteroscedastic model, Example 2

Covariance Parameter Estimates

Cov Parm Subject Group Estimate Variance patid trt A 8.4105 CS patid trt A 2.5618 Variance patid trt P 6.5777 CS patid trt P 7.7555

-2 Res Log Likelihood 1051.1

Solution for Fixed Effects

StandardEffect trt Estimate Error DF t Value Pr > |t|Intercept 15.0030 0.4700 98 31.92 <.0001trt A 1.1340 0.5968 98 1.90 0.0604trt P 0 . . . .

LRT: = 1054.7 - 1051.1 = 3.6; 22 2

2,.05 5.99 ; no evidence of het.

35

Mixed Models TheoryGeneral form of a mixed model: Y = X + Zu + e

Where:X denotes the fixed effects part: X is the fixed design matrix; is the fixed unknown parameter vector Zu denotes the random effects part: Z is a fixed design matrix;u is a random unobservable vector of random effects with E(u) = 0 and Cov(u) = G

e denotes the random unobservable vector of errors, with E(e) = 0 and Cov(e) = R.

u and e are assumed uncorrelated (and ideally, normal).

36

More exercises!!!

Exercise 12: Write models C-1, C-2, and D of exercise 1 in the form of the mixed model. Identify all model terms, including the covariance matrices G and R.

Exercise 13: Write down the heteroscedastic covariance modelfrom example 2 in the form of the mixed model, identifying all terms, including G and R. Use “…” as needed.

37

Modeling with PROC MIXED

• MODEL Statement X and • RANDOM Statement Z, u, and G • REPEATED Statement R

Different (often equivalent) ways to model covariance: • Ignore Z and u and model all within-subject covariance using R • Model within-subject covariance using Z and u

38

Notes on PROC MIXED SyntaxYou can have more than one RANDOM statement.

You can have only one REPEATED statement

SUBJECT= class variable name indicates a collection of observations that are correlated within the levels of the SUBJECT variable and are uncorrelated between levels

GROUP = class variable name indicates a collection of observations that have common covariance parameters within a level of the GROUP variable, but different covariance parameters between levels

39

The Covariance Matrix of Y

Cov(Y) = V = Z G Z’ + R

Exercise 14: Prove it!

Exercise 15: Write down V for model C-1 of exercise 1, anduse it to find the correlation between y3A and y3B.

40

Estimation and Testing of

GLS: 1 1ˆ ( ' ) 'X V X X V Y

EGLS: 1 1ˆ ˆ ˆ( ' ) 'X V X X V Y Where ˆˆ ˆ'V ZGZ R

Parameters in V̂ and ̂R are estimated using ML methods

Standard errors: if c’ is estimable, then 1ˆ ˆ. .( ' ) '( ' )s e c c X V X c

Exercise 16: Derive standard error formula.

41

Why GLS?????

You could use OLS, so why not?

Estimates, s.e.’s : ̂ ( ' ) 'X X X Y 1ˆ ˆ. .( ' ) '( ' )s e c c X V X c where

1 0 0 0

0 1 0 0ˆ 0 0 1 0

0 0 0 1

V MSE

Problems with OLS: - Inefficient estimates - Incorrect s.e.’s

42

Estimation of G and R1 1

1

1 1

Let ( ' ) '

1For ML : ( , ) ln | | ln ' ( )

2 21 1

For REML : ( , ) ln | | ln ' ln | ' | ( )2 2 2

r Y X X V X X V Y

nll G R V r V r Const

n pll G R V r V r X V X Const

   

Maximize ll (log likelihood) with respect to the parameters in G andR to get estimates.

Note: REML (the default) provides the “usual” unbiased estimatesIn simple cases; eg, SSE/(n-p) in OLS, rather than SSE/n (the MLE).

43

Modeling a Multi-Center Clinical Trial

Data:

Baseline Diastolic Blood Pressure (dbp) on each patientNine-week dbp measure (LOCF)Treatments A,B, and C Data come from 35 centers (1 – 40 patients per center)

44

A Quiz

Exercise 17: Write down a model of the form yabc… = …with appropriate subscripts. State and defend the assumptions of your model. Exercise 18: Sketch a graph showing how the regression lines (fory = 9 week dbp, x = baseline dbp) might look for treatments A, B and C in center 1; draw another graph showing the same for center 2.

Exercise 19: Write down your model for the multicenter clinicaltrial in the form of the general mixed model, identifying all terms,including G and R. Use “…” as needed.

Exercise 20: Write down V for your multi-center model – use“…” as needed.

45

PROC MIXED syntax for Multi-Center Model

(Data and SAS files for the Brown and Prescott book availableat http://www.ed.ac.uk/phs/mixed/html/Download.html )

DATA a; INFILE 'c:\research\bp.dat'; INPUT patient visit center treat $ dbp dbp1 cf cf1;run;

DATA a; SET a; BY patient; IF last.patient; run;

PROC MIXED covtest; CLASS center treat; TITLE 'MODEL 1'; MODEL dbp = dbp1 treat/ SOLUTION; random intercept treat/ subject=center; LSMEANS treat/ DIFF PDIFF CL;run; quit;

46

Subscript-based model

DBPijk = + b(pre)ijk + (trt)k + (center)j + (centertrt)jk + eijk

i = patient id (distinct)j = center id (distinct) k = treatment (A, B or C)

Assumes , b, trtA, trtB, trtC are fixed; the (center)i, (centertrt)jk, and eijk are random, independent mean 0, with Var((center)i) = , Var((centertrt)jk) = , and Var(eijk) =

2c 2

ct 2

47

Equivalent Model using SUBJECT=

Replace random center treat*center; with random intercept treat/ subject=center;

This creates several independent models and “stacks” them:

Model 1 (Center=1): DBPik = + b(pre)ik + (trt)k + m + (rtrt)k + eik

, b and trtk are fixed; m, rtrtk and eik are independent, mean 0, with Var(m) = , Var((rtrt)k)= , and Var(eijk) =

Model 2 (Center=2): DBPik = + b(pre)ik + (trt)k + m + (rtrt)k + eik

Random effects differ from model 1 (they are independent) but fixed effects and variance components are identical

Model 3, Model 4 … similar story

2c 2

ct 2

48

Printing the V, G and R matrices

With RANDOM statement: - use the G option (very big matrix) - with SUBJECT= option, use V= <list of subject values> to see blocks of V

With REPEATED statement: - Use R to see whole thing - with SUBJECT= option, use R= <list of subject values> to see blocks of R

49

Example Covariance Structures

Compound Symmetry CS

First-Order Autoregressive AR(1)

Toeplitz with Two BandsTOEP(2)

First-Order Autoregressive Moving-Average ARMA(1,1)

50

Repeated Measures Examples:First example - One Subject

onlyPredict yi = ln(CorpProfits)i

from xi = ln(GNP)i

i=year, from 1960 to 1991 (n=32)

Data set available at http://www2.tltc.ttu.edu/westfall/images/5349/corp_prof.htm

Plot of lprof*lgnp. Legend: A = 1 obs, B = 2 obs, etc.

lprof ‚ ‚ ‚ 5.5 ˆ ‚ ‚ A AA ‚ A A ‚ A A 5.0 ˆ A A A ‚ A A A ‚ ‚ A A A ‚ 4.5 ˆ A A ‚ A ‚ ‚ ‚ A 4.0 ˆ A ‚ A AA A A ‚ A ‚ A ‚ A 3.5 ˆ A ‚ ‚ AA ‚ ‚ 3.0 ˆ ‚ Šˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆ 6.0 6.5 7.0 7.5 8.0 8.5 9.0

lgnp

51

SAS Filedata corp1; infile "c:\research\corp.txt"; input year corpprof gnp; lprof = log(corpprof); lgnp = log(gnp); run;proc reg data=corp1; title "OLS analysis - assumes unc. errors"; model lprof = lgnp / dw; run;proc mixed data=corp1; title "Identical to OLS"; model lprof = lgnp/s; run;proc mixed data=corp1; title "Assumes CS covariance structure"; model lprof = lgnp/s; repeated /subject = intercept type = cs; run;proc mixed data=corp1; title "Assumes Autoregressive covariance structure"; model lprof = lgnp/s; repeated /subject = intercept type = ar(1); run;proc mixed data=corp1; title "Assumes ARMA(1,1) covariance structure"; model lprof = lgnp/s; repeated / subject=intercept type = arma(1,1) ; run;

proc mixed data=corp1; title "Assumes banded Toeplitz covariance structure - 1 lag"; model lprof = lgnp/s; repeated /subject = intercept type = toep(2) ;

proc mixed data=corp1; title "Assumes banded Toeplitz covariance structure - 2 lags"; model lprof = lgnp/s; repeated /subject = intercept type = toep(3) ;

proc mixed data=corp1; title "Assumes banded Toeplitz covariance structure - 3 lags"; model lprof = lgnp/s; repeated /subject = intercept type = toep(4) ;

proc mixed data=corp1; title "Assumes banded Toeplitz covariance structure - 4 lags"; model lprof = lgnp/s; repeated /subject = intercept type = toep(5) ;run;quit;

52

0.7

0.75

0.8

0.85

0.9

0.92

0.94

0.96

0.97

0.98

0.99

0.05 0.07 0.09

0.11

-200

-150

-100

-50

0

50

LogL

Rho (not drawn to scale)

Sigma**2

Plot of Log Likelihood FunctionAR(1) Error Covariance Model for Corp Prof

53

Quiz

Exercise 21: Write down the AR(1) corp profits model using “subscript” notation. State all model assumptions and defend (or criticize) them.

Exercise 22: Identify Y, X, Z, u, , G and R for the AR(1)corp profits model. Use “…” as needed.

Exercise 23: Repeat exercise 20 and 21 for the CS model.

Exercise 24: Write the CS model as a “random effects” model.Use this model to show why there is a problem with using the CS model for these data.

54

Repeated Measures: Multiple Subjects

Case description:  Workers perform various lifting tasks throughout the day.   There is an amount of stress associated with each lift.  Ergonomists have defined various measurements of stress.  In the data set there are two such measures; one is called li81 (published in 1981) and the other is li91 (published in 91).

The input data has the logs of these stress measures.  The funding agency wants to know if the li81 measure is predictive of the li91 measure. Our analysis must account for the repeated measures on each subject, which implies that the observations within a person are correlated. http://www2.tltc.ttu.edu/westfall/images/5349/li%20case_description.htm

55

Repeated Measures Regression Model

Yij = b0 + b1 Xij + eij

Y = ln(li91)X = ln(li81)i = workerj = observation within workerb0 = “population” interceptb1 = “population” slope

Note: We must model correlation between (ei1 ei2 …)

56

Quiz

Exercise 25: Write down the random effects model in “subscript” form that will allow a CS covariance matrix.Write the CLASS, MODEL and RANDOM statementsTo correspond.

Exercise 26: Write down the model in “mean and covariancematrix form” to allow a CS covariance matrix. Write the CLASS, MODEL and REPEATED statements to correspond.

57

Random Intercept and CS Models

Random Intercept:data liftindx; infile "c:\research\liftindex.txt"; input id$ idgrp$ lnli91 lnli81;run;

proc mixed data=liftindx covtest; title "Repeated Measures Using Random Intercept - Same as CS"; class id; model lnli91 = lnli81/s; random int / sub=id v=1,10,19,29,33;run;

Exercise 27: Write the “random” line without using sub=id.

CS covariance Structure: use repeated/ subject=id type=cs r=1,10,19,29,33;

58

Random Coefficients

Random intercept and slope (no covariance structure equivalent)

Random Coefficients model:yij = + 1xij + 0i + 1ixij + ij , where E(0i)=E(1i)=E(ij)=0; Var(0i)=00, Var(1i)=11, Cov(0i,1i)=01, Var(ij)=2; and {0i, 1i} , {ij} independent.(No way to anticipate the “mean – covariance matrix” form)

proc mixed data=liftindx covtest; title "Repeated Measures Using Random Coefficients Model"; class id; model lnli91 = lnli81/s; random int lnli81 / type=un sub=id g gcorr v=1,10,19;run;

59

Random Subject-Specific RegressionsAs predicted by the degenerate model

-3

-2

-1

0

1

2

3

4

-3 -1 1 3

ln(Li81)

Ln

(Li9

1)

Y1

Y2

Y3

Y4

Y5

Y6

Y7

Y8

Y9

60

Quiz

Exercise 28: Write the form of the X, , Z, and u matrices (or vectors) for the RC model in this case. Use “…” notationas needed.

Exercise 29: Write the theoretical form of the V matrix.Use “…” notation as needed.

Exercise 30: Would other (preferably, more parsimonious) covariance structures for the random coefficients make sense?Discuss.

61

Seemingly Unrelated Regressions

(Related to crossover trials and time-dependent covariates)

Yij = investment by in year i by company j X1ij = market value of company j in year iX2ij = capitalization of company j in year i

j = GM, Chrys, GenElec, WestingH, USSTeeli = 1935-1954

Data available at http://www2.tltc.ttu.edu/westfall/images/5349/grunfelds_investment_data.htm

There are 5 regression equations, one for each firm, and they are “seemingly unrelated”

Exercise 30: Write the regression models, and explain how and why the residuals might be correlated.

62

Missing Data In SUR

One beauty of the Mixed Model approach is that complete casesare not needed – you can use all of the data that you have.

Caveat: Missing cases should be missing AT RANDOM! (the bad news is that this assumption is frequently not satisfied!)

Example: Investment data, with data for various companies Missing at particular years.

63

SAS file showing missing data handling in SUR

data inv; infile "c:\research\grunf_fake.txt"; input year Inv mktval cap comp$;run;Proc sort; by year;Proc print; title "Data structure used by PROC MIXED";proc mixed order=data; title "SUR Using PROC MIXED"; class year comp; model inv = comp mktval*comp cap*comp/s noint; repeated comp/subject=year type=un r=1,2,3,21,22,23 ;run;

64

Details of Growth Curve CaseAnalysis of Hemodialyzers

data dial; infile "C:\research\dial.txt"; input sub qb tmp ufr index; tmp = tmp/100; ufr = ufr/100; run;

Response variable:UFR = ultraftration rate

Predictor variables:QB = blood flow (200 dl/min or 300 dl/min)TMP = transmembrane pressure (.24, .505, .995, 1.485, 2.02, 2.495 and 3.0 dmHg)INDEX = TMP number (1,2,3,4,5, or 6)SUB = Dialyzer (1,…,20)

65

S o lid lin e s a re Q B = 2 0 0 a n d d a s h e d lin e s a re Q B = 3 0 0

s u b 1 2 3 4 5 67 8 9 1 0 1 1 1 2

1 3 1 4 1 5 1 6 1 7 1 81 9 2 0 2 1 2 2

u fr

0

1 0

2 0

3 0

4 0

5 0

6 0

7 0

tm p

0 1 2 3 4

Plot For Growth Curve Example

66

Quiz!!!!

Exercise 31: Write a mixed model to predict UFR. What assumptions are you making? Do those assumptions seem reasonable?

Exercise 32: Write the SAS PROC MIXED code needed to fit your growth curve model.

67

BLUPs

Example: Consider estimating the means of treatment A in the blood pressure trial for different centers.

Objective: Rank the centers according to average dbp in theirpatient populations.

Problem: The simple averages may be inappropriate when the number of patients per center varies.

Solution: Rank centers using BLUPs (best linear unbiased predictors),which shrink the estimates toward an overall mean.

68

Mixed Model Equations

General Mixed Model:

Y = X + Z + , with Cov()=G and Cov() = R

BLUP estimates for and GLS estimates for are obtained simultaneously as follows:

69

SAS FileDATA a; INFILE 'c:\research\bp.dat'; INPUT patient visit center treat $ dbp dbp1 cf cf1; run;DATA a; SET a; BY patient; IF last.patient; if treat="A"; run;proc sort; by center; run;proc means; output out=meandbp mean (dbp) = dbp n (dbp) = ndbp; by center; run;proc mixed data = a; class center treat; model dbp = treat/ noint s outpred=pred; random center/ s; run; data pred1; set pred; by center; if first.center; run;data both; merge pred1 meandbp; by center; run;proc sort; by dbp; run;proc print; var dbp pred ndbp center; run;

70

Final Words

• Mixed Model Methodology provides a powerful, flexible tool for estimating effects with repeated measures data

• Likelihood-based methodology allows for simple model comparisons

• Caveats: The methods are suspect with– Gross nonnormality– Badly misspecified covariance structure (including non-modelled

heteroscedasticity)– Data missing not at random– Nonlinearity– Omitted variable bias – Confounding