the chicago guide to writing about multivariate analysis, 2nd edition. conducting post-hoc tests of...

The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.

Conducting post-hoc tests of compound coefficients using

simple slopes for a categorical by continuous interaction

Jane E. Miller, PhD



Overview

• Simple slopes defined• Application of simple slopes to interactions• Calculation of standard errors for simple slopes• Charts to show conclusions of simple slope tests for

interactions



Post-hoc probing

• Post-hoc (“after the fact”) probing is a way to conduct formal statistical tests of differences that were not specified for a priori inferential testing, e.g., – Comparisons other than against the reference category

• See podcast on testing statistical significance of differences across coefficients (s)

– Those involving more than one , e.g., interactions

• Can be conducted from output that is easily obtained from standard software packages

• Does not require respecifying the model


Simple slopes calculation for compound coefficients

• The simple slopes technique is a way to calculate the point estimate of effect size for a combination of two coefficients from the same model

• The point estimate of effect size for each of those combinations is a compound coefficient– A linear combination of two is

• Models involving main effects and interaction terms require summing more than one to obtain the overall effect of the variables involved in the interaction


Review: Calculation of overall effect of an interaction

• The predicted value of the dependent variable from a model with a two-way interaction between predictors X1 and X2 can be written as a function of the estimated coefficients βi:

Y = β0 + β1 X1 + β2X2 + β3 X1_X2

• Rearranging terms to group those that involve the focal predictor (X1 ) yields:

(β0 + β2 X2) + (β1 +β3X2) X1 ,simple intercept ώ0 simple slope ώ1


Review: Focal and modifier variables in an interaction

• In a statistical interaction between two independent variables X1 and X2 – X1 is sometimes referred to as the focal predictor

– X2 is called the moderator or modifier variable because it alters the association between X1 and Y

• Identifying which of the IVs in an interaction is the modifier depends on the research question– E.g., race/ethnicity is specified as the modifier because it is

hypothesized to change the association between education and birth weight


Standard errors for simple slopes

• Both the simple intercept and the simple slope are compound coefficients– Each is a linear combination of two estimated is

• Simple slope: ώ0 = β0 + β2 X2

• Simple intercept : ώ1 = (β1 +β3X2) X1

• The inferential statistical tests for a compound coefficient require calculating the standard error of the simple slope using the estimated variances and covariances for those iss.e. (ώ1 | X2) = √ [var(β1) + (2 × X2× cov(β1,β3)) + (X2)2× var(β3)]


Calculating the standard error for a compound coefficient

• s.e. (ώ1 | X2) is the standard error of the simple slope ώ1

s.e. (ώ1 | X2) = √ [var(β1) + (2 × X2× cov(β1β3)) + (X2)2× var(β3)]

• Where var(j) and var(k) are the variances of j and k, respectively

cov(j, k) is the covariance between j and k

• The variance-covariance matrix can be requested as part of a regression command

• Note that s.e. (ώ1 | X2) depends on the value of the moderator variable in the interaction, X2


Formulas to calculate difference in birth weight by race for selected values of IPR

Income-to-poverty ratio (IPR)

Difference in birth weight compared to NHW with IPR = 0.0

Non-Hispanic white Non-Hispanic black0 = (0 × βIPR) = 0 = βNHB + (0 × (βIPR + βNHB_IPR)) = βNHB

1 = (1× βIPR) = βNHB + (1 × (βIPR + βNHB_IPR)) …4 = (4 × βIPR) = βNHB + (4 × (βIPR + βNHB_IPR))

• More than one β is involved in calculations of the overall effect for any subgroup that is not in the reference category for either IV in the interaction– In this case when race is non-Hispanic black and IPR > 0


Example: Calculation of the standard error of the simple slope for one race_IPR category

Variance-covariance matrix for the estimated coefficients (βs)

βNHB βIPR_NHB

βNHB 729.7βIPR_NHB –276.4 177.1

• In this example, NHB is X1, coded

1 = non-Hispanic black 0 = other racial/ethnic groups

IPR is X2, a continuous variable with values ranging from 0 to 5 E.g., when IPR = 3,

s.e. (ώ1 | IPR) = √ [var(βNHB) + (2 × IPR × cov(βNHBβIPR_NHB)) + (IPR)2× var(βIPR_NHB)]

s.e. (ώ1 | IPR) = √ [729.7 + (2 × 3× (–276.4)) + (3)2× 177.1] = 25.8


Predicted birth weight with 95% CI,by race and income-to-poverty ratio (IPR)

Confidence intervals can be evaluated re whether they overlap this value: predicted birth weight for non-Hispanic whites at IPR = 0


Which contrasts are possible with non-Hispanic white, IPR = 0 as the reference category

Predicted birth weight (grams), by race/ethnicity and income-to-poverty ratio (IPR)

IPRNon-Hispanic

whiteNon-Hispanic

blackMexican American

0.0 3,106 2,929 3,108

0.5 3,118 2,938 3,108

1.0 3,130 2,947 3,109

1.5 3,141 2,956 3,110

2.0 3,153 2,965 3,110

… … … …

4.0 3,200 3,002 3,113

• The circled cell is the reference category (Non-Hispanic white, mother’s education > HS)

• Yellow-shaded cells can be compared to the reference category based on the standard errors of the associated main effects terms alone

• Green-shaded cells can be compared to the reference category using standard errors calculated using the simple slope for a compound coefficient


Ballpark assessment of other contrasts• If there is substantial overlap between confidence

intervals for two values, you can usually safely conclude that they are not statistically significantly different from one another– It is very unlikely that taking the covariance between the

two βs into account when computing the standard error of their simple slopes would alter that conclusion

• If the confidence intervals do not overlap or overlap only slightly, formally test the statistical significance of that difference by respecifying the model with a different reference category– See the podcast on that topic


Online calculator for simple slopes• Preacher has created an online calculator to compute

simple intercepts, simple slopes, and their standard errors from regression output– Coefficients (βs)– Variances and covariances of the βs– Values of the moderator variable for which to calculate

standard errors of the simple slope– Degrees of freedom for the model

• Can also graph the shape of the interaction with confidence intervals, given values of the focal predictor and moderator variables

• See http://www.quantpsy.org/interact/index.html


Presenting results of post-hoc tests• In methods section, mention use of simple slopes technique to

calculate standard errors for compound coefficients.– Provide citation to that method

• Create a table to present the is, standard errors, and goodness-of-fit statistics from the multivariate model

• Conduct post-hoc tests for your substantive hypotheses behind the scenes – Calculate the simple slope– Calculate the standard error of the simple slope

• Create a table or chart to report predicted values and confidence intervals calculated from is and standard errors– Use symbols or prose to convey which contrasts other that against the

reference category are statistically significantly different from one another


Summary• Overall pattern of an interaction involves calculation of

compound coefficients• Standard errors for compound coefficients can be

calculated using the simple slope technique based on variance-covariance matrix from regression output

• Comparison of predicted values of the dependent variable are against the reference category, with all continuous independent variables set to 0– To conduct contrasts for other values of IVs in the interaction

• Respecify the model with different reference categories• Use the all-interaction-dummies approach

• Conduct calculations behind the scenes, present the conclusions of those tests in the text


Separate podcasts


Suggested resources• Cohen, Jacob, Patricia Cohen, Stephen G. West, and Leona S. Aiken.

2003. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, 3rd Edition. Florence, KY: Routledge, chapters 7, 8, and 9.

• Figueiras, Adolfo, Jose Maria Domenech-Massons, and Carmen Cadarso. 1998. Regression Models: Calculating the Confidence Interval of Effects in the Presence of Interactions. Statistics in Medicine 17: 2099–2105.

• Preacher, Kristopher J., Patrick J. Curran, and Daniel J. Bauer. 2006. “Computational Tools for Probing Interaction Effects in Multiple Linear Regression, Multilevel Modeling, and Latent Curve Analysis.” Journal of Educational and Behavioral Statistics.31: 437–448.

• Miller, J. E. 2013. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. University of Chicago Press, chapter 16.



Suggested online resources

• Preacher, Kristopher J. 2011. “Probing Interactions in Multiple Linear Regression, Latent Curve Analysis, and Hierarchical Linear Modeling: Interactive Calculation Tools for Establishing Simple Intercepts, Simple Slopes, and Regions of Significance.”

• Available online at http://www.quantpsy.org/interact/index.html


Suggested online resources, cont.• Podcasts on

– Calculating the overall shape of an interaction from OLS coefficients

– Approaches to testing statistical significance of interactions– Using alternative reference categories to test statistical

significance of interactions


Suggested practice exercises

• Study guide to The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.– Question #5 in the problem set for chapter 16– Suggested course extensions for chapter 16

• “Reviewing” exercise #2• “Applying statistics and writing” exercise #2



Contact information

Jane E. Miller, [email protected]

Online materials available athttp://press.uchicago.edu/books/miller/multivariate/index.html


the chicago guide to writing about multivariate analysis, 2nd edition. conducting post-hoc tests of...

Documents