Lucy Hives

PS4700 Structural Equation Modelling Assignment

1. What were the stated aim(s) of this study?

The aim of the research was to fill in the gaps of previous literature, by conducting a longitudinal study, to investigate the interaction between loneliness, depressive symptoms and suicide ideation, in an adolescent population. The researchers tested 5 hypotheses, which were: a) ‘loneliness is a correlate of depressive symptoms, independent of demographics, multiple psychosocial variables, and social desirability’, b) ‘Loneliness predicts depressive symptoms across time, independent of initial depressive symptoms, later loneliness, and demographics’, c) ‘depressive symptoms predict loneliness across time, independent of initial loneliness, later depressive symptoms, and demographics’, d) ‘loneliness is not a correlate of suicide ideation, independent of depressive symptoms’, and e) ‘loneliness does not predict suicide ideation across time, independent of depressive symptoms’.

2. What was the relation of theory to the aims of the analysis?

The investigators explain that previous studies, having mostly included only two out of the three constructs of loneliness, depressive symptomatology and suicide ideation, have either lacked investigation into an adolescent population (Hagerty and Williams, 1999; Jackson and Cochran, 1991; Cacioppo et al., 2006a; Alpass and Neville, 2003), or yielded contrasting findings (Roberts et al., 1998; Rich and Bonner, 1987; Rich et al., 1992). Also, those which have found links between all three constructs in adolescents have failed to provide longitudinal data, (Garnefski et al., 1992; Roberts et al, 1998), leading the investigators of the present study to question the links between the three constructs over time.

Hypothesis a) is based on the expected reciprocal relationship between loneliness and depressive symptoms. This hypothesis is grounded in previous research and theory, with loneliness having been recognised as a correlate of depressive symptoms for a long time. Research like, for example, Mahon and colleagues’ (2006) meta-analysis reports that the size of the relationship between loneliness and depressive symptoms is large, but despite this, loneliness and depression are still regarded as two distinct constructs (Cacioppo et al., 2006b; Weeks et al., 1980). Previous studies (Hagerty and Williams, 1999; Jackson and Cochran, 1991; Alpass and Neville, 2003) having controlled for a range of factors, have found loneliness to be a specific correlate of depressive symptoms, leading the Lasgaard and colleagues to go on to hypothesise that the reciprocal relationship between loneliness and depressive symptoms will occur independent of demographic, psychosocial and social desirability variables.

Hypothesis b) predicts that initial loneliness will predict later depressive symptoms, as this has been found in many studies with different participant samples (Rotenberg et al., 2004; Koenig and Abrams, 1999; Bonner and Rich, 1991; Joiner, 1997; Heikkinen and Kauppinen, 2004). In addition, some of these studies satisfy the rationale for hypothesis c), because they conclude that depressive symptoms can predict later loneliness. There have, however, been a lack of and conflicting findings for the effect of controlling for cross-

Lucy Hives

sectional relations. Despite this, both hypothesis b) and c) predict that these relationships will be found independent of cross-sectional relations.

Hypothesis d) predicts that loneliness is not a correlate of suicide ideation, mainly taken from the fact that previous research into these two constructs has yielded mixed findings. On one hand, Roberts and colleagues (1998) found loneliness to be a predictor of suicide ideation in adolescents, even when controlling for variables such as depression, life-time suicide attempts, demographics and mental health. However, in contrast, Rich and colleagues (1992) found that loneliness was not a predictor of suicide ideation in another study of adolescents, when controlling for the effects of depression, hopelessness, substance abuse, and few reasons for living. Hypothesis e) suggesting that loneliness will not predict suicide ideation over time, is based on the mixed findings of cross-sectional studies, and the lack of longitudinal research into loneliness and suicide ideation.

3. How did the authors select the variables for inclusion?

The investigators selected the variables for inclusion based on existing research and theory. It has been found that there is a link between loneliness and depressive symptoms in adolescents across time (Koenig and Abrams, 1999; Rotenberg et al., 2004) which therefore provides a rationale for the inclusion of these two test variables. The third, suicide ideation, has been found to be associated with higher levels of loneliness and depressive symptoms (Garnefski et al., 1992; Roberts et al., 1998). The control variables selected to be measured are those which have been commonly associated with adolescent loneliness in previous research (e.g. by Mahon et al, 2006), for example anxiety, perceived stress, social support and network orientation)

4. What latent variables were used?

The latent variables were loneliness, which was measured using the UCLA Loneliness Scale (Lasgaard, 2007), depressive symptoms measured using the Beck Depression Inventory for Youth (Thastum et al, 2009), and suicide ideation measured using the Suicide Ideation Subscale from the Suicide Probability Scale (Cull and Gill, 1988). Measures of anxiety, social support, perceived stress, network orientation, and social desirability were also latent variables.

5. The authors parcel the items rather than use all items of each questionnaire as indicator variables of that construct. Using the Little et al. (2002)1 paper, consider whether parcelling items was appropriate in the current study.

It would seem from Little and colleagues (2002) paper that there are a good deal of arguments for the use of parcels, which rely heavily on the disadvantages of using item level data as indicator variables. The use of item level indicator variables has many disadvantages, for example models may have lower reliability, and it is much more likely that correlations in the data would occur by chance alone. Compared with item level data, parcels can help to

Lucy Hives

reduce sampling error, are more representative of a construct under investigation, and they are more reliable and efficient during analysis.

Despite the advantages of parcelling items there are assumptions which need to be met before its use, the first of these being dimensionality. Analysis may become problematic if parcelling is used with multi-dimensional constructs. However, due to the constructs of loneliness, depressive symptoms and suicide ideology being measured in a unidimensional manner, the dimensionality is therefore not a problem in this case. A second assumption the question of why the analysis is being done. It is advised that if the focus of the analysis is on the relations among latent variables, which in Lasgaard’s (2010) study it is, then parcelling is accepted because indicators are only used as tools to construct the measurement model of a latent construct. However if the analyses were to explore and fully understand the relationships among variables then parcelling would make it difficult to understand the full extent of the data.

Parcelling has also been found to improve model fit, though Little warns that model fit will improve, even in those models which are not correctly specified. This would lead to a reduction in ability to identify misspecified models and therefore lead to a type two error. It is therefore important that researcher who use parcelling, remove hidden sources of error before carrying out their analysis, therefore avoiding misspecification.

Lasgaard and colleagues use random assignment as a method of constructing parcels rather than using all items of each questionnaire as indicator variables of each construct (loneliness, depressive symptoms and suicide ideation). According to Little and colleagues (2002), parcelling items in a random fashion should, on average, lead to parcels which have approximately equal common factor variances. Little explains that the items should each be measured in the same way, and Lasgaard’s study conforms to this with loneliness, depressive symptoms and suicide ideology items each being measured on 4-point likert scales

In conclusion, parcelling was appropriate to be used in the current study due to the unidimensionality of the constructs under investigation, the aim of the analysis being to discover relations among latent variables, and the random assignment of items to parcels resulting in equal common factor variances.

Lucy Hives

6. What did the final model look like (please draw this). Remember to include any error terms [both disturbance and/or measurement] in your diagram (these may not appear in the paper, but they are important – see Tabachnick and Fidell, 2007) and all concurrent associations.

7. Which statistical software was used for the analysis?

The statistical software used for analysis was LISREL (Linear Structural RELationships) version 8.54.

8. In the paper, Lasgaard et al. use a cross-lagged panel design and test the different models as they add certain paths to the model. Do the authors provide a table summarising the model fits at each stage? If so, comment on its usefulness.

The authors provide a table summarising the model fits for models A through H, with the modified version of model H and model H with demographic predictors added, having the same results as model H. As part of structural equation modelling, the researcher must evaluate how well a model fits the sample data. There are a range of fit indices which have been developed, however they, by no means, result in the same evaluation of a given model. For example, in table 6, model A is found to be a good fit of the data by the RMSEA, the IFI and the CFI, but not by the Satorra-Bentler scaled chi-square and the ECVI. The table of fit indices is therefore useful as it shows the progression of how well each model fits the data, however, the large array of measures available often complicates model evaluation.

Perhaps the most reliable of the fit indices reported are the RMSEA and the CFI. The RMSEA, is regarded as ‘one of the most informative fit indices’ (Diamantopoulos and Siguaw, 2000) and the Comparative Fit Index takes into account sample size performing well even when sample size is small (Tabachnick and Fidell, 2007).

Lucy Hives

9. Which model fit indices were used?

The model fit indices which were used included the Satorra-Bentler scaled Chi-square, the Incremental Fit Index, the Comparative Fit Index, the Root Mean Square Error of Approximation, and the Expected Cross Validation Index.

10. Are the model fit indices interpreted correctly for the final model, where non- significant cross-lagged paths are excluded from the model?

The researchers exclude the non-significant cross-lagged paths from Model E to produce Model H. The fit indices table (table 6) shows that Model E and Model H do not differ significantly in terms of their model fit, and the researchers therefore adopt Model H as their final model. The model fit indices are interpreted correctly for the final model, however it is model B which would seem to be a better fit of the data than model H.

Firstly, a non-significant Santorra-Bentler scaled chi-square value would represent an acceptable model fit, however the data is significant for model H meaning that the chi-square statistic indicates that model H is a bad fit of the data. For model B, however, the chi-square value is not significant, indicating that model B is an acceptable model fit. A RMSEA value of 0.06 or less is considered to indicate a good fit and the value reported in the table for model H is 0.06, indeed showing a good fit of the model to the data. Values greater than .95 for the IFI and CFI indexes indicate that the model is an acceptable fit of the data, and table 6 shows that this is the case for model H with an IFI value of 0.97 and a CFI value of 0.97. Lastly the smallest ECVI value in the table indicates the best fitting model, and with a value of 0.92, model H is found not to be as good a fit as model B. In fact, model B is shown to be an acceptable or good fit for all of the fit indices in table 6.

11. Were statistical tests carried out to ensure the assumptions for SEM were met? If so, what did they find and what does this mean? If no mention is made of statistical assumptions having been met or tested, what should have been included?

As far as can be seen, the researchers did not ensure that all of the assumptions for carrying out SEM were met. Assumptions of normality, missing data, sufficiently large sample size, and correct model specification should be met before carrying out Structural Equation Model analysis.

The first assumption ensures that the data gained from participants has a normal distribution If the distribution of data is not normal then this has have a big impact on the validity of subsequent statistics, for example the maximum likelihood estimation, standard errors which would be underestimated and tests of model fit which would be overestimated. Lasgaard and colleagues should have included histograms to show the distribution of the data and should have also conducted a goodness of fit test.

Lasgaard and colleagues do screen their data for errors prior to data analysis and therefore satisfy assumption two, that there are no missing values in the data set. They

Lucy Hives

describe the percentage of missing values as acceptable, however it is unclear as to whether the missing values are missing completely at random (MCAR) or not. If so, then the researchers are justified in using the Expectation Maximization algorithm method for dealing with the missing data. However if the data is not MCAR then the researchers should consider removing these cases.

Both numbers of participants at time one (1009) and time two (541) are sufficiently large, however Lasgaard and colleagues make no reference to the fact that this satisfies one of the assumptions of Structural Equation Model analysis.

The final assumption is that the model is correctly specified. Lasgaard and colleagues specify their model, in figure 1, in accordance to their hypotheses, however this only includes the structural part of the diagram (not the measurement part) and also leaves out the demographic information.

12. What are the limitations of the SEM analysis carried out here?

Lasgaard and colleagues explain in their discussion that their SEM analysis has three main limitations or flaws. Firstly, at time 1, there were 1009 participants who took part in the study, whereas later at time 2 this amount dropped significantly to 541, a decrease of 54%. This high rate of attrition affects both the internal and external validity of the analysis and therefore the longitudinal data is difficult to interpret correctly. The researchers explain that the attrition rate is even more disturbing because the students who did not take part in the study at time 2 (due to non-attendance or dropping out of the course) had higher levels of symptoms of loneliness, depression and suicide ideation at time 1.

The researchers also found that their results didn’t reflect the previous findings of other studies, in the case of the symptoms of their sample being less severe. However it is explained that those studies with high symptom severity were conducted in clinical samples and therefore one would expect these results.

The final limitation of this study is that loneliness was identified as one construct, when there are indeed different types of loneliness. As is highlighted in text, Goossenns and colleagues (2009) explain that there are different types of loneliness, and some of which have a greater impact upon adolescents than others, for example peer-related loneliness. Perhaps it is the case then that the construct of loneliness should have been broken down into these different types.

13. What were the main conclusions from the SEM analyses? Include in your answer whether you consider it difficult to make conclusions or offer practical guidance based on the findings.

It is difficult to make conclusions from the data because the researchers have not drawn out the final model or included any of the parameter values. Also the model fit indices table shows model B to be the best fit of the data and so it is unclear as to why this model was not chosen as the final model. The main conclusions are drawn from the modified version of model H and these either support or disconfirm the hypotheses.

Lucy Hives

Firstly hypothesis A was confirmed due to the researchers finding that loneliness was correlated with depressive symptoms, when controlling for gender and other demographic factors, multiple psychosocial variables and social desirability. Hypothesis B was disconfirmed because Loneliness did not predict a difference in depressive symptoms across time, independent of initial depressive symptoms, subsequent loneliness and demographics. Hypothesis C was confirmed with higher levels of depressive symptoms predict higher loneliness in adolescents across time independent of cross-sectional relations and time effects. Hypothesis D was only half confirmed with loneliness being correlated with suicide ideation, however there is disconfirmation that this correlation occurs independent of depressive symptoms at the cross-sectional level. Hypothesis E was confirmed because loneliness did not predict differences in suicide ideation across time independent of depressive symptoms, cross-sectional relations and time effects.

14. If you wanted to repeat the authors’ SEM analyses, outline the important considerations you would need to take into account.

There are 5 steps which need to be considered when conducting structural equation model analysis. There are: specification, identification, estimation, testing, and modification.

Firstly, specification is important and involves making conclusions from previous relevant research so that a theoretical model can be drawn. The impact of the demographic factors on the latent variables also needs to be controlled for and so these should have been added to the theoretical model, with the use previous theory and research. It is important that only the important factors are included in the model, as failure to include important factors or exclude unimportant factors can lead to the inaccuracy of estimations and the model being a bad fit of the data. Also important in include in this initial model, is the measurement part of the model, if there are any variables which are latent (in this case the latent variables are loneliness, depressive symptoms and suicide ideology). The measurement pat of the model should indicate which items measure which of the latent variables and their relationships should be clearly defined.

The second stage is model identification. It is not clear from the research paper whether Lasgaard and colleagues identified their model. This includes identifying whether the parameters of the model can be solved uniquely, or not. There are three levels of identification: underidentified, if any of the parameters cannot be solved uniquely; just-identified, when there is just enough information to solve the parameters; and over-identified, when there is more than one way to solve the parameters.

Model estimation is the third step, which usually involves calculating the maximum likelihood estimation. This step is characterised by repeated attempts to obtain estimates of parameters that result in the “best fit” of the model to the data, and this is indicated using goodness of fit statistics in the 4th stage.

Model testing takes place next, and its aim is to test how well the model fits the data. Lasgaard and colleagues do present a table of model fit data, although there is no table of parameter estimates or their statistical significances. It would be useful to provide a table showing the parameter estimates and significance levels, rather than just dismissing these from the model because they are not significant.

Lucy Hives

The final stage is model modification. Lasgaard and colleagues practice this step when they removed the non-significant path from model E (depressive symptoms did not predict suicide ideation across time) in creating model H, although there was no significant difference between the model fit of Model E and Model H. it is also important to note that the fit indices for the variations of model H need to be recorded, because this is important information.

Further considerations include a test of the normality of the data before the analysis takes place and also the researchers need to justify their use of parcelling.

References:

Alpass, F.M. & Neville, S. (2003). Loneliness, health and depression in older males. Ageing and Mental Health, 7, 212-216

Bonner, R.L. & Rich, A.R. (1991). Predicting vulnerability to hopelessness: a longitudinal analysis. Journal of Nervous and Mental Disease, 179, 29-32.

Cacioppo, J.T., Hughes, M.E., Waite, L.J., Hawkley, L.C., & Thisted, R.A. (2006a). Loneliness as a specific risk factor for depressive symptoms: cross-sectional and longitudinal analyses. Psychology and Aging, 21, 140-151.

Cacioppo, J.T., Hawkley, L.C., Ernst, J.M., Burleson, M., Berntson, G.G., Nouriani, B., et al. (2006b). Loneliness within a nomological net: an evolutionary perspective. Journal of Research in Personality, 40, 1054-1085.

Cull, J.G., & Gill, W.S. (1998). Suicide probability scale (SPS) manual. Los Angeles: Western Psychological Services.

Garnefski, N., Diekstra, R.F., & de Heus, P. (1992). A population-based survey of the characteristics of high school students with and without a history of suicidal behaviour. Acta Psychiatrica Scandinavica, 86, 189-196.

Goossens, L., Lasgaard, M., Luyckx, K., Vanhalst, J., Mathias, S., & Masy, E. (2009). Loneliness and solitude in adolescence: a confirmatory factor analysis of alternative models. Personality and Individual Differences, 47, 890-894.

Hagerty, B.M., & Williams, A.R. (1999). The effects of sense of belonging, social support, conflict and loneliness on depression. Nursing Research, 48, 215-219.

Heikkinen, R.L., & Kauppinen, M. (2004). Depressive symptoms in late life: a 10-year follow-up. Archives of Gerontology and Geriatrics, 38, 239-250.

Jackson, J., & Cochran, S.D. (1991). Loneliness and psychological distress. Journal of Psychology, 125, 257-262.

Joiner, T.E. (1997). Shyness and low social support as interactive diatheses, with loneliness as mediator: testing an interpersonal-personality view of vulnerability to depressive symptoms. Journal of Abnormal Psychology, 106, 386-394.

Lucy Hives

Koenig, L.J., & Abrams, R.F. (1999). Adolescent loneliness and adjustment: a focus on gender differences. In K.J. Rotenberg & S. Hymel (Eds.), Loneliness in childhood and adolescence (pp. 296-322). New York: Cambridge University Press.

Lasgaard, M. (2007). Reliability and validity of the Danish version of the UCLA loneliness scale. Personality and Individual Differences, 42, 1359-1366.

Lasgaard, M., Goossens, L., & Elkit, A. (2010). Loneliness, depressive symptomatology, and suicide ideology in adolescence: cross-sectional and longitudinal analyses. Journal of Abnormal Child Psychology, 39, 137-150.

Little,T.D., Cunningham, W.A., & Shahar, G. (2002). To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits. Structural Equation Modelling, 9, 151-173.

Mahon, N.E., Yarcheski, A., Yarcheski, T.J., Cannella, B.L., & Hanks, M.M. (2006). A meta-analytic study of predictors for loneliness during adolescence. Nursing Research, 55, 308-315.

Rich, A.R., & Bonner, R.L. (1987). Concurrent validity of a stress-vulnerability model of suicidal ideation and behaviour. Suicide and Life-Threatening Behaviour, 17, 265-270.

Rich, A.R., Kirkpatrick-Smith, J., Bonner, R.L., & Jans, F. (1992). Gender differences in the psychosocial correlates of suicide ideation among adolescents. Suicide and Life Threatening Behaviour, 22, 364-373.

Roberts, R.E., Roberts, C.R., & Chen, Y.R. (1998). Suicidal thinking among adolescents with a history of attempted suicide. Journal of the American Academy of Child and Adolescent Psychiatry, 37, 1294-1300.

Rotenberg, K.J., McDougall, P., Boulton, M.J., Vaillancourt, T., Fox, C., & Hymel, S. (2004). Cross-sectional and longitudinal relations among peer-reported trustworthiness, social relationships, and psychological adjustment in children and early adolescents from the United Kingdom and Canada. Journal of Experimental Child Psychology, 88, 46–67.

Thastum, M., Ravn, K., Sommer, S., & Trillingsgaard, A. (2009). Reliability, validity and normative data for the Danish Beck Youth Inventories. Scandinavian Journal of Psychology, 50, 47–54.

Weeks, D.G., Michela, J.L., Peplau, L.A., & Bragg, M.E. (1980). Relation between loneliness and depression: a structural equation analysis. Journal of Personality and Social Psychology, 39, 1238–1244.