Identifying vulnerability in grief: psychometric propertiesof the Adult Attitude to Grief Scale

Julius Sim • Linda Machin • Bernadette Bartlam

Accepted: 1 October 2013 / Published online: 16 October 2013

� Springer Science+Business Media Dordrecht 2013

Abstract Purpose Grief is a reaction to a significant loss

that can profoundly affect all aspects of life and capacity to

function well. The consequences can vary from severe psy-

chological distress through to physical disturbances and sig-

nificant social problems. This study sought to identify a

measure of vulnerability in grief, by examining the psycho-

metric properties of the Adult Attitude to Grief (AAG) scale in

a sample of 168 people seeking help in their bereavement.

Methods The factor structure of the scale, its internal

consistency, its construct validity and optimum classifica-

tion cutoffs were tested.

Results Confirmatory factor analysis broadly supported the

factor structure of the AAG, but identified one item that

could profitably be reworded. Internal consistency of the

three subscales was acceptable. Construct validity and

discriminative validity were supported by correlations with

allied constructs (depression and anxiety) and a significant

difference between scores for clients with Prolonged Grief

Disorder and those without. A correlation with counsellors’

own clinical ratings of vulnerability demonstrated crite-

rion-related validity of the AAG. Using receiver operating

characteristic methods, optimum cutoff scores on the scale

were identified for the classification of different levels of

vulnerability.

Conclusion The AAG was found to be a psychometrically

promising tool for identifying vulnerability in grief.

Keywords Adult Attitude to Grief scale �Vulnerability � Psychometrics � Validity � Reliability �Factor analysis

List of symbols

AAG Adult Attitude to Grief scale

CFI Comparative fit index

GAD-7 Generalized Anxiety Disorder Assessment 7

PG-13 Prolonged Grief Disorder Scale

PGD Prolonged grief disorder

PHQ-9 Patient Health Questionnaire 9

RMSEA Root means square error of approximation

ROC Receiver operating characteristic

TLI Tucker–Lewis Index

WLSMV Weighed least squares mean and variance

adjusted

Introduction

There were 484,367 deaths registered for England and

Wales in 2011 [1]. While some people demonstrate resil-

ience in response to traumatic or disturbing life events,

quality of life can be seriously impaired for many bereaved

people, and it is estimated that 10–15 % of the general

bereaved population are susceptible to complicated grief [2].

Certain individuals are therefore vulnerable in their grief,

and identifying this group is a practice and research

imperative, both because intervening inappropriately can

cause harm [3, 4] and because of the need to use increas-

ingly scarce health and social care resources effectively [5].

Identifying the components of complicated grief remains

a challenge for both researchers and practitioners. Foremost

in this field of research have been Prigerson and Macie-

jewski [6], who have produced a definition of prolonged

grief disorder (PGD) that is now proposed as an officially

recognized psychiatric disorder [7]. It is worth noting that,

while other psychiatric conditions such as depression and/or

anxiety have been regarded as components of complicated

J. Sim (&) � L. Machin � B. Bartlam

Research Institute for Social Sciences, Keele University,

Staffordshire ST5 5BG, UK

e-mail: j.sim@keele.ac.uk

123

Qual Life Res (2014) 23:1211–1220

DOI 10.1007/s11136-013-0551-1

grief, they do not provide a complete definition of the

complex spectrum of grief responses that may make a per-

son vulnerable in their bereavement.

The Range of Response to Loss (RRL) model has been

developed to explain individuals’ responses to grief [8].

The characteristics of grief articulated within the RRL

model became evident in a study of 97 bereaved people [9],

and the pattern was confirmed in clinical practice. Initially,

three categories of response were identified: overwhelmed,

controlled and balanced [8, 10]. Reflecting the language

used by clients, the RRL model was found to resonate with

practitioners and provides clear conceptual links into their

work, while at the same time having parallels with other

key theories (Table 1).

The Adult Attitude to Grief scale (AAG) scale was

devised to reflect the categories in the RRL. In the scale,

three items reflect qualities associated with being over-

whelmed—stressfulness, irreversibility and loss of mean-

ing in life [15]; three items represent control—restricted

acknowledgement of distress, exaggerated need for self-

reliance and avoidance of grief through overly engaging in

day-to-day life [15]; and three items reflect balance (later

defined as resilience)—courage, resourcefulness and opti-

mism [16, 17]. It should be noted that while the concept of

‘control’ might usually be seen as an appropriately adap-

tive response to loss in the context of clinical practice, here

it is conceptualized as a potentially problematic feature in

which loss disrupts the usual controlling coping mecha-

nisms. The nine items are each scored on a five-point

Likert scale from ‘strongly agree’ to ‘strongly disagree’.

On the basis of a previous exploratory factor analysis [8],

these items provided support for the three dimensions:

overwhelmed (items 2, 5, and 7), controlled (items 4, 6 and

8) and resilient (items 1, 3 and 9). The research also sug-

gested that the scale could be used to provide an individual

profile of grief based on the relative presence or absence of

the overwhelmed, controlled and balanced/resilient

components reported by respondents [8]. Subsequent

studies [18, 19] have supported the practical usefulness of

the AAG scale to achieve a picture of individual grief and

its changes overtime. The findings from those studies also

indicated the value of practitioners using the self-report

statements as prompts in exploring clients’ experiences of

loss in greater depth and in determining the direction of

therapeutic intervention.

Further theoretical debate about the interactive dynamic

between stressors and coping as the basis for understanding

individual adjustment to bereavement [20] and practice-

based reflection have helped to clarify the functioning of

the elements within the RRL model. As a result of this

process, it was clear that the model is describing two dis-

tinct dimensions in response to loss. The first of these

consists of the core grief impact (stressor), producing

variable overwhelmed and controlled reactions. These

states are not by definition evidence of vulnerability but are

dependent on the second dimension, that of coping, in

which the capacity to respond to grief with equilibrium is

characteristic of resilience, while a limited capacity to

manage grief is characteristic of vulnerability. These

reflections have resulted in vulnerability being included as

a fourth component in the RRL model.

Using this theoretical rationale, it was proposed that the

fourth component of the RRL may be calculated using the

AAG scale. Specifically, it was hypothesized that vulner-

ability is a higher-order component directly related to

scores on the overwhelmed and controlled subscales and

inversely related to the score on the resilient subscale. The

rationale for this proposition is that the overwhelmed and

the controlled items on the scale represent a spectrum of

core grief reactions, while the resilient items indicate a

capacity to cope with the consequences of grief. By off-

setting the stressful elements of grief against the positive

mechanisms of coping with grief, an emergent picture of

relative vulnerability can potentially be drawn from the

AAG scale. Hence, an indication of vulnerability can be

derived by summating those items on the scale that are

indicative of vulnerability, i.e. the overwhelmed and con-

trolled items, and discounting the mediating resilient items.

Scores are recorded on a vulnerability rating grid on which

the resilient scores are reversed to allow for simple addition

of all the nine items in the AAG scale (Table 2). The

resulting range of possible scores is from 0 to 36, with

higher scores indicating greater vulnerability.

The aim of this study was to evaluate the AAG scale as a

measure to identify vulnerability, by examining its psy-

chometric properties and determining its potential utility as

a tool to calculate the degree of vulnerability in people

presenting for support in their loss, as an indicative guide

to intervention. Specifically, the research sought to test four

key psychometric properties of the AAG scale:

Table 1 Conceptual comparisons between the range of response to

loss model and other related theories

Other

theories

Range of response to loss model

Overwhelmed Resilient Controlled

Attachment

theory [11]

Anxious/

ambivalent

Secure Avoidant

Stress theory

[12]

Intrusion Avoidance

Dual process

model [13]

Loss

orientation

Oscillation Restoration;

orientation

Personality-

related

theory [14]

Intuitive

grief:

emotional

coping

Blended grief:

emotional and

cognitive coping

Instrumental

grief:

cognitive

coping

1212 Qual Life Res (2014) 23:1211–1220

123

• its factor structure

• its internal consistency

• its construct and criterion-related validity and

• optimum cutoffs for classification.

Methods

Sampling

Following ethical approval, participants were recruited

from three hospices and one community-based bereave-

ment service. Those clients accessing the collaborating

services—in which the AAG scale was routinely used as a

standard method of exploring their grief—were included in

the study unless they chose to opt out. Sample size was

determined in relation to the proposed CFA analysis. It is

recommended that there should be a minimum of between

5 and 10 cases per estimated parameter [21]. With 20

parameters to be estimated, a sample size of at least 160

was sought, so as to provide an intermediate figure of 8

cases per parameter.

Analysis

To examine the dimensionality of the scale, a factor

structure previously identified through exploratory factor

analysis [8], with the addition of the vulnerability indicator

(Fig. 1), was tested on the data by means of confirmatory

factor analysis (CFA), using Mplus 7 (www.statmodel.

com). A WLSMV estimator was employed, and variables

were designated as ordered categorical. The fit of the

hypothesized factor structure to the data was quantified

using a range of fit indices: the Tucker–Lewis index (TLI),

the comparative fit index (CFI) and the root mean square

error of approximation (RMSEA) [22]. Hu and Bentler [23]

recommend using a combination of such fit indices, as no

single index has been found to be optimal. The three

indices have in common that they are not influenced by

sample size and—in the case of the TLI and the RMSEA—

reward parsimony in the hypothesized factor structure [24].

Each fit index normally produces a value between 0 and 1

(though the RMSEA can produce values above, and the

TLI values both above and below, this range). In the case

of the TLI and the CFI, values close to 1 indicate good fit,

whereas for the RMSEA values close to 0 indicate good fit.

Benchmarks for the fit indices have been proposed by a

number of authors: values on the TLI and the CFI should

ideally exceed .95, and those on the RMSEA should ideally

lie below .05 [25, 26]. Marsh et al. [27] warn, however,

against a mechanistic interpretation of these fit indices. The

v2 statistic for each model was also calculated; while the

associated p value should be non-significant for a well-

fitting model, its magnitude is sensitive to sample size and

is not therefore a reliable indicator of fit [28]. Interpretation

of the statistic is also dependent on the degrees of freedom

of the model, and the normed v2 was therefore also cal-

culated (v2/degrees of freedom); lower values of this sta-

tistic indicate better fit. The v2 statistic also served as a

means of comparing the difference in fit of nested models.

Modification indices and item characteristic curves [29]

were generated to help investigate items in the scale that

appeared to be responsible for any lack of fit.

The dimensionality of the AAG was tested in two

models. In the first model, the nine items were posited to

measure just the three dimensions represented by the sub-

scales: overwhelmed (items 2, 5, and 7), controlled (items

4, 6 and 8) and resilient (items 1, 3 and 9). The second

model stated additionally that these three dimensions

Table 2 The Adult Attitude to Grief Scale shown in its domains and with a scoring system to obtain an indication of vulnerability

Domain AAG items Response options

Strongly

agree

Agree Neither agree

nor disagree

Disagree Strongly

disagree

Overwhelmed 2. For me, it is difficult to switch off thoughts about the person

I have lost

4 3 2 1 0

5. I feel that I will always carry the pain of grief with me 4 3 2 1 0

7. Life has less meaning for me after this loss 4 3 2 1 0

Controlled 4. I believe that I must be brave in the face of loss 4 3 2 1 0

6. For me, it is important to keep my grief under control 4 3 2 1 0

8. I think its best just to get on with life after a loss 4 3 2 1 0

Resilient 1. I feel able to face the pain which comes with loss 0 1 2 3 4

3. I feel very aware of my inner strength when faced with grief 0 1 2 3 4

9. It may not always feel like it but I do believe that I will

come through this experience of grief

0 1 2 3 4

Numbering of the items indicates the order in which they appear in the scale

Qual Life Res (2014) 23:1211–1220 1213

123

together constituted a vulnerability indicator, in the way

described in the description of the AAG scale above and

illustrated in Fig. 1. In addition to an evaluation at the level

of the scale as a whole, dimensionality was examined at the

level of individual items, by calculating polychoric corre-

lations between each item and the three domain scores

(with the item in question excluded from the calculation of

its own domain score). Kline [30] and Nunnally and

Bernstein [31] propose that the correlation of each item

with its own domain should exceed .3. Additionally, one

would expect an item to correlate more strongly with its

own domain than with other domains.

Internal consistency was quantified by calculating an

index of composite reliability for each subscale [32]. This

method generates coefficients that are analogous to Cron-

bach’s alpha and are interpreted in a similar way. A value

of .7 or greater is generally regarded as acceptable [33–35].

The construct validity of the AAG scale was assessed in

two ways. First, Pearson correlations were calculated with

two measures to which, on theoretical grounds, the AAG

should be related: a measure of anxiety, the Generalized

Anxiety Disorder Assessment 7 (GAD-7) [36], and a

measure of depression, the Patient Health Questionnaire 9

(PHQ-9) [37]. These measures have been shown to be

psychometrically sound [38, 39]. Clients with high vul-

nerability scores might be expected to score high on anx-

iety and/or depression; therefore, a correlation between the

AAG scale and each of these measures would support the

construct validity of the AAG scale. Second, the discrim-

inative validity of the AAG was tested using a ‘known

groups’ approach [40] in relation to a classification of PGD

based on the 13-item Prolonged Grief Scale (PG-13) [7]. A

classification of PGD is based on a criterion of more than

6 months since the bereavement and the fulfilment of ten

other criteria; this classification has shown good diagnostic

validity [7]. On theoretical grounds, we would expect cli-

ents with PGD to be more vulnerable than clients without

PGD, and thereby, a significant difference to be found

between the AAG scores for these two groups. This ana-

lysis was restricted to those clients (n = 76) who were

potentially eligible for classification as having prolonged

grief (more than 6 months since bereavement).

Criterion-related validity of the AAG scale was tested in

relation to its correlation (Spearman’s rho) with counsel-

lors’ clinical assessments of vulnerability, which were

available for 162 clients. Counsellors were asked to place

Fig. 1 Hypothesized factor

structure of the Adult Attitude

to Grief Scale (see Table 2 for

item descriptions). The ellipses

denoted by e represent residual

error terms

1214 Qual Life Res (2014) 23:1211–1220

123

the client in one of four categories in response to the

question ‘What level of vulnerability best describes your

client?’ The classifications produced were as follows: low

vulnerability (n = 19, 12 %); moderate vulnerability

(n = 59, 36 %); high vulnerability (n = 76, 47 %); and

severe vulnerability (n = 8, 5 %). These assessments were

made by counsellors without knowledge of a client’s AAG

score.

The counsellors’ clinical assessments of vulnerability

were also used to identify threshold scores on the AAG

scale, so as to establish whether the scale can be used to

generate a classification of vulnerability. Using receiver

operating characteristic (ROC) analysis, a C statistic was

calculated for three dichotomous classifications based on

the boundaries of adjacent categories of vulnerability, i.e.

those between (1) severe and high, (2) high and moderate,

and (3) moderate and low. This statistic represents the area

under the ROC curve and can range from 0 to 1 [40]. A C

statistic of 1 indicates that the scale has the greatest pos-

sible sensitivity and specificity with regard to the dichot-

omous classification concerned (all categorizations of

individuals with respect to this classification are perfectly

accurate), and one of 0 denotes the worst possible sensi-

tivity and specificity (all such categorizations are totally

inaccurate); an intermediate value of .5 indicates that the

scale categorizes no more accurately than guessing. It

follows that the closer the C statistic is to 1, the more

accurately the scale categorizes individuals.

Additionally, specific threshold scores were identified

on the AAG that separate adjacent classifications in the

counsellors’ assessments of vulnerability with the greatest

sensitivity and specificity, using the Youden index (J). This

statistic expresses the maximum difference between sen-

sitivity and 1–specificity (i.e. between the true positive rate

and the false positive rate) for a given cutoff score, on a 0–

1 scale [41]:

J ¼ sensitivity þ specificity�1ð Þ:

For example, a cutoff with a sensitivity .61 of and a

specificity of .72 would yield a Youden index of .61 ? (.72

- 1) = .33. A cutoff score with the largest Youden index

is therefore the one for which sensitivity and specificity for

a particular classification are simultaneously maximized.

Findings

The AAG scale was completed in full by 168 clients: 40

male (23.8 %) and 128 female (76.2 %). Four other clients

had missing data on one or more items in the scale and

were excluded from analysis. The median age category was

46–55 (age was recorded in intervals rather than continu-

ously for reasons of anonymity).

Mean (standard deviation [SD]) scores on the subscales

of the AAG were as follows: overwhelmed 8.92 (2.53);

controlled 7.98 (2.31); and resilient 5.25 (2.46). The mean

(SD) for the total score was 22.15 (4.38). Figure 2 shows

the distribution of values for the domain scores, and Fig. 3

shows the distribution of the total score. The negative skew

of scores on the overwhelmed domain suggests that there

may be a ceiling effect for this subscale. The other domain

scores and the total score exhibit a reasonably symmetrical

distribution.

Factor structure

The results of the CFA are shown in Table 3. The fit

indices do not show a very good fit. From the X2 test for

difference, the model with just the domain scores seems to

fit slightly better than the domain and total score model—

the difference in fit is statistically significantly

(X2 = 14.09. df = 1, p \ .001).

Table 4 shows polychoric correlations of each item with

its own domain and with other domains. Across all items,

with the exception of item 8, correlations with the item’s

own domain are C.442 (exceeding the benchmark value of

.3 [30, 31]), and correlations with other domains are B.280.

Item 8 had a stronger absolute correlation with the resilient

domain (-.357) than with its own controlled domain

(.264). Thus, the pattern of correlations, together with

examination of the modification indices from the CFA

output, suggested that item 8 does not just load on the

controlled domain, as hypothesized in the original model.

To explore this further, item 8 was temporarily re-assigned

to the resilient domain, producing a slight improvement in

fit (Table 5). However, when item 8 was allowed to load on

both the controlled and the resilient domains, fit was

noticeably improved over the original model, and the val-

ues for TLI, CFI and RMSEA were close to the desirable

cutoffs; the improvement in fit over the original model was

significant (X2 = 41.62. df = 1, p \ .001). The perfor-

mance of the item was investigated in more detail through

an item characteristics curve; this plots the probability of

endorsement of an item (in this case, the endorsement of

the highest category on the scale) against the underlying

latent construct (h) on a scale standardized to SD units

(Fig. 4). Such a curve should show a steep incline at a point

corresponding to .5 probability (indicating good discrimi-

nation for the item) and should indicate a probability of

endorsement close to 1.0 at the upper extreme of h (indi-

cating an appropriate threshold on h at which respondents

endorse the highest point on the scale; analogous to

appropriate ‘difficulty’ for an item testing knowledge) [29].

Figure 4 shows that when included in the controlled

domain (solid curve), item 8 has poor discrimination, and

even when h is high (?4 SDs), the probability of

Qual Life Res (2014) 23:1211–1220 1215

123

endorsement of the highest category on the item is only just

over .5. The item’s performance when included in the

resilient domain (dashed curve) is markedly better in these

respects, though not ideal.

Internal consistency

The coefficients of composite reliability, with 95 % con-

fidence intervals (CIs), for the three subscales (as per the

Fig. 2 Distribution of the three subscale scores on the Adult Attitude

to Grief Scale. The dashed vertical line indicates the mean score

Fig. 3 Distribution of total scores on the Adult Attitude to Grief

Scale. The dashed vertical line indicates the mean score

Table 3 Results of the confirmatory factor analysis for the original

hypothesized model

Statistic Domain scores and

total score

Domain scores only

TLI .652 .714

CFI .758 .809

RMSEA .153 .139

X2 123.53 (df = 25) p \ .001 101.84 (df = 24) p \ .001

Normed X2 4.941 4.243

TLI Tucker–Lewis index, CFI comparative fit index, RMSEA root

mean square error of approximation

Table 4 Item polychoric correlations with domain scores

Item Domain

Overwhelmed Controlled Resilient

2 .465 .012 .103

5 .526 .252 .241

7 .475 .017 .256

4 .159 .446 -.167

6 .283 .514 -.045

8 -.105 .264 -.357

1 .149 -.127 .482

3 .184 -.269 .479

9 .231 -.172 .442

An item’s correlation with its own domain is shown in bold

1216 Qual Life Res (2014) 23:1211–1220

123

original model) were as follows: overwhelmed .730 (95 %

CI .689, .771); controlled .683 (95 % CI .638, .728); and

resilient .692 (95 % CI .649, .736). These values either

closely approached or exceeded the benchmark of .7 for

acceptable internal consistency.

Construct and criterion-related validity

As indications of construct validity, the AAG scale had a

correlation of .470 (95 % CI .338, .584; p \ .001;

n = 156) with the PHQ-9 and .442 (95 % CI .308, .559;

p \ .001; n = 160) with the GAD-7. The mean (SD) score

on the AAG for clients who were negative for prolonged

grief (n = 47) on the PG-13 was 21.13 (4.03), and that for

clients who were prolonged grief positive (n = 29) was

24.38 (3.76); the latter group therefore had a score 3.25

scale points higher (95 % CI 1.40, 5.10; p = .001), rep-

resenting 9.03 % of the scale and a standardized difference

of .83 [42]. As regards criterion-related validity, the cor-

relation of the AAG scale with the counsellors’ assess-

ments of vulnerability was .521 (95 % CI .399, .625;

p \ .001; n = 162).

Classification of vulnerability

The ROC analysis generated C statistics of .837 (95 % CI

.743, .932) for a classification of severe vulnerability, .760

(95 % CI .688, .833) for a classification of high or severe

vulnerability and .846 (95 % CI .755, .937) for a classifi-

cation of low vulnerability.

Table 6 shows the results of the ROC analysis in rela-

tion to classification thresholds (cutoff scores). The opti-

mum cutoff between adjacent classifications of

vulnerability is the point on the AAG scale at which the

Youden index is largest. If sensitivity and specificity are

regarded as equally important, the optimum thresholds for

classifying a client are 24 for severe vulnerability, 23 for

high or severe vulnerability and 21 for vulnerability that is

moderate or greater. However, a false negative may be

more serious than a false positive in the diagnosis of vul-

nerability; it is a potentially worse error to withhold pro-

fessional intervention from a client who needs it than to

provide such help to a client who does not need it.

Accordingly, Table 6 also shows the optimum cutoffs

when sensitivity is required to be at least .85; these are

lower for classifications of high or severe vulnerability (21)

and of moderate or greater vulnerability (20).

Table 6 also shows that the Youden index is lower when

classifying clients at the ‘midpoint’ of vulnerability (severe

or high vs. moderate or low), indicating that the scale

produces less clinically reliable classifications at this point;

this reflects the lower C statistic for this classification

threshold.

Discussion

This study supports the view that the AAG provides a

measure of vulnerability capable of wide practice appli-

cation that is distinct from existing measures such as the

PG-13 [7]. The PG-13 asks questions about symptoms of

grief relating to feelings, thoughts and actions, which if

persisting for 6 months or more are seen to be associated

with significant functional impairment. In contrast, the

AAG aims to provide a broader grief profile, in which both

core grief reactions and the coping response to them are

seen to function interactively, providing a way of estab-

lishing to what degree the symptoms of grief are being

Table 5 Results of the confirmatory factor analysis for the domain

and total score model with adjustments to the loading of item 8

Statistic Item 8 loading on

resilient domain

Item 8 loading on both controlled

and resilient domains

TLI .778 .873

CFI .846 .915

RMSEA .122 .093

X2 87.75 (df = 25)

p \ .001

58.55 (df = 24) p \ .001

Normed

X23.51 2.44

TLI Tucker–Lewis index, CFI comparative fit index, RMSEA root

mean square error of approximation

Fig. 4 Item characteristic curve for item 8 in relation to the

controlled domain (solid curve) and the resilient domain (dashed

curve). The horizontal axis represents the latent variable (h) for these

domains. The vertical axis represents probability of endorsement of

the highest category on the five-point scale for the item. Item 8 is

hypothesized to have an inverse relationship with the resilient

domain, but for the purpose of clarity has been reverse-scored so that

the curve lies in the same direction as that for the controlled domain

Qual Life Res (2014) 23:1211–1220 1217

123

managed effectively. The AAG does not seek to identify a

time point at which vulnerability becomes more problem-

atic. Rather, the use of the AAG scale in practice makes

more variable assumptions about individual vulnerability

dependent on wider circumstantial factors, e.g. the death of

a child is known to generate a more extended period of

vulnerability, which is accepted as normal under such

circumstances.

The CFA results suggest that the hypothesized factor

structure of the AAG, with the vulnerability indicator

added, does not provide an optimum fit to the sample data

in this study. Although the pattern of correlations for item

8—positive with the controlled domain and negative with

the resilient domain—is in keeping with the calculation of

the vulnerability score, where the controlled and resilient

domain are hypothesized to be inversely related, its item

characteristic curve suggests that it does not perform well.

It is likely, therefore, that this item would benefit from a re-

evaluation of its wording, so that it loads highly on just one

domain and displays greater discrimination. Other items’

correlations with their own and with other domains support

the dimensionality of the scale.

The internal consistency of the three subscales is

acceptable and is highest for the overwhelmed subscale.

The overwhelmed subscale appears to be subject to a

ceiling effect, such that it may be hard to differentiate

between clients who lie at the upper extreme of this

domain, and some modification of one or more of the items

in this domain may be required—for example, making the

descriptors somewhat stronger so that they would receive

lower average levels of endorsement, thereby shifting the

distribution of the subscale scores downwards.

As regards construct validity, the AGG exhibited sta-

tistically significant ‘medium’ correlations (i.e. C.3; [42])

with measures of depression (PHQ-9) and anxiety (GAD-

7), which reflect the hypothesized theoretical relationships

between vulnerability and these other constructs. The dis-

criminative validity of the scale was also supported by the

findings of a statistically significant difference of ‘large’

magnitude (i.e. C.8 [42]) between the scores of those

clients with prolonged grief disorder and those without.

The ‘large’ correlation (i.e. C.5 [42]) between the AAG

scale and counsellor’s own assessments of vulnerability

provides evidence for the criterion-related validity of the

scale.

Overall, as judged by the C statistic, the AAG scale

performed well when classifying clients’ vulnerability at

the extremes (severe vulnerability vs. lower levels of vul-

nerability or low vulnerability vs. higher levels of vulner-

ability). Optimum cutoff scores on the scale were also

identified for the classification of different levels of vul-

nerability. If the sensitivity of the classification is required

to be at least .85, so as to reduce the risk of false negatives,

a score of 24 or greater best differentiates clients with

severe vulnerability from those with lower levels of vul-

nerability, a score of 21 or greater best differentiates those

with at least high vulnerability from those with lower levels

and a score of 20 or lower best differentiates those with

moderate or greater levels of vulnerability from those with

low levels. The cutoff separating those with at least high

vulnerability from those with lower levels appears to be

clinically less reliable than the other two cutoffs.

Conclusion

Overall, the AAG has sound psychometric properties,

though modification of one item in particular would be

fruitful; item 8 should ideally load on just one domain, and

more highly than it does currently, and should show greater

discrimination. The imperative for providers of bereave-

ment services is to target care appropriately to those most

in need, based on the complexity and persistence of grief

symptoms. This demands a measure that can give a con-

fident indication of vulnerability. The AAG is already used

to profile and explore the dynamics of individual grief and

as a guide for therapy, and its extension to provide an

indication of vulnerability would be an important devel-

opment for individual practitioners and for service provi-

sion more broadly. Further development of the

Table 6 Evaluation of optimum cutoff scores on the AAG scale; these are provided for the situation where sensitivity and specificity are equally

weighted and for the situation where sensitivity is prioritized (C.850)

Sensitivity and specificity equally weighted Sensitivity required to be at least .850

Optimum

cutoff

Youden

index

Sensitivity Specificity Optimum

cutoff

Youden

index

Sensitivity Specificity

Severe (n = 8) versus high/moderate/

low (n = 154)

24 .630 1.000 .630 24 .630 1.000 .630

Severe/high (n = 84) versus

moderate/low (n = 78)

23 .373 .655 .718 21 .240 .881 .487

Severe/high/moderate (n = 143)

versus low (n = 19)

21 .618 .776 .842 20 .544 .860 .684

1218 Qual Life Res (2014) 23:1211–1220

123

psychometric properties of the scale will strengthen its role

in this respect.

Acknowledgments This study was funded by the North Stafford-

shire Medical Institute. The authors wish to thank the Dove Service

(North Staffordshire, UK), St Giles Hospice (Lichfield, UK) and the

Marie Curie Hospices (Belfast and Hampstead, UK) for their help

with data collection, Aisling Bartlam for help with data entry, and

Kelvin Jordan for advice on the manuscript.

References

1. Office for National Statistics (2012). Births and deaths in Eng-

land and Wales 2011. London: Office for National Statistics

www.statistics.gov.uk/statbase/Product.asp?vlnk=14409.

2. Prigerson, H. G., & Jacobs, S. C. (2001). Traumatic grief as a

distinct disorder: A rationale, consensus criteria, and a pre-

liminary empirical test. In M. S. Stroebe, R. O. Hansson, W.

Stroebe, & H. Schut (Eds.), Handbook of bereavement research:

consequences, coping and care (pp. 613–645). Washington, DC:

American Psychological Association.

3. Neimeyer, R. A. (2000). Searching for the meaning of meaning:

Grief therapy and the process of reconstruction. Death Studies,

24, 541–558.

4. Neimeyer, R. A. (2010). Grief counselling and therapy—the case

for humility. Bereavement Care, 29, 4–7.

5. House of Lords (2013). Select committee on public service and

demographic change—first report: Ready for ageing? http://www.

publications.parliament.uk/pa/ld201213/ldselect/ldpublic/140/14002.

htm.

6. Prigerson, H. G., & Maciejewski, P. K. (2006). A call for sound

empirical testing and evaluation of criteria for complicated grief

proposed by the DSM V. Omega, 52, 9–19.

7. Prigerson, H. G., Horowitz, M. J., Jacobs, S. C., Parkes, C. M., Aslan,

M., Goodkin, K., et al. (2009). Prolonged Grief Disorder: Psycho-

metric validation of criteria proposed for DSMV and ICD-11. PLoS

Medicine, 6(8), e1000121. doi:10.1371/journal.pmed.1000121.

8. Machin, L. (2001). Exploring a framework for understanding the

range of response to loss; a study of clients receiving bereavement

counselling. Unpublished PhD thesis: Keele University, UK.

9. Machin, L. (1980) Living with Loss. Research Report for the

Lichfield Diocesan Board for Social Responsibility (unpublished).

10. Machin, L. (2009). Working with loss and grief: A new model for

practitioners. London: Sage.

11. Ainsworth, M. D. S., Blehar, M. C., Waters, E., & Wall, S.

(1978). Patterns of attachment: A psychological study of the

strange situation. Hillsdale, NJ: Lawrence Erlbaum.

12. Horowitz, M. J. (1997). Stress response syndromes, PTSD, grief,

and adjustment disorders (3rd ed.). Northvale, NJ: Jason Aronson.

13. Stroebe, M., & Schut, H. (1999). The Dual Process Model of

coping with bereavement: rationale and description. Death

Studies, 23, 197–224.

14. Martin, T. L., & Doka, K. L. (2000). Men don’t cry…women do:

Transcending gender stereotypes of grief. Philadelphia, PA:

Brunner/Mazel.

15. Mikulincer, M., & Florian, V. (1998). The relationship between

adult attachment styles and emotional and cognitive reactions to

stressful events. In J. A. Simpson & W. S. Rholes (Eds.),

Attachment theory and close relationships (pp. 143–165). New

York: Guilford Press.

16. Seligman, M. E. P. (1998). Building human strength: Psychol-

ogy’s forgotten mission. American Psychological Association

Monitor, 29, 1.

17. Greene, R. (2002). Holocaust survivors: A study in resilience.

Journal of Gerontological Social Work, 37, 3–18.

18. Machin, L., & Spall, R. (2004). Mapping grief: A study in

practice using a quantitative and qualitative approach to explor-

ing and addressing the range of response to loss. Counselling and

Psychotherapy Research, 4, 9–17.

19. Machin, L. (2007). The Adult Attitude to Grief Scale as a tool of

practice for counsellors working with bereaved people. Report

sponsored by Age Concern, Tameside and Keele University.

20. Stroebe, M. S., Folkman, S., Hansson, R. O., & Schut, H. (2006).

The prediction of bereavement outcome: Development of an

integrative risk factor framework. Social Science and Medicine,

63, 2440–2451.

21. Schumacker, R. E., & Lomax, R. G. (2010). A beginner’s guide to

structural equation modeling (3rd ed.). New York: Routledge.

22. Thompson, B. (2004). Exploratory and confirmatory factor

analysis: Understanding concepts and applications. Washington,

DC: American Psychological Association.

23. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in

covariance structure analysis: Conventional criteria versus new

alternatives. Structural Equation Modeling, 6, 1–55.

24. Marsh, H. W., Hau, K.-T., & Grayson, D. (2005). Goodness of fit

evaluation in structural equation modeling. In A. Maydeu-Oliv-

ares & J. J. McCardle (Eds.), Contemporary psychometrics: a

festschrift to Roderick P. McDonald (pp. 275–340). New York:

Routledge.

25. Blunch, N. J. (2008). Introduction to structural equation model-

ling using SPSS and AMOS. Thousand Oaks, CA: Sage.

26. Byrne, B. M. (2012). Structural equation modeling with Mplus:

Basic concepts, applications and programming. New York:

Routledge.

27. Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden

rules: Comment on hypothesis-testing approaches to setting cut-

off values for fit indexes and dangers in overgeneralizing Hu and

Bentler’s (1999) findings. Structural Equation Modeling, 11,

320–341.

28. Brown, T. A. (2006). Confirmatory factor analysis for applied

research. New York: Guilford Press.

29. DeMars, C. (2010). Item response theory. Oxford: Oxford Uni-

versity Press.

30. Kline, P. (1993). The handbook of psychological testing. London:

Routledge.

31. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory

(3rd ed.). New York: McGraw-Hill.

32. Raykov, T. (1997). Estimation of composite reliability for con-

generic measures. Applied Psychological Measurement, 21,

173–184.

33. Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (1991).

Criteria for scale selection and evaluation. In J. P. Robinson, P.

R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality

and social psychological attitudes (pp. 1–16). New York: Aca-

demic Press.

34. Kaplan, R. M., & Saccuzzo, D. P. (1997). Psychological testing:

principles, applications and issues (4th ed.). Pacific Grove, CA:

Brooks/Cole.

35. DeVellis, R. F. (2003). Scale development: Theory and applica-

tions (2nd ed.). Thousand Oaks, CA: Sage.

36. Spitzer, R. L., Kroenke, K., Williams, J. B., & Lowe, B. (2006).

A brief measure for assessing generalised anxiety disorder: The

GAD-7. Archives of Internal Medicine, 166, 1092–1097.

37. Kroenke, K., & Spitzer, R. L. (2002). The PHQ-9: A new

depression and diagnostic severity measure. Psychiatric Annals,

32, 509–521.

38. Lowe, B., Decker, O., Muller, S., Brahler, E., Schellberg, D.,

Herzog, W., et al. (2008). Validation and standardization of the

Qual Life Res (2014) 23:1211–1220 1219

123

Generalized Anxiety Disorder screener (GAD-7) in the general

population. Medical Care, 46, 266–274.

39. Kroenke, K., Spitzer, R., & Williams, J. (2001). The PHQ-9:

Validity of a brief depression severity measure. Journal of

General Internal Medicine, 16, 606–613.

40. Streiner, D. L., & Norman, G. R. (2008). Health measurement

scales: A practical guide to their development and use (4th ed.).

Oxford: Oxford University Press.

41. Youden, W. J. (1950). An index for rating diagnostic tests.

Cancer, 3, 32–35.

42. Cohen, J. (1988). Statistical power analysis for the behavioral

sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

1220 Qual Life Res (2014) 23:1211–1220

123