sheryl l. bishop, phd

Screening for PTSD in a community sample after Hurricane Ike: evaluating the psychometric utility of a brief screening instrument to assess coping, worry, and depression

SHERYL L. BISHOP, PHD

2012 National State of the Science Congress on Nursing Research

Parent Study: The Impact of Stress and Coping on Physical and Mental Health after Hurricane Ike

Phase I: At 4-6 months post-hurricane, online baseline assessment of measures of hurricane impact, perceived stress, depression, PTSD, resource loss, resilience, positive growth, physical symptoms, coping, social support, worry, displacement, damage, injury, financial data, health utilization data, employment & work disruption & four biological measures of stress via salivary samples were collected.

SL Bishop, University of Texas Medical BranchRJ Ruiz, University of Texas at Austin

R Stowe, Microgen Labs, IncCR Brewin, University College, London

CC Benight, University of Colorado at Colorado Springs


Parent Study: The Impact of Stress and Coping on Physical and Mental Health after Hurricane Ike

Phase II: Repeated assessment of measures of all variables were collected at the 12 month and 24 month anniversary of Hurricane Ike.


Specific Aims• Among many foci of interest was the opportunity to validate a

short screening instrument (10 items) for post-traumatic stress symptomology, the Trauma Screening Questionnaire (TSQ) by Brewing et al.1

• Developed from the PSS-SR2 a 17 item instrument• We used a modified PSS-SR, the MPSS-SR3, that included

intensity as well as frequency for each of the 17 items as our ‘gold standard’.

1. Chris R. Brewin, Suzanna Rose, Bernice Andrews, John Green, Philip Tata, Chris McEvedy, Stuart Turner and Edna B. Foa, Brief Screening Instrument for post-traumatic stress disorder, British Journal of Psychiatry (2002), 181, 158-162

2. Foa, E. B., Riggs, D. S., Dancu,C.V., et al (1993) Reliability and validity of a brief instrument for assessing post-traumatic stress disorder. Journal of Traumatic Stress, 6, 459-473.

3. Falsetti, S. A., Resnick, H. S., Resick, P. A., & Kilpatrick, D. (1993). The Modified PTSD Symptom Scale: A brief self-report measure of posttraumatic stress disorder. The Behavioral Therapist, 16, 161-162.


Sample

• UTMB community: Faculty, staff and students– 7th largest employer in the region (13,000 employees

statewide) – 6 hospitals– State’s oldest schools of Medicine, Nursing, Health

Professions and Biomedical Sciences– 2,338 students(2,094 from 122 TX counties), over 700

interns, residents, and fellows, and 1,116 faculty.• 391 respondents with complete data for this analysis

Male15%

Female 85%

Gender

White77%

Black or African Ameri-

can1%

Hispanic12%

Asian8%

American Indian or Alaska Native

1%

Males by Race

White83%

Black or African Ameri-

can3%

Hispanic10%

Asian3%

American Indian or Alaska Native

1%

Females by Race

N=477 Respondents

Displac

ed

Damag

eInjury

Inconve

nience

Family

Community1

1.5

2

2.5

3

3.5

4

4.5

5

Hurricane Exposure Ratings by Gender

MalesFemales

A great deal

None


Instrument CharacteristicsMean Sd Range Items Scale

TSQ 3.35 2.65 0-10 10 Yes (at least twice a week)/no

MPSS-SR 20.06 17.46 0-104 17 0 (not at all)- 3 (5 or more times a week)

Criterion for affirmative PTSD symptomology for TSQ is a score of 6 and 46 for the MPSS-SR.


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 52 57 65 72 940

5

10

15

20

25

30

MPSS-SR

Fre

qu

ency

8.2%


21.2%

N=401


N=393

48.8%

34.6%

N=393


Validating

• Correlation TSQ x MPSS-SR=.767, p<.001However correlation only indicates that a positive relationship exists. It cannot indicate whether evaluations AGREE. Therefore, it is NOT the appropriate approach.


Indicators of Agreement• Interclass correlation coefficient – ICC

– Must transform both instruments into the same scale metric…z scores computed for this purpose.

– Indicates moderately high agreement however, Bland and Altman (1990) provide compelling arguments against the use of the ICC to evaluate whether two methods are equivalent.

Intraclass Correlation Coefficient: Two-way random effects modelICC 95% Confidence Interval F Test

Lower Bound Upper Bound Value SigSingle Measures .767 .723 .805 7.581 .000Average Measures .868 .839 .892 7.581 .000


Crosstabulation Analyses• Kappa – compares rating agreement when

variables are dichotomized into DX = Yes/No.– Clearly the criteria used to determine cut-offs is

critical– And when evaluating a new instrument against a

parent instrument, the assumption that the parent instrument is ‘true’ may be unsupported. Thus conclusions about concurrence may be biased in favor of the reference instrument.


•Kappa =.414, p<.001 is moderate agreement•Sensitivity (True Positives)=78.95%•Specificity (True Negatives)=84.7%•False Positives (Type I error)=15.3%•False Negatives (Type II error)= 21.1% •PV+=35.7%•PV-=97.4%

T1ptsd (reference) Total

DX+ (yes) DX- (no) t1tsq DX+(yes) 30 54 84 DX-(no) 8 299 307

Total 38 353 391


Distribution of Difference Scores

d=-.0102sd=.68153

Limits of Agreement:d-2s = -.0102-(1.36306)= -1.37326d+2s=-.0102+(1.36306)= 1.35286N=13 N=7


Convergent Validity

• Forward stepwise regressions were conducted with the Depression the as outcome variable

• The predictor set was hurricane exposure scores, worry, 14 coping styles and either the TSQ or the MPSS-SR PTSD scores

• Convergent validity would be demonstrated if both instruments comparably predicted depression.


Predicting DepressionTSQ: R2=.672 MPSS-SR: R2=.684

PREDICTORS t Partial t Partial

PTSD SCORE 8.484*** .405 14.322*** .598

Self Blame 5.774*** .289 3.855*** .197

Worry 5.799*** .290 5.425*** .272

Behavioral Disengage.

4.345*** .221 2.063* .107

Positive Reframing

-3.454*** -.177 -4.669*** -.236

Hurricane Exposure

2.688** .139 ns --

Humor -2.284* .118 ns --


Summary

• All ‘traditional’ assessments of equivalency show a high degree of ‘agreement’ over response patterns or classifications.

• Newer procedures also support a moderate to high degree of equivalency between the two instruments.

• Limits of agreement indicate a fairly narrow range of variability between assessments between the two instruments.


Summary Continued• TSQ does a fairly good job of identifying both true

positives and true negatives. • False positive and false negative rates indicate that

TSQ missed more people WITH PTSD than incorrectly diagnosing those as having PTSD who did not suffer

• High Negative Predictive value = 97.4% = accurate in identifying those that are NOT likely to be suffering from PTSD,

• Low Moderate Positive Predictive value = 35.7% = fair at identifying those that definitely have symptoms of PTSD.


Summary continued

• Regression analyses further demonstrated convergent validity by highly similar and reflective patterns of prediction of depression.

• There is some indication of support for more complex modeling with the use of the TSQ to allow for finer distinctions in coping behaviors.


Conclusion

• The TSQ appears to be a good screening tool for use in early triage situations where false positives are more preferable to false negatives. It has high negative predictive value and can reliably rule out those individuals that do not exhibit PTSD symptoms but may incorrectly classify some as having symptoms that will later be ruled out on further testing.

QUESTIONS?

sheryl l. bishop, phd

Documents