Validation of English and Spanish Versions of a Client Progress Measure for Use in

Healthcare-based Domestic Violence Programs

Authors: Jeanne Hathaway, MD, MPH; Bonnie Zimmer, LICSW; Sue Chandler, MPH, MSW; Lisa Hartwick, LICSW; Jennifer Robertson; Lisa Tieszen, LICSW; Jay Silverman, PhD

Internal Consistency of Current Situation Rating Scale

Methods: We measured the internal consistency of the Current Situation Rating Scale of the Client Feedback Form. This test indicates how similar ratings are between items on each person’s survey.

Sample: n = 155 ( 98 - English; 57 - Spanish)

Main analytic test: Cronbach’s alpha

Results: Cronbach’s alphas for the Current Situation Rating Scale were .86 (English) and .87 (Spanish).

Conclusions: The internal consistency of the Current Situation Section is uniformly high on both English and Spanish versions of the Client Feedback Form.

Revisions to Client Feedback Form: None based on these results.

Response Bias

Methods: We measured response bias by comparing responses on Client Feedback Forms reviewed WITH a DV program advocate to responses on Client Feedback Forms NOT reviewed with an advocate.

Sample: n = 48 “With Advocate” group; n = 50 “No Advocate” group; (English only) The “With Advocate” group was slightly more likely than the “No Advocate” group to be of “Other” race/ethnicity (p = .09), born outside the U.S. (p = .04), and to have some income (p = .09). Groups did not differ significantly in terms of types of abuse, court or legal issues, or other barriers to change/healing reported.

Main analytic test: Linear regression

Results: Controlling for race/ethnicity, immigration status, and income, we found no significant association between group and clients’ ratings of their “Current Situation” (p = .75) or how participation in advocacy or support groups had affected their lives (p = .52).

Conclusions: We found no evidence of response bias, that is, Client Feedback Forms reviewed WITH a DV program advocate did not have “better” (or “worse”) responses than forms NOT reviewed with an advocate.

Revisions to Client Feedback Form: None based on these results.

How did it feel to complete the Client Feedback Form?

“It felt good to let out some of the bottled feelings, especially writing out on paper was a

slight relief.”

“Me dado de cuenta de muchas cosas que no veia antes.”

[It gave me insight into things I didn’t see before.]

1) Current Situation Rating Scale–Covers a range of broad categories (personal safety, empowerment, emotional support, etc.) –Uses 1-5 likert rating scale–List other formal supports

2) Barriers to Change / Healing–Abuse and controlling behavior –Court and legal issues–Other potential barriers (lack of resources, health concerns, etc.)

3) Effects of Advocacy or Support Groups–Better/Worse ratings of same categories as Current Situation Rating Scale –Detailed questions re: health/health care effects–Two open-ended questions: Ways most helpful? Anything worse?

4) Client’s Goals–Two open-ended questions: Current goals? Ways advocate can help?

Main Sections of Client Feedback Form

How administered: The Client Feedback Form is completed by the client (with or without advocate assistance), then reviewed with the client’s advocate. The form is currently completed on paper, however some programs are considering computer-administration. We decided that clients should complete forms, rather than advocates or healthcare providers, since clients are the best source of information on how well the program is achieving its goals and how program advocates can best assist them.

When administered: The Client Feedback Form is completed 3 and 6 months after intake, then every 6 months thereafter. The “Initial” version of the form, which asks about lifetime partner abuse, is used at the first administration. The “Follow-up” version of the form, which asks about partner abuse in the past 6 months, is used subsequently.

Length: The Client Feedback Form is 6 pages and takes approximately 15 - 20 minutes to complete (not including review with the advocate.)

Languages: Currently available in English and Spanish

We used a convenience sample of 155 women receiving services from four healthcare-based DV programs in the Boston area. Eligible clients were female, age 18 or over, spoke English or Spanish, had 3 or more phone or in-person contacts with a DV advocate, and were not in acute emotional distress (as determined by the DV advocate).

Study Sample

Latino 49%

Asian 4%

Afr-Amer 18%

White 21%

Other 8%

U.S. Born 52%

Born Outside U.S. 48%

• Average age: 35 y.o. (range: 18 – 59 y.o.)

• Average length time in program: 1 year and 4 months

7%

45%

8%

14%

33%

5%

0%

10%

20%

30%

40%

50%

Own Job

Finan

cial

Ass

ist.

SSI/ret

irem

ent

Other

Sou

rce

Partn

er

No Inco

me

Pe

rce

nta

ge

of

Pa

rtic

ipa

nts

Single, Never

married-40%

Separated or Divorced-

39%

Married-19%

Widowed-2%

20

Highest Education

< H.S.-24%

H.S. or equiv.-36%

< 4 yrs. college-24%

4 or more yrs. college-16%

Medicaid/care, Masshealth or Free Care -

60%

No Health Insurance -8%

Other Health Insurance -2%

Private Health Insurance -

30%

Study Components

• Concurrent Validity of Five Key Concepts

• Internal Consistency of Current Situation Rating Scale

• Response Bias

• Acceptability and Clarity

We administered written surveys at one time point to women receiving services from healthcare-based DV programs to assess the following aspects of the Client Feedback Form:

Concurrent Validity of Five Key Concepts

Methods: We measured the concurrent validity of five key constructs in the Client Feedback Form (partner abuse, physical health, mental health, self-efficacy, and quality of life) by comparing Client Feedback Form responses to scores on standardized measures covering similar constructs.

Sample: n = 100 (43 – English; 57 – Spanish)

Main analytic test: Spearman Correlation Coefficients

Results:

ConstructClient

Feedback Form Item(s)

Comparison Measure

Spearman Correlation Coefficient

P-value

Partner abuse Abuse and Controlling Behavior Section

Index of Spouse Abuse

.58 (Eng)

.55 (Sp)

<.0001

<.0001

Mental Health Item #3 on Current Situation Rating Scale

COOP Functional Assessment Charts (Feelings)

.68 (Eng)

.38 (Sp)

<.0001

.004

Physical Health Item #4 on Current Situation Rating Scale

COOP Functional Assessment Charts (Pain, Fitness)

.67 (Eng)

.06 (Sp)

<.0001

.63

Self-efficacy Item #10 on Current Situation Rating Scale

General Self-efficacy Scale

.33 (Eng)

.43 (Sp)

.04

.002

Quality of Life Current Situation Rating Scale

Quality of Life Index

.71 (Eng)

.59 (Sp)

<.0001

<.0001

Conclusions: In both language versions, correlation coefficients were high for partner abuse and quality of life. Correlations were also high for mental and physical health on surveys completed in English. Correlations were low for self-efficacy in both language versions, as well as for mental and physical health on surveys completed in Spanish.

Revisions to Client Feedback Form: Added new self-efficacy/ empowerment item to Current Situation Rating Scale and modified wording of mental and physical health items on Spanish version.

Okay or Fine - 31%

Some Discomfort -

11%

Comfort Level Mixed - 15%

Good or Very Good - 43%

Methods: Assessment of the acceptability and clarity of the Client Feedback Form were obtained through a brief 3-item survey with both open and closed-ended questions. Sample: n = 105 (49 – English; 56 - Spanish)

Main analyses: Frequencies and content analysis

Results: How did it feel to complete the Client Feedback Form?

- Responses did not differ significantly by language version. - Discomfort was mainly attributed to being reminded of past abuse.

Acceptability and Clarity

Were any parts of the Client Feedback Form confusing or unclear?

15% of respondents (7 – English; 9 - Spanish) reported that some aspect of the form was confusing or unclear. Of these 16, six did not specify what was unclear and no more than three respondents cited confusion over any single item or section. Some comments were:

“I needed my advocate to explain it to me because English is hard for me.” – Asian

respondent

“Court and legal issues [were confusing] – some still ongoing, but not necessarily active

or court-related.”

Conclusions: The Client Feedback Form was acceptable and clear to the majority of respondents, regardless of language version. Approximately one in four respondents experienced some discomfort, however, mainly due to the abuse questions. Minor changes should be made to clarify certain items or sections.

Revisions to Client Feedback Form: We developed an instruction guide that outlines precautions advocates can take to minimize clients’ risk of emotional discomfort and reminds advocates that some clients may need additional support after completing the form.

We also revised the directions to the Court/legal Section and replaced one open-ended question on informal supports with two closed-ended questions on emotional and material support in the Current Situation Section.

Additional minor revisions were made to the Client Feedback Form following this study based on:

• Responses written in under “Other” in several sections

• Missing descriptions of change in the Program Impact Section

• Responses to Current Goals questions

• Back-translation of the form from Spanish to English

Summary

These initial assessments indicate that English and Spanish versions of the DV Program Client Feedback Form are generally acceptable and clear to respondents, have good validity of items covering partner abuse and quality of life, and that review of client’s responses with a DV advocate does not cause significant response bias.

Future Research: Additional studies are needed to validate items covering self-efficacy on both language versions and mental and physical health on the Spanish version.

Future research should also assess the reliability and validity of the Client Feedback Form among different populations, including other racial/ethnic groups, adolescents, people in same-sex relationships and male victims. It may also be worthwhile to assess utility of the Client Feedback Form in other types of domestic violence services.

Type Health InsuranceSources of Income

Marital Status

Race/ethnicity

Immigration Status

Photos are for illustration only and do not represent actual program clients.

Photos are for illustration only and do not represent actual program clients.

Photos are for illustration only and do not represent actual program clients.

• Improve clients’ safety,• Improve clients’ health and well-being,• Connect clients to needed resources, and• Increase clients’ ability to advocate for themselves.

• Healthcare-based DV program directors and advocates

• Focus groups with White, African-American, and Latino clients of healthcare-based DV programs

• Findings from previous research with clients from one of the healthcare-based DV programs involved (HAVEN)

• Intake and activity forms from healthcare-based DV programs

• Other measures, including the DV Survivor Assessment by Dienemann and Campbell

In addition to program evaluation, the DV Council also intends to use the Client Feedback Form as an advocacy tool. We therefore designed the form to help assess clients’ needs, help clients recognize their strengths, and help advocates focus their work with clients.

Efforts were made when developing the Client Feedback Form to minimize any ways in which completing it could be harmful to clients or the client-advocate relationship. This was done through attention to the wording, order, and length of each section. We also developed an instruction guide for administering the form, since how the form is introduced and used will also affect clients’ experiences.

In summary, the Client Feedback Form tested in this study was developed over a period of two years with input from:

Background and Study ObjectivesBackground: Despite the increasing number of domestic violence (DV) programs based in health care settings, few of these programs have been evaluated. This is partly due to a lack of appropriate outcome measures. To address this gap, the Conference of Boston Teaching Hospitals (COBTH) DV Council has developed a new progress measure, the DV Program Client Feedback Form for use in healthcare-based DV programs.

Study objectives: To conduct initial tests of the reliability and validity of English and Spanish versions of the “Client Feedback Form.”

Purpose of Client Feedback Form: The Client Feedback Form is designed to be used for both research and advocacy purposes. In terms of research, the DV Council intends to use the form to evaluate how well healthcare-based DV advocacy programs are meeting their programs’ goals. The four common goals of these programs are to:

How did it feel to complete the Client Feedback Form?

“Good - but I’m a little depressed because there are things that happened that I don’t want to

remember.”

“Me siento un poco triste. Triste porque hay cosas que pasan que uno no quisiera recordar.”[I feel a little sad. Sad because there are things that happened that I don’t want to remember.]