multilingual computer assisted self interviews in the ...events.irct.org/uploads/archives/28/p...
TRANSCRIPT
Multilingual Computer Assisted Self Interviews in the
Assessment of Traumatized Refugees residing in Australia
Jorge Aroche, Chief Executive Mariano Coello, Clinical Services and Research Coordinator Shakeh Momartin, Senior Research Officer
NSW Service for the Treatment And Rehabilitation of Torture and Trauma Survivors
The problem• Many reasons and incentives to better measure our
client’s problems and intervention outcomes • A very linguistically diverse client group ( 70+
langs.) • Some clients have low literacy in their own
language • Large volume of clients (approx. 4,000 per year) • Very expensive interpreters • Use of interpreters affects the consistency of
questions asked, and thus the reliability of measures utilized.
• High interpreter and data entry costs make Pen & Paper pre and post psychometrics inaccessible (2 X $100 X 4000 =$800,000 per year)
Finding a solution• A few years ago I
met Julia Muller and Kristine Knaevelsrud at an ECOTS/ESTSS conference
• Following many discussions and lots of translation and IT work , MultiCASI has became part of our standard assessment process
Multi CASI has now been in use at STARTTS since 2014, and is installed in all laptops utilized by staff at counselling rooms and home visits. Several thousand questionnaires have been completed.
What is MultiCASI?• Multilingual Computer-Assisted Self Interview • Created by Christine Knaevelsrud & Julia Müller (2008) at
Berlin Center for Torture Victims
• Software platform for administering multiple choice self-report questionnaires and exporting data for analysis
• A platform that enables users to upload different questionnaire translations and recorded readings of questions
• It then enables users to present these questionnaires to clients in their own language, in both written and verbal form.
• Clients fill the questionnaire using a mouse or a touch screen (better for non-computer literate clients)
• Functionality increased by add-ons (Wizards) by STARTTS (Australia) and CTP (Denmark).
STARTTS Core Battery of Questionnaires used in MultiCASI to assess Refugee Trauma
• Harvard Trauma Questionnaire (Symptoms – 16 Questions)
• Hopkins Symptom Checklist (Anxiety – 10 Questions) • Hopkins Symptom Checklist (Depression – 15 Questions) • General Self-Efficacy Scale (10 Questions) • Modified Depression Anxiety Stress Scale (DASS,
modified for use with adolescents and adults – 21 Questions)
• Client Satisfaction Questionnaire (8 Questions) • Additional Questions assessing client satisfaction of
STARTTS service (2 Questions)
• Child PTSD Symptom Scale (24 Questions) (available in most used languages only)
Languages translated by STARTTSPrimary Languages in Core Battery (all questionnaires available)
1. Arabic 2. Dari 3. Farsi 4. Tamil 5. Hazaragi 6. Vietnamese 7. Kirundi 8. Swahili 9. Mandarin (Simplified) 10. Nepali 11. Assyrian 12. S'gaw Karen*
Additional Languages translated only for HTQ and Hopkins Symptom Checklist
1. Kurdish 2. French 3. Russian 4. Portuguese 5. Bosnian 6. German 7. Turkish 8. Albanian 9. TibetanAll languages translated using the WHO recommended method
Text and voice for all languages and all questionnaires have been entered into MultiCASI.
How do we use MultiCASI at STARTTS?
• Small assessment battery administered to all clients (optional for clients).
• Usually at end of 1st session or at 2nd session.
• Explained through an interpreter or video.
• Client fills questionnaire following the session.
• Battery administered again every 6-10 sessions or at discharge
What it looks like: Selection Wizard
What it looks like: Client view
What it looks like: End of session
What it looks like: Results Wizard
Advantages over pen and paper • Can be used when interpreters not available • Can be used with illiterate and semi-literate clients • Can be used by clients in private (without interpreters) • Standardised delivery • Prevents missing or ambiguous responses • Increases reliability of results • Lower cost of administration • Data in digital form • No data-entry needed (zero errors) • Results immediately available for analysis • Allows client to revise and answer at own pace • Automatically scores tests and provides data to
clinician • Exports data to clinical data base.
Challenges of Multi-CASI versus paper form
Multi-CASI ▪ Questionnaire translation ▪ Questionnaire administration ▪ Some refugees speak languages
without written form ▪ Some refugees lack the level of
reading comprehension required in own language and English
▪ Peoples attitude and knowledge of computers
▪ Reliability based on standardised presentation
▪ Interpreter might change meaning/concept of items
▪ Interpreter’s delivery may influence response
▪ Clients may be less forthcoming with interpreter/privacy issues
▪ Interpreter’s cultural background might be problematic
✓
✓
✓
✓
Paper Assessment
Reasons for Pilot Study• Important to ascertain that presenting the test
in a computerized form : – Does not alter the test’s validity – Does not affect acceptance by clients
• Is equal or better than a pen and paper administration format
• This is important from a service provision point of view, but even more so to support studies based on the utilization of computer administered psychometric tests.
• Essentially, this procedural validation study is a prelude to more interesting studies later on
Pilot Procedural Validation StudyAims: • To assess the feasibility, acceptability and time consumption
of Multi-CASI (MC) based self-assessment in a population of traumatized refugees living in Australia compared with clinician administered paper-and-pencil assessment method (CAPP),
• To assess relative cost effectiveness in terms of therapist costs, interpreter costs and translations needed for each client
• To Examine: Whether MultiCASI has procedural validity compared to pen and paper administration of psychological questionnaires
• To Examine: Whether clients report that MultiCASI is an acceptable alternative to pen and paper questionnaires
Hypothesis: • That there would be no significant differences between MC
and CAPP in reported symptoms of depression and anxiety • that the proportion of people reporting problems with
completing MC would be no greater than in CAPP • that participants completing MC would take less time than
CAPP
MethodParticipants: • 61 clients, convenient sample, recruited from STARTTS • Randomly assigned to either MC or CAPP assessment mode • Received Information Statement and completed Consent
Form Materials: • Hopkins Symptoms Checklist-25 (10 anxiety and 15
depression items) • Acceptability questionnaire (Knaevelsrud, & Mueller, 2007
revised version) (11 items) • All questionnaires translated and recorded in Multi-CASI • Analyzed in SPSS
Design
• Multicasi Group
• Pen & Paper Group
MultCASI Platform tests
Pen & Paper tests
Acceptability Questionnaire
Acceptability Questionnaire
Pen & Paper tests
Pen & Paper tests
Results (1)
Results (2)Table 1
Results of Self-Report Questionnaires for Clinician-Administered Pen-and-Paper (CAPP)
sample (n =29) and MultiCASI (MC) sample (n = 32)
Test scores
CAPP
M (SD)
MC
M (SD)
Diff.
t(59)
p
r
HSCL Anxiety 2.83 (0.88) 2.71 (0.82) 0.12 0.53 .60 .07
HSCL Depression 2.83 (0.76) 2.78 (0.73) 0.05 0.28 .78 .04
Both groups were about equally educated (ie univers
2
Conclusions of Pilot study• No significant differences between MultiCASI and paper-based
questionnaire groups in Hopkins Symptom Checklist scores for reporting anxiety and depression.
• No significant differences between MultiCASI and paper-based questionnaire groups in time taken to complete questionnaires
• No significant differences between MultiCASI and paper-based questionnaire groups in problems reported
• Interestingly, while 24.1% of participants in the pen and paper group said that they imagined that completing a computerised questionnaire would be ‘uncomfortable’ compared to completing the questionnaire with a staff member, only 6.2% of those in the computerised group agreed, Χ² (1, N = 61) = 3.87, p = .05, r = .25.
• This is a significant difference which supports the hypothesis that using the computerised questionnaire would be less uncomfortable than imagined.
Future Plans1. Larger, international multicentre procedural
validation study (currently under way: STARTTS, UNSW, ASeTTS, RANZ, AUT, CTP).
2. Development of a new version of a MultiCASI like computer based assessment platform from scratch to improve various aspects of MultiCASI (underway)
3. Development of a new, culturally robust psychometric tool that addresses a larger number of dimensions pertinent to rehabilitation work with torture and trauma survivors (Consortium formed).
4. Intention to make available 2 and 3 above for free to IRCT members when completed (ideally through IRCT).
Multicenter MultiCASI Procedural Validation Study
• Consortium members: – STARTTS, NSW, Australia – The University of NSW Psychiatry Research and
Teaching Unit, Psychiatry Research And Teaching Unit (PRTU), Liverpool Hospital, NSW
– ASeTTS, WA, Australia – Refugees As Survivors (RANZ), Auckland, New Zealand – Auckland University of Technology, New Zealand – Competence Centre for Transcultural Psychiatry
(CTP), Denmark. • 280 subjects • Estimated completion: December 2017
CAMLAPComputer Administered Multi-Lingual
Assessment Platform• Development commissioned from MultiCASI
original developer in Switzerland • Estimated Completion Date: June 2017 • Improvements in compatibility with new
operation systems • Modular design. • More user friendly business side • Full IP, so it can be distributed to other
centres for their own use.
