Emergency Room Emergency Room SurveySurvey

Survey Planning, Survey Planning, Administration and CostAdministration and Cost

SIDUC: AN INTEGRATED SYSTEM OF SIDUC: AN INTEGRATED SYSTEM OF DRUG USE SURVEYSDRUG USE SURVEYS

Key Informants

SIDUC STANDARDIZATIONSIDUC STANDARDIZATION

I nformation Source

Reference Population

Analized Population

Data Collection

Periodicity

Treatment

Centers

Patients

Patients

I nterview

Continue

Emergency

Rooms

Patients

12 to 64 years

Sample of hospitals and

Patients

I nterview

Annual

Student Surveys

Students

Grades 8, 10, 12

Sample of Schools & classes

Self admnistered

Questionnaire

2 years

Household Surveys

Population in households

12 to 64 years

Sample of areas,

Households & persons

I nterview / Self admin

3 years

EMERGENCY ROOMSEMERGENCY ROOMS

Population of referencePopulation of reference:: Patients of Emergency Rooms.Patients of Emergency Rooms.

Analyzed populationAnalyzed population:: a) all patients b) “typical weeks”. a) all patients b) “typical weeks”.

Judgmental sample.Judgmental sample. Method of data collectionMethod of data collection::

Interview & laboratory analysis.Interview & laboratory analysis.

FORENSIC MEDICINEFORENSIC MEDICINE

Population of reference:Population of reference: Death by accident, homicide, Death by accident, homicide,

suicide.suicide. Analyzed Population:Analyzed Population:

a) all , b) “Typical weeks”.a) all , b) “Typical weeks”. Method of data collectionMethod of data collection::

Autopsy and laboratory analysis.Autopsy and laboratory analysis.

Steps to be TakenSteps to be Taken

A funding source or multiple sources A funding source or multiple sources must be found (or no survey)must be found (or no survey)

Decisions must then be made about Decisions must then be made about who will direct the research who will direct the research undertakingundertaking

What types of expert and staff support What types of expert and staff support will be needed, and will be needed, and

What is a realistic timetableWhat is a realistic timetable– for the overall endeavorfor the overall endeavor– for the major components for the major components

Session OutlineSession Outline

Core Survey ActivitiesCore Survey Activities PersonnelPersonnel BudgetBudget ScheduleSchedule

Core ActivitiesCore Activities

1.1. Establishment of Advisory CommitteeEstablishment of Advisory Committee

2.2. Develop ProposalDevelop Proposal

3.3. Seek Permission to conductSeek Permission to conduct

4.4. Finalize MethodologyFinalize Methodology

5.5. TrainingTraining

6.6. Survey ImplementationSurvey Implementation• Data Collection, Handling, Analysis, Data Collection, Handling, Analysis,

Report WritingReport Writing

Advisory CommitteeAdvisory Committee

Stakeholders identification/representationStakeholders identification/representation Ministry of HealthMinistry of Health Public Hospital AuthorityPublic Hospital Authority ER DirectorER Director Police Police Ministry of TransportMinistry of Transport Rep from private hospitalRep from private hospital COB Research UnitCOB Research Unit

N.B> Can avoid problems at the end N.B> Can avoid problems at the end Can help to “sell” the surveyCan help to “sell” the survey

Communications Regarding Communications Regarding SurveySurvey

Ministerial Responsibility (CMO)Ministerial Responsibility (CMO) Institutional ResponsibilityInstitutional Responsibility

– Public Hospital Authority (PMH, RMH)Public Hospital Authority (PMH, RMH) Institutional Review Boards/Ethics Institutional Review Boards/Ethics

Comm.Comm. How to contact the chosen hospitalsHow to contact the chosen hospitals

– AdministratorAdministrator– ER ManagerER Manager

Contact With Selected Contact With Selected InstitutionsInstitutions

Step 1Step 1 AAn introductory letter to the n introductory letter to the AdministratorAdministrator, ,

informing him/her of the study and its informing him/her of the study and its purposespurposes. It will also:. It will also:– Ask tAsk to inform the o inform the ER Manager of ER Manager of

decisiondecision

Step 2Step 2 CContact the ER manager via telephone to ontact the ER manager via telephone to

confirm that everything is in orderconfirm that everything is in order just just prior. At this point:prior. At this point:

Project PersonnelProject Personnel

Lead Investigator(s)Lead Investigator(s) Core StaffCore Staff

– Site managersSite managers– Data ManagersData Managers

Data Collection StaffData Collection Staff– DoctorsDoctors– NursesNurses– Medical Records ClerksMedical Records Clerks

Consultant(s)Consultant(s)

Personnel Selection Process Personnel Selection Process OptionsOptions

1.1. Advisory Committee to provide oversight of the Advisory Committee to provide oversight of the enterprise and the selection of the lead enterprise and the selection of the lead investigatorsinvestigators– AD: can help to engage people or organizations needed AD: can help to engage people or organizations needed

to fund, conduct, or make use of the researchto fund, conduct, or make use of the research– DIS: “too many cooks in the kitchen” and may introduce DIS: “too many cooks in the kitchen” and may introduce

political considerations into the processpolitical considerations into the process– REC: committee should deal with the broad policy and REC: committee should deal with the broad policy and

financial issues and leave the scientific decisions to the financial issues and leave the scientific decisions to the scientists scientists

2.2. Recruitment of one or more individuals to run the Recruitment of one or more individuals to run the projectproject

3.3. Individual researchers may take the initiative to Individual researchers may take the initiative to conduct such a surveyconduct such a survey

Lead InvestigatorLead Investigator

Ideally will be a part of the activity Ideally will be a part of the activity from beginning to endfrom beginning to end

Will provide the planning and Will provide the planning and integration needed to be sure that the integration needed to be sure that the end product matches the needs and end product matches the needs and objectivesobjectives

Should be trained social scientists Should be trained social scientists with some experience in survey with some experience in survey research techniquesresearch techniques

Core StaffCore Staff(Site Managers)(Site Managers)

These key individuals will remain with the These key individuals will remain with the study for its durationstudy for its duration

Participate in various activities and make sure Participate in various activities and make sure that they are carried out according to plan.that they are carried out according to plan.

Most likely will supervise various components Most likely will supervise various components of the study, under the general direction of of the study, under the general direction of the lead investigator(s)the lead investigator(s)

Should be well educated and preferably have Should be well educated and preferably have some experience with research activities.some experience with research activities.

Data Collection StaffData Collection Staff

Decide whether the data will be collected Decide whether the data will be collected by individuals from outside the hospitals by individuals from outside the hospitals or not or not – ER Staff: If the patients will trust the MDs or ER Staff: If the patients will trust the MDs or

Nurse to protect their confidentialityNurse to protect their confidentiality– Outside Staff: If the staffing situation presents Outside Staff: If the staffing situation presents

a challenge, then staff members must be hired, a challenge, then staff members must be hired, trained, supervised and usually compensatedtrained, supervised and usually compensated trained health social workers, trained field trained health social workers, trained field

interviewers from a survey research organization, interviewers from a survey research organization, university students, etcuniversity students, etc

ConsultantsConsultants

Overall PlanningOverall Planning To direct technical assistance at various stages, depending of course on the To direct technical assistance at various stages, depending of course on the

areas of expertise of the lead investigators areas of expertise of the lead investigators If the lead investigators are new to this area of research, consider a short-If the lead investigators are new to this area of research, consider a short-

term consultation with an experienced expert in the initial planning stage, term consultation with an experienced expert in the initial planning stage, again at the analysis planning stage, and perhaps at the interpretation stage again at the analysis planning stage, and perhaps at the interpretation stage

If brought in from abroad, seek the assistance and support of one of the If brought in from abroad, seek the assistance and support of one of the international organizations that deal with the control of drug abuse (UNODC, international organizations that deal with the control of drug abuse (UNODC, OAS)OAS)

Sample PlanningSample Planning Consultation with a sampling statistician also is likely to be very helpful, Consultation with a sampling statistician also is likely to be very helpful,

again at an early point in the planning effort again at an early point in the planning effort

Statistical AnalysisStatistical Analysis If neither a general consultant from the substance abuse field nor statistician If neither a general consultant from the substance abuse field nor statistician

is able to help on data analyses, seek an expert on statistical analyses.is able to help on data analyses, seek an expert on statistical analyses. The job is usually not to actually conduct the analyses but to advise on the The job is usually not to actually conduct the analyses but to advise on the

choice of analyses and appropriate computer programs for conducting them. choice of analyses and appropriate computer programs for conducting them.

Budget PlanningBudget Planning1.1. Personnel CostsPersonnel Costs

2.2. Non-Personnel CostsNon-Personnel Costs

Factors that May Impact Factors that May Impact Personnel BudgetPersonnel Budget

Labor costs in the countryLabor costs in the country Whether it is necessary to use data collection Whether it is necessary to use data collection

staff from outside the hospitalsstaff from outside the hospitals The method of data entryThe method of data entry The need for technical assistanceThe need for technical assistance The size of the sample needed and the size of the The size of the sample needed and the size of the

country across which that sample will be spreadcountry across which that sample will be spread– influence both staff and travel costsinfluence both staff and travel costs

Amount of the personnel assigned to plan and Amount of the personnel assigned to plan and conduct the ER survey who are already on the conduct the ER survey who are already on the payroll of a participating agency.payroll of a participating agency.

Sample Budget OutlineSample Budget Outline

PERSONNEL COSTSPERSONNEL COSTS

Lead investigatorsLead investigators

Field staff (if applicable) for __ weeksField staff (if applicable) for __ weeks–Supervisors/Site ManagerSupervisors/Site Manager–Facilitators/MDs, Nurse, Med Rec.Facilitators/MDs, Nurse, Med Rec.

Core support staffCore support staffSecretarial/ClericalSecretarial/Clerical

Subcontractor CostsSubcontractor Costs–For data entry (if applicable)For data entry (if applicable)–For any other services being purchasedFor any other services being purchased

ConsultantsConsultants–Fringe benefitsFringe benefits

NON-SALARY COSTSNON-SALARY COSTS

OfficeOffice–RRental (if applicable)ental (if applicable)–FFurniture (if applicable)urniture (if applicable)–EEquipment (as needed)quipment (as needed)–PhonesPhones, , FaxFax, , Copy machineCopy machine, , ComputersComputers–SSuppliesupplies–Telephone serviceTelephone service

Advertising/Recruiting CostsAdvertising/Recruiting CostsPrintingPrinting

–QuestionnairesQuestionnaires–ManualsManuals–Brochures, instruction sheets, etc.Brochures, instruction sheets, etc.–Final reportFinal report

Shipping and postageShipping and postage–Questionnaires to hospitalsQuestionnaires to hospitals–Questionnaires back from hospitalsQuestionnaires back from hospitals–Other Other

Travel CostsTravel Costs–For investigatorsFor investigators–For field staff (if applicable)For field staff (if applicable)

Overhead charges (if any)Overhead charges (if any)

Estimated Budget for Bahamas Emergency Estimated Budget for Bahamas Emergency Room Drug SurveyRoom Drug Survey

Phase Specific Tasks Time Frame

Cost ($US)

 

Planning      

         

Data Collection Training    

Payment to Facilitators

   

Other (Travel etc)    

         

Data Handling Coding and Editing

   

Data Entry    

Post Cleaning    

Data Analysis    

Estimated Budget for Bahamas Emergency Estimated Budget for Bahamas Emergency Room Drug Survey (Cont’d)Room Drug Survey (Cont’d)

Report Writing Tabulation and Graphics

   

Draft Report    

Editing of Report    

Printing and Reproduction

   

   

Administrative Costs

?

Coordinator Fees    

Supervisors Fees    

Supplies    

Questionnaire and other Printing

   

General administrative costs

   

Total Estimated Cost    

Survey SchedulingSurvey Scheduling

Scheduling TipsScheduling Tips Considerable time and expense can be saved by Considerable time and expense can be saved by

undertaking several streams of activity undertaking several streams of activity simultaneouslysimultaneously

Anticipating which efforts need to be completed Anticipating which efforts need to be completed before the next steps can proceed before the next steps can proceed

Schedule should not be too abbreviated as there Schedule should not be too abbreviated as there may be some unexpected developmentsmay be some unexpected developments

Effort should be made to make realistic estimates Effort should be made to make realistic estimates of the time necessary to complete each line of of the time necessary to complete each line of activityactivity

Field staff should not be promised work until the Field staff should not be promised work until the investigators actually expect to proceed with the investigators actually expect to proceed with the data collection (increased costs)data collection (increased costs)

Survey SchedulingSurvey Scheduling

Hospital RecruitmentHospital Recruitment Data CollectionData Collection Analysis and reportingAnalysis and reporting Dissemination and utilization Dissemination and utilization

activitiesactivities

Scheduling: Hospital Scheduling: Hospital RecruitmentRecruitment

If participation is decided by central edict If participation is decided by central edict (Ministry of Health), the process may be fairly (Ministry of Health), the process may be fairly rapid.rapid.– If involved in the planning, this assures their willingness If involved in the planning, this assures their willingness

to cooperateto cooperate If the individual hospitals or hospital authorities If the individual hospitals or hospital authorities

have the authority to decline cooperation, the have the authority to decline cooperation, the process of securing cooperation can be a process of securing cooperation can be a substantial and time-consuming one.substantial and time-consuming one.– Investigator(s) may have to write to each hospital Investigator(s) may have to write to each hospital

administrator inviting participationadministrator inviting participation– Conduct a follow-up call (or possibly a series of calls) to Conduct a follow-up call (or possibly a series of calls) to

urge the hospital’s participation and answer questionsurge the hospital’s participation and answer questions– Communicate with higher authorities, if their approval is Communicate with higher authorities, if their approval is

also required. also required.

Scheduling: Data CollectionScheduling: Data Collection

Once the main data collection is proceeding, the Once the main data collection is proceeding, the responsible investigators should be monitoring responsible investigators should be monitoring carefully the quality of the data being collected, carefully the quality of the data being collected, to be sure that those collecting it in the field are to be sure that those collecting it in the field are following instructions, and to identify problems following instructions, and to identify problems early that might be rectified.early that might be rectified.

Plans can also be underway for how the data is to Plans can also be underway for how the data is to be collected and forwarded and how the returned be collected and forwarded and how the returned data are to be coded and/or editeddata are to be coded and/or edited

Also, the investigators can begin planning the Also, the investigators can begin planning the analyses that they would like to conduct at the analyses that they would like to conduct at the completion of data collection and data cleaning.completion of data collection and data cleaning.

Scheduling: Analysis and Scheduling: Analysis and ReportingReporting

Often not accorded the attention that they Often not accorded the attention that they deserve because not enough time and resources deserve because not enough time and resources were set aside for them at the outset of the study.were set aside for them at the outset of the study.

Be sure to leave a significant interval for the Be sure to leave a significant interval for the analysis, interpretation, and writing of results.analysis, interpretation, and writing of results.

At this late stage, normally only the lead At this late stage, normally only the lead investigator(s) and an analyst or two are still investigator(s) and an analyst or two are still needed on the study staff, perhaps in addition to needed on the study staff, perhaps in addition to a secretary.a secretary.

Scheduling: Dissemination Scheduling: Dissemination and Utilization Activitiesand Utilization Activities

Once the report is completed, arrangements Once the report is completed, arrangements should be made to get it into the hands of should be made to get it into the hands of people likely to be influenced by its results.people likely to be influenced by its results.

The lead investigators may wish to meet The lead investigators may wish to meet with certain groups, or make presentations with certain groups, or make presentations to particular audiences to whom the work to particular audiences to whom the work has relevance. has relevance. – Policy and programmatic levelPolicy and programmatic level

Flow Chart of Activities for the Flow Chart of Activities for the Conduct of a School SurveyConduct of a School Survey

  INSTRUMENT INSTRUMENT DEVELOPMENT AND DEVELOPMENT AND

REFINEMENTREFINEMENT

Develop questionnaireDevelop questionnaire

↓↓

Pretest questionnairePretest questionnaire

↓↓

Pilot test questionnaire Pilot test questionnaire & &

field proceduresfield procedures

↓↓

Make final revisions to Make final revisions to questionnaire & field questionnaire & field

proceduresprocedures

PERSONNELPERSONNEL

Select lead Select lead investigatorsinvestigators

↓↓

Hire core staffHire core staff

    

FIELD FIELD PREPARATIONSPREPARATIONS

Develop field Develop field proceduresprocedures

↓↓

Hire and train field Hire and train field staff staff

(if applicable)(if applicable)

↓↓

Develop instruction Develop instruction manual manual

for survey leadersfor survey leaders

  SAMPLE & SAMPLE & HOSPITAL HOSPITAL

RECRUITMENTRECRUITMENT

Design sampling planDesign sampling plan

↓↓

Secure data for Secure data for

selecting hospital selecting hospital sample sample

↓↓

Draw sample of Draw sample of hospitalshospitals

↓↓

Recruit the sampled Recruit the sampled hospitalshospitals

↓↓

Schedule the data Schedule the data collection collection

in each hospitalin each hospital

Flow Chart of Activities for the Flow Chart of Activities for the Conduct of a E.R. SurveyConduct of a E.R. Survey

CONDUCTING CONDUCTING

THE MAIN SURVEYTHE MAIN SURVEY

Conduct full data collectionConduct full data collection

↓↓

Code/clean/edit questionnairesCode/clean/edit questionnaires

↓↓

Enter dataEnter data

↓↓

Build computer fileBuild computer file

↓↓

Conduct data analysesConduct data analyses

↓↓

Write reportsWrite reports

↓↓

Disseminate/utilize resultsDisseminate/utilize results

Ethical ConsiderationsEthical Considerations Data must remain completely confidential or Data must remain completely confidential or

completely anonymouscompletely anonymous– no identifying information on a patient’s questionnaireno identifying information on a patient’s questionnaire

If specific identifying information for the If specific identifying information for the individual is contained on the questionnaires de-individual is contained on the questionnaires de-identify at the earliest possible stageidentify at the earliest possible stage– Store the personally identifying information separately Store the personally identifying information separately

from the answers to the rest of the questionnaire with from the answers to the rest of the questionnaire with some type of link systemsome type of link system

Avoid publicly identifying individual hospitals if Avoid publicly identifying individual hospitals if possiblepossible

Permission to participate at all levelsPermission to participate at all levels Commitment to use resultsCommitment to use results

Emergency Room Emergency Room SurveySurvey

QUESTIONNAIRE QUESTIONNAIRE DEVELOPMENTDEVELOPMENT

Factors That Impact Final Factors That Impact Final ContentContent

1.1. Amount of time the medical staff can afford to Amount of time the medical staff can afford to make available in an A&E settingmake available in an A&E setting

– Quality, completenessQuality, completeness– Treatment Vs PreventionTreatment Vs Prevention

2.2. SIDUC RulesSIDUC Rules3.3. Possibility of including other topics due to cost Possibility of including other topics due to cost

sharing with other agenciessharing with other agencies4.4. Ability of respondents to complete the Ability of respondents to complete the

questionnairequestionnaire– Complexity should not exceed respondents’ Complexity should not exceed respondents’

capabilitiescapabilities

Levels of PriorityLevels of Priority

Highly Highly RecommendedRecommended

Necessary to achieve Necessary to achieve objectives. Questions likely objectives. Questions likely to be important to almost to be important to almost any epidemiology study of any epidemiology study of substance usesubstance use

RecommendedRecommended Those that should be given Those that should be given very serious consideration very serious consideration Risk and Protective FactorsRisk and Protective Factors

–Other sources– Injury Other sources– Injury Surv.Surv.

Questions from Questions from StakeholdersStakeholders

OptionalOptional May not measure concepts May not measure concepts of vital interest in every of vital interest in every studystudy

Sequence and Priority of Sequence and Priority of Elements In The Elements In The QuestionnaireQuestionnaire

IntroductionIntroduction Background and demographic Background and demographic

characteristicscharacteristics– Age, Gender, Employment, OccupationAge, Gender, Employment, Occupation

Use of alcoholUse of alcohol Use of illicit substancesUse of illicit substances Use of controlled substancesUse of controlled substances

List of DrugsList of Drugs

AlcoholAlcohol Cocaine HCl.Cocaine HCl. Coca PasteCoca Paste

MarijuanaMarijuana TranquilizersTranquilizers MethamphetaminMethamphetamineses

Solvents andSolvents and

InhalantsInhalantsSedativesSedatives Flunitrazepam Flunitrazepam

(Rohypnol, etc)(Rohypnol, etc)

HallucinogensHallucinogens AmphetaminesAmphetamines Crack CocaineCrack Cocaine

HeroineHeroine AnticholinergicAnticholinergicss

EcstasyEcstasy

Opium/Opium/MorphineMorphine

Anti-Anti-depressantsdepressants

OtherOther

Defining Drugs For The Defining Drugs For The RespondentRespondent

Names and descriptions must be reviewed to see if they Names and descriptions must be reviewed to see if they are are appropriate in the cultural setting in which they will be askedappropriate in the cultural setting in which they will be asked..

The main point is that the names used accurately The main point is that the names used accurately communicate to respondents which substance(s) communicate to respondents which substance(s) shouldshould be be included in what they report, and which substances included in what they report, and which substances should should notnot, and under what circumstances., and under what circumstances.

To determine a list of appropriate slang or street names for To determine a list of appropriate slang or street names for various drugs, the investigators may want to speak with various drugs, the investigators may want to speak with treatment professionals and known drug users in the age treatment professionals and known drug users in the age group under study.group under study.

If a longer question stem is required to get the definition If a longer question stem is required to get the definition clarified for your respondents, then that may be justified. (See clarified for your respondents, then that may be justified. (See SIDUC Manual)SIDUC Manual)

For legally prescribed drugs, it is important that the For legally prescribed drugs, it is important that the respondents understand what occasions of use they should respondents understand what occasions of use they should and should not report in answering the questions.and should not report in answering the questions.

Definition of DrugsDefinition of Drugs

Formal NameFormal Name Street NameStreet Name

CocaineCocaine Coke, crack, white Coke, crack, white lady, blow, etc.lady, blow, etc.

MarijuanaMarijuana Weed, herb, grass, Weed, herb, grass, refer, pot, ganja, etc.refer, pot, ganja, etc.

Example of A Stem Example of A Stem ChangeChange

Concern: General question Vs Concern: General question Vs specific questions that will ensure specific questions that will ensure that all questions are answered by that all questions are answered by all respondentsall respondents

Original (OAS)Original (OAS) ChangeChangeHave you taken any Have you taken any drugs within the 6 drugs within the 6 hours preceding your hours preceding your injuryinjury

Have you taken any of Have you taken any of the following within the the following within the 6 hours preceding your 6 hours preceding your injuryinjury

–AlcoholAlcohol–Marajuana etcMarajuana etc

Layout In The Event of Lab Layout In The Event of Lab ConfirmationConfirmation

Blind Vs Not BlindBlind Vs Not Blind If Not, questions and Lab results on same If Not, questions and Lab results on same

form form – No means to connect necessaryNo means to connect necessary

IF blind, questions and lab results are IF blind, questions and lab results are separatedseparated– Unique identifiers are needed to re-uniteUnique identifiers are needed to re-unite

NB: identifiers can be added after A&E but NB: identifiers can be added after A&E but before transfer to labbefore transfer to lab

Other Useful SuggestionsOther Useful Suggestions Skip patternsSkip patterns are more difficult for are more difficult for

respondents to follow correctly than a simple, respondents to follow correctly than a simple, uninterrupted series of questions; thereforeuninterrupted series of questions; therefore, , it it is advised that they be minimized.is advised that they be minimized.

Pre-testPre-test by getting a limited number of “like” by getting a limited number of “like” respondents to complete the questionnaire.respondents to complete the questionnaire.– Determine average time to complete.Determine average time to complete.– individually interview them about whether the individually interview them about whether the

instructions were clear and whether there were any instructions were clear and whether there were any questions or answers that they had difficulty questions or answers that they had difficulty understanding or using.understanding or using.

– Determine if they understood each class of drugDetermine if they understood each class of drug

Other Useful Suggestions Other Useful Suggestions Cont’dCont’d

Pilot testing to:Pilot testing to:– see how the actual administration see how the actual administration

procedures in the ER will go,procedures in the ER will go,– see how long it takes the patients to see how long it takes the patients to

answer the questions andanswer the questions and– identify remaining problems in the identify remaining problems in the

content and clarity of the questionnaire. content and clarity of the questionnaire.

Emergency Room Emergency Room SurveySurvey

Data Collection ProcedureData Collection Procedure

Decisions Prior to CollectionDecisions Prior to Collection

WHATWHAT What data will be collected What data will be collected WHOWHOWho will be responsible for data Who will be responsible for data

collectioncollection WHENWHEN When to collect data to ensure When to collect data to ensure

a “normal” samplea “normal” sample WHEREWHERE What hospitals are to be What hospitals are to be

includedincluded HOWHOWHow to conduct all aspects of How to conduct all aspects of

survey administrationsurvey administration

What data will be collectedWhat data will be collected

QuestionnairesQuestionnaires– SIDUCSIDUC– BahamasBahamas

Lab confirmation dataLab confirmation data Site reports on:Site reports on:

– # refusals# refusals– # ineligibles# ineligibles

Who will be responsible for Who will be responsible for data collectiondata collection

Conducting the interview of each section of the Conducting the interview of each section of the questionnairequestionnaire– How to select survey leadersHow to select survey leaders

A&E Data FlowA&E Data Flow– Lab sample flowLab sample flow– ““routinely” collected informationroutinely” collected information

Ensuring that a continuous supply of Ensuring that a continuous supply of questionnaires are availablequestionnaires are available

Ensuring a continuous supply of lab sampling Ensuring a continuous supply of lab sampling equipment are available (?)equipment are available (?)

Ensuring that completed forms and lab samples Ensuring that completed forms and lab samples are routed properlyare routed properly

Selection of Survey LeaderSelection of Survey LeaderER StaffER Staff

ProsPros::– already in the already in the ERER– know the know the ERER and are and are

familiar with familiar with ERER routines routines– Patients may feel more Patients may feel more

comfortable with MD or comfortable with MD or NurseNurse

– least expensive least expensive ConsCons::

– Competing interestsCompeting interests – ER Services Vs survey ER Services Vs survey

completioncompletion

Research AssistantResearch Assistant ProsPros::

– More consistency More consistency across hospitalsacross hospitals

ConsCons– More expensiveMore expensive– doubts regarding the doubts regarding the

ability to ability to convince patients convince patients to participateto participate

E.g., Social Workers in E.g., Social Workers in HaitiHaiti

When to collect dataWhen to collect data

SIDUC - SampleSIDUC - Sample

Vs Vs

Institutionalized – routinely collected Institutionalized – routinely collected as part of diagnosisas part of diagnosis

Timing of Data CollectionTiming of Data Collection IIt is important to choose a period which should not be preceded t is important to choose a period which should not be preceded

by any holiday, ensuring that the by any holiday, ensuring that the patientspatients refer to a “normal” refer to a “normal” week or month when answering the questionnaireweek or month when answering the questionnaire

If more than one If more than one hospitalhospital is included, it is a must to administer is included, it is a must to administer the data collection at the same the data collection at the same timetime in all in all hospitals hospitals

In the event of inter-country comparisons (SIDUC), In the event of inter-country comparisons (SIDUC), the time of the time of data collection must be as similar as possibledata collection must be as similar as possible since the use of since the use of alcohol and other drugs alcohol and other drugs vary at certain identifiable periods.vary at certain identifiable periods.

e.g. e.g. December (Christmas) ↑December (Christmas) ↑

January (Recuperation) ↓January (Recuperation) ↓

Holidays ↑Holidays ↑

Lent ↓Lent ↓

Where will data collection take Where will data collection take placeplace

Geographic locationGeographic location– IslandsIslands

HospitalsHospitals– PublicPublic

Princess Margaret (Nassau)Princess Margaret (Nassau) Rand Memorial (Freeport)Rand Memorial (Freeport)

– PrivatePrivate Doctors (Nassau)Doctors (Nassau)

Activities During Activities During AdministrationAdministration

IInstructions to the Surveynstructions to the Survey Data Data Collection StaffCollection Staff

Instructions to the patientsInstructions to the patients SStress anonymity and confidentialitytress anonymity and confidentiality

Who Should Complete Who Should Complete SurveySurvey

1.1. Inclusion CriteriaInclusion Criteria

2.2. Exclusion CriteriaExclusion Criteria• Persons < 12 years of agePersons < 12 years of age• Gynae patientsGynae patients• Language barriers (Non-English speaking)Language barriers (Non-English speaking)• Unconscious patients (?)Unconscious patients (?)

• If hospitalized and no identifiers to follow-upIf hospitalized and no identifiers to follow-up

Patients who RefusePatients who Refuse

Concern is dConcern is due to the possible ue to the possible association between refusal and drug association between refusal and drug use:use:

It is important that the number of refusals It is important that the number of refusals be documentedbe documented

Could compare % refusals from low and Could compare % refusals from low and high prevalence areashigh prevalence areas

It is important that It is important that refusalsrefusals are treated in are treated in the same way in all participating the same way in all participating hospitals hospitals and countries.and countries.

Survey Administration:Survey Administration:Instructions to PatientsInstructions to Patients

Survey Instructions can Survey Instructions can be written on the front be written on the front pagepage of the questionnaire and should include of the questionnaire and should include information information on:on:

– The The purpose of the study (If not incorporated into the purpose of the study (If not incorporated into the system) system)

– TThe selection of he selection of patients (All selected during study period)patients (All selected during study period)– TThe study is anonymous andhe study is anonymous and/or /or confidentialconfidential– Steps to ensure anonymitySteps to ensure anonymity– Participation is important but if not willing, that is their Participation is important but if not willing, that is their

choicechoice

To Ensure To Ensure AnonymAnonymityity

Following SIDUC MethodFollowing SIDUC Method QuestionnairesQuestionnaires should not contain any should not contain any

identifiersidentifiers– NameName– hospital record #, etc.hospital record #, etc.

Survey Administration:Survey Administration:ReportReport

AA report should be completed (periodically) by report should be completed (periodically) by the survey leader the survey leader – Total number of refusalsTotal number of refusals– If the study only targets a specific group, the If the study only targets a specific group, the

number of those excludednumber of those excluded

Checklist Checklist For Data For Data CollectionCollection

Chose a survey leader trusted by the Chose a survey leader trusted by the patientspatients Instructions to the survey leaderInstructions to the survey leader Describe how to treat Describe how to treat patientspatients not belonging to the target not belonging to the target

populationpopulation Describe how to treat Describe how to treat refusalsrefusals Careful planning of the contacts with selected Careful planning of the contacts with selected institutionsinstitutions Safe transportation of materialSafe transportation of material (lab component) (lab component) Detailed planning of the survey administration, including Detailed planning of the survey administration, including

– stress anonymity and confidentialitystress anonymity and confidentiality– instructions to the survey leaderinstructions to the survey leader– instructions to the instructions to the patientspatients– Shift Shift reportreport

Overview of Overview of Methodological Methodological

Issues:Issues:

Interpretation Of DataInterpretation Of Data

Overview of SessionOverview of Session

Representativeness – Representativeness – the extent to which the extent to which a sample mirrors the population of interesta sample mirrors the population of interest

Reliability – Reliability – the extent to which repeated the extent to which repeated measurements used under the same conditions measurements used under the same conditions produce the same resultproduce the same result

Validity – Validity – the extent to which answers are the extent to which answers are accurate representations of the underlying reality accurate representations of the underlying reality that they are intended to measurethat they are intended to measure

Factors that Influence Factors that Influence RepresentativenessRepresentativeness

The sampling methodThe sampling method The size of the sampleThe size of the sample Response rate: The number of Response rate: The number of

eligible non-gynae emergency room eligible non-gynae emergency room patients that agree to participate in patients that agree to participate in the surveythe survey

The Method of Sampling:The Method of Sampling: Target Population Target Population

The target population in The target population in the Emergency Room the Emergency Room survey is, by definition, survey is, by definition, persons persons who who chose to chose to come to the emergency roomcome to the emergency room and excluding and excluding other other individualsindividuals who, for whatever reason, chose not who, for whatever reason, chose not to attend.to attend.

– Beware that all injured persons do not come in due to a Beware that all injured persons do not come in due to a number of reasons (choice, inaccessibility, etc.)number of reasons (choice, inaccessibility, etc.)

– Less-severely injured persons Less-severely injured persons who who do not come in may be the do not come in may be the result of reasons related to result of reasons related to substancesubstance use use

– Large Large differences in differences in emergency roomemergency room attendance (access, attendance (access, availability) between countries may make it difficult/impossible availability) between countries may make it difficult/impossible to make meaningful international comparisons to make meaningful international comparisons

Method of Achieving Method of Achieving RepresentativenessRepresentativeness

Random sampling is fundamental to Random sampling is fundamental to obtaining a sample that is obtaining a sample that is representative of the populationrepresentative of the population

OROR Sample the entire target groupSample the entire target group

Sample Size and Sample Size and RepresentativenessRepresentativeness

the number of sampled the number of sampled patientspatients must must be of sufficient sizebe of sufficient size

considerations considerations must be given to the must be given to the extent of the extent of the analysis of drug habits in analysis of drug habits in different subgroupsdifferent subgroups– always important to sample enough always important to sample enough patientspatients to be to be

able to analyse data separately for able to analyse data separately for males and males and femalesfemales

Response rate must be sufficiently high Response rate must be sufficiently high to enable representative data to be to enable representative data to be obtainedobtained

Categories of Response Categories of Response RatesRates

Institutional CooperationInstitutional Cooperation– Public and Private HospitalsPublic and Private Hospitals

Patient Cooperation For InterviewPatient Cooperation For Interview Proportion Participating in Lab Proportion Participating in Lab

AnalysisAnalysis

Issues Impacting Issues Impacting Institutional CooperationInstitutional Cooperation

AAn increased number of n increased number of proposedproposed surveys has made surveys has made hospitalshospitals in some in some countries somewhat reluctant to countries somewhat reluctant to allow allow patients topatients to participate participate directly directly in such in such surveyssurveys– highlightshighlights the necessity of approaching the necessity of approaching

institutions with emergency rooms formally institutions with emergency rooms formally and and in a way that makes them feel that in a way that makes them feel that they are part of an important studythey are part of an important study

Issues Impacting Patient Issues Impacting Patient ParticipationParticipation

PParticipation should always be voluntaryarticipation should always be voluntary Suspicions regarding “True” objectivesSuspicions regarding “True” objectives has made has made

patientspatients somewhat reluctant to take time to somewhat reluctant to take time to participate in such participate in such drug drug surveyssurveys– highlightshighlights the necessity of approaching the necessity of approaching patientspatients in a way in a way

that makes them feel that they are part of an important that makes them feel that they are part of an important studystudy

AAll questionnaires ll questionnaires shouldshould be treated confidential be treated confidentiallyly– TThe use of questionnaires without names or other kinds he use of questionnaires without names or other kinds

of identificationof identification– GGuaranties of confidential treatment of questionnaires uaranties of confidential treatment of questionnaires

and dataand data– PPromises not to report data for individual romises not to report data for individual patientspatients

Decisions Regarding Non-Decisions Regarding Non-ParticipationParticipation

If If hospitals with a large percentage of hospitals with a large percentage of overall ER visits overall ER visits refuse to participate it is refuse to participate it is essential to do a careful analysis of the essential to do a careful analysis of the reasonsreasons

IfIf systematic errors systematic errors isis suspected, suspected, interpretation might be difficult and interpretation might be difficult and international comparisons may be international comparisons may be jeopardised.jeopardised.E.g. E.g. – refusing refusing patientspatients come from areas where drug come from areas where drug and/or and/or

alcoholalcohol consumption is known to be high consumption is known to be high

ReliabilityReliability

The extent to which repeated The extent to which repeated measurements used under the same measurements used under the same conditions produce the same resultconditions produce the same result

OrOr

Whether an indicator is consistent across Whether an indicator is consistent across time and observerstime and observers

Methods of Measuring Methods of Measuring Survey ReliabilitySurvey Reliability

To conduct repeated studies using To conduct repeated studies using same methodologysame methodology

BBy using data from different y using data from different questions within a questionnairequestions within a questionnaire

Laboratory ConfirmationLaboratory Confirmation

Factors Influencing Study ReliabilityFactors Influencing Study Reliability

CompletenessCompleteness - - proportion of total proportion of total information information knownknown– All respondents answer questionsAll respondents answer questions– All targeted respondents take surveyAll targeted respondents take survey

Sources of ErrorSources of Error– InstrumentInstrument– PersonPerson

MD, Nurse, Med. Rec ClerkMD, Nurse, Med. Rec Clerk Study Subject or Respondent Study Subject or Respondent

– May occur at May occur at subject selectionsubject selection data collection (Subjective)data collection (Subjective)

– Kappa StatisticKappa Statistic analysis and interpretationanalysis and interpretation

Sources of Error: Sources of Error: Instrument and ProceduresInstrument and Procedures

Poor selection of questions and variables Poor selection of questions and variables (SIDUC)(SIDUC)

Ambiguous questions (additional questions)Ambiguous questions (additional questions)

Non-response Non-response

Poor selection of study subjects (friendliest)Poor selection of study subjects (friendliest)

Failure to validate measurements, recordingsFailure to validate measurements, recordings

– lablab

Sources of Error:Sources of Error:PersonPerson

InterviewerInterviewer Inter-observer variation Inter-observer variation

– the way medical staff explain and/or ask the way medical staff explain and/or ask questionsquestions

– Opinions on extent drug/alcohol use contributed Opinions on extent drug/alcohol use contributed to injuryto injury

SubjectSubject Inter-subject variationInter-subject variation

– Different Interpretation of questions (what drugs Different Interpretation of questions (what drugs were thought of without prompting)were thought of without prompting)

– willingness, survey environmentwillingness, survey environment

To Achieve Reliable DataTo Achieve Reliable Data

Thorough knowledge of study populationThorough knowledge of study population Unbiased selection of study subjectsUnbiased selection of study subjects Standardized, calibrated and consistent Standardized, calibrated and consistent

instrument(s)instrument(s) Clear, unambiguous questionsClear, unambiguous questions Minimal personal discomfort or Minimal personal discomfort or

discomfiture discomfiture Interviewers, recorders thoroughly and Interviewers, recorders thoroughly and

equally well trained equally well trained Need to validate Need to validate

ValidityValidity

The extent to which answers are accurate The extent to which answers are accurate representations of the underlying reality representations of the underlying reality that they are intended to measure.that they are intended to measure.

In the context of emergency room surveys, In the context of emergency room surveys, the degree to which the questionnaire the degree to which the questionnaire measures the aspects of patients’ drug measures the aspects of patients’ drug consumption that it was intended to consumption that it was intended to measure.measure.

Strategies to Enhance Strategies to Enhance ValidityValidity

Studies must guarantee anonymity and Studies must guarantee anonymity and confidentialityconfidentiality– One is to use a data collection leader trusted by the One is to use a data collection leader trusted by the

patientspatients– Stress Stress anonymity anonymity during the during the introduction introduction – no names or other identification marks should be on the no names or other identification marks should be on the

questionnaire (SIDUC)questionnaire (SIDUC) The patients must also have enough time to The patients must also have enough time to

answer the questionnaireanswer the questionnaire– important that the questionnaire is not too longimportant that the questionnaire is not too long

TThey must understand the questionshey must understand the questions TThey must be willing to answer the questions hey must be willing to answer the questions

honestlyhonestly

Strategies to Assess ValidityStrategies to Assess Validity

PatientsPatients’’ willingness to co-operate willingness to co-operate Patients’Patients’ comprehension comprehension MMissing data ratesissing data rates LLogical consistencyogical consistency RReported willingness to answer honestlyeported willingness to answer honestly TThe cultural context in which a survey is he cultural context in which a survey is

conductedconducted

Strategies to Assess Strategies to Assess Validity:Validity:

C Cultural ultural CContext ontext QQuestions uestions must be must be culturally or locally appropriateculturally or locally appropriate– Use Use the appropriate “street-names” or “nicknames” used the appropriate “street-names” or “nicknames” used

for different drugsfor different drugs WWillingness to admit drug use may be influenced by illingness to admit drug use may be influenced by

the attitudes towards drugs in a given societythe attitudes towards drugs in a given society– perceived risk of substance useperceived risk of substance use, , disapproval of different disapproval of different

kinds of substance usekinds of substance use and and the availability of different the availability of different drugs differ between countries.drugs differ between countries.

– Low availability + Negative attitude = Low availability + Negative attitude = less willing less willing to admitto admit– High availability + Positive attitudes = more willing to High availability + Positive attitudes = more willing to

admitadmit Survey TraditionSurvey Tradition

– PersonPersons in countries where surveys are less common s in countries where surveys are less common may may feel less comfortable answering questions about sensitive feel less comfortable answering questions about sensitive behaviours. behaviours.

Checklist For Survey Checklist For Survey MethodologyMethodology

RepresentativenessRepresentativeness– Define the target populationDefine the target population– Assess the importance of non-students in the same age Assess the importance of non-students in the same age

groups as the target populationgroups as the target population– Decide a proper time for the data collection (if Decide a proper time for the data collection (if

international comparisons are planned)international comparisons are planned)– Assess the importance of non-participating Assess the importance of non-participating

schools/classesschools/classes– Assess the importance of non-participating studentsAssess the importance of non-participating students

ReliabilityReliability– Assess reliability (whenever possible by using data in Assess reliability (whenever possible by using data in

the questionnaire)the questionnaire)

Checklist For Survey Checklist For Survey Methodology Cont’dMethodology Cont’d

ValidityValidity– Anonymous and confidential data collectionAnonymous and confidential data collection– Measure and reportMeasure and report– Number of eliminated questionnairesNumber of eliminated questionnaires– Survey leader information (from the classroom Survey leader information (from the classroom

report)report)– Time to answer the questionnaireTime to answer the questionnaire– Proportion of unanswered questionsProportion of unanswered questions– Logical consistencyLogical consistency– Possible use of a “willingness question”Possible use of a “willingness question”– Possible reported use of a dummy drugPossible reported use of a dummy drug– Construct validityConstruct validity

The EndThe End

ReallyReally

The EndThe End