emergency room survey survey planning, administration and cost
TRANSCRIPT
Emergency Room Emergency Room SurveySurvey
Survey Planning, Survey Planning, Administration and CostAdministration and Cost
SIDUC: AN INTEGRATED SYSTEM OF SIDUC: AN INTEGRATED SYSTEM OF DRUG USE SURVEYSDRUG USE SURVEYS
Key Informants
SIDUC STANDARDIZATIONSIDUC STANDARDIZATION
I nformation Source
Reference Population
Analized Population
Data Collection
Periodicity
Treatment
Centers
Patients
Patients
I nterview
Continue
Emergency
Rooms
Patients
12 to 64 years
Sample of hospitals and
Patients
I nterview
Annual
Student Surveys
Students
Grades 8, 10, 12
Sample of Schools & classes
Self admnistered
Questionnaire
2 years
Household Surveys
Population in households
12 to 64 years
Sample of areas,
Households & persons
I nterview / Self admin
3 years
EMERGENCY ROOMSEMERGENCY ROOMS
Population of referencePopulation of reference:: Patients of Emergency Rooms.Patients of Emergency Rooms.
Analyzed populationAnalyzed population:: a) all patients b) “typical weeks”. a) all patients b) “typical weeks”.
Judgmental sample.Judgmental sample. Method of data collectionMethod of data collection::
Interview & laboratory analysis.Interview & laboratory analysis.
FORENSIC MEDICINEFORENSIC MEDICINE
Population of reference:Population of reference: Death by accident, homicide, Death by accident, homicide,
suicide.suicide. Analyzed Population:Analyzed Population:
a) all , b) “Typical weeks”.a) all , b) “Typical weeks”. Method of data collectionMethod of data collection::
Autopsy and laboratory analysis.Autopsy and laboratory analysis.
Steps to be TakenSteps to be Taken
A funding source or multiple sources A funding source or multiple sources must be found (or no survey)must be found (or no survey)
Decisions must then be made about Decisions must then be made about who will direct the research who will direct the research undertakingundertaking
What types of expert and staff support What types of expert and staff support will be needed, and will be needed, and
What is a realistic timetableWhat is a realistic timetable– for the overall endeavorfor the overall endeavor– for the major components for the major components
Session OutlineSession Outline
Core Survey ActivitiesCore Survey Activities PersonnelPersonnel BudgetBudget ScheduleSchedule
Core ActivitiesCore Activities
1.1. Establishment of Advisory CommitteeEstablishment of Advisory Committee
2.2. Develop ProposalDevelop Proposal
3.3. Seek Permission to conductSeek Permission to conduct
4.4. Finalize MethodologyFinalize Methodology
5.5. TrainingTraining
6.6. Survey ImplementationSurvey Implementation• Data Collection, Handling, Analysis, Data Collection, Handling, Analysis,
Report WritingReport Writing
Advisory CommitteeAdvisory Committee
Stakeholders identification/representationStakeholders identification/representation Ministry of HealthMinistry of Health Public Hospital AuthorityPublic Hospital Authority ER DirectorER Director Police Police Ministry of TransportMinistry of Transport Rep from private hospitalRep from private hospital COB Research UnitCOB Research Unit
N.B> Can avoid problems at the end N.B> Can avoid problems at the end Can help to “sell” the surveyCan help to “sell” the survey
Communications Regarding Communications Regarding SurveySurvey
Ministerial Responsibility (CMO)Ministerial Responsibility (CMO) Institutional ResponsibilityInstitutional Responsibility
– Public Hospital Authority (PMH, RMH)Public Hospital Authority (PMH, RMH) Institutional Review Boards/Ethics Institutional Review Boards/Ethics
Comm.Comm. How to contact the chosen hospitalsHow to contact the chosen hospitals
– AdministratorAdministrator– ER ManagerER Manager
Contact With Selected Contact With Selected InstitutionsInstitutions
Step 1Step 1 AAn introductory letter to the n introductory letter to the AdministratorAdministrator, ,
informing him/her of the study and its informing him/her of the study and its purposespurposes. It will also:. It will also:– Ask tAsk to inform the o inform the ER Manager of ER Manager of
decisiondecision
Step 2Step 2 CContact the ER manager via telephone to ontact the ER manager via telephone to
confirm that everything is in orderconfirm that everything is in order just just prior. At this point:prior. At this point:
Project PersonnelProject Personnel
Lead Investigator(s)Lead Investigator(s) Core StaffCore Staff
– Site managersSite managers– Data ManagersData Managers
Data Collection StaffData Collection Staff– DoctorsDoctors– NursesNurses– Medical Records ClerksMedical Records Clerks
Consultant(s)Consultant(s)
Personnel Selection Process Personnel Selection Process OptionsOptions
1.1. Advisory Committee to provide oversight of the Advisory Committee to provide oversight of the enterprise and the selection of the lead enterprise and the selection of the lead investigatorsinvestigators– AD: can help to engage people or organizations needed AD: can help to engage people or organizations needed
to fund, conduct, or make use of the researchto fund, conduct, or make use of the research– DIS: “too many cooks in the kitchen” and may introduce DIS: “too many cooks in the kitchen” and may introduce
political considerations into the processpolitical considerations into the process– REC: committee should deal with the broad policy and REC: committee should deal with the broad policy and
financial issues and leave the scientific decisions to the financial issues and leave the scientific decisions to the scientists scientists
2.2. Recruitment of one or more individuals to run the Recruitment of one or more individuals to run the projectproject
3.3. Individual researchers may take the initiative to Individual researchers may take the initiative to conduct such a surveyconduct such a survey
Lead InvestigatorLead Investigator
Ideally will be a part of the activity Ideally will be a part of the activity from beginning to endfrom beginning to end
Will provide the planning and Will provide the planning and integration needed to be sure that the integration needed to be sure that the end product matches the needs and end product matches the needs and objectivesobjectives
Should be trained social scientists Should be trained social scientists with some experience in survey with some experience in survey research techniquesresearch techniques
Core StaffCore Staff(Site Managers)(Site Managers)
These key individuals will remain with the These key individuals will remain with the study for its durationstudy for its duration
Participate in various activities and make sure Participate in various activities and make sure that they are carried out according to plan.that they are carried out according to plan.
Most likely will supervise various components Most likely will supervise various components of the study, under the general direction of of the study, under the general direction of the lead investigator(s)the lead investigator(s)
Should be well educated and preferably have Should be well educated and preferably have some experience with research activities.some experience with research activities.
Data Collection StaffData Collection Staff
Decide whether the data will be collected Decide whether the data will be collected by individuals from outside the hospitals by individuals from outside the hospitals or not or not – ER Staff: If the patients will trust the MDs or ER Staff: If the patients will trust the MDs or
Nurse to protect their confidentialityNurse to protect their confidentiality– Outside Staff: If the staffing situation presents Outside Staff: If the staffing situation presents
a challenge, then staff members must be hired, a challenge, then staff members must be hired, trained, supervised and usually compensatedtrained, supervised and usually compensated trained health social workers, trained field trained health social workers, trained field
interviewers from a survey research organization, interviewers from a survey research organization, university students, etcuniversity students, etc
ConsultantsConsultants
Overall PlanningOverall Planning To direct technical assistance at various stages, depending of course on the To direct technical assistance at various stages, depending of course on the
areas of expertise of the lead investigators areas of expertise of the lead investigators If the lead investigators are new to this area of research, consider a short-If the lead investigators are new to this area of research, consider a short-
term consultation with an experienced expert in the initial planning stage, term consultation with an experienced expert in the initial planning stage, again at the analysis planning stage, and perhaps at the interpretation stage again at the analysis planning stage, and perhaps at the interpretation stage
If brought in from abroad, seek the assistance and support of one of the If brought in from abroad, seek the assistance and support of one of the international organizations that deal with the control of drug abuse (UNODC, international organizations that deal with the control of drug abuse (UNODC, OAS)OAS)
Sample PlanningSample Planning Consultation with a sampling statistician also is likely to be very helpful, Consultation with a sampling statistician also is likely to be very helpful,
again at an early point in the planning effort again at an early point in the planning effort
Statistical AnalysisStatistical Analysis If neither a general consultant from the substance abuse field nor statistician If neither a general consultant from the substance abuse field nor statistician
is able to help on data analyses, seek an expert on statistical analyses.is able to help on data analyses, seek an expert on statistical analyses. The job is usually not to actually conduct the analyses but to advise on the The job is usually not to actually conduct the analyses but to advise on the
choice of analyses and appropriate computer programs for conducting them. choice of analyses and appropriate computer programs for conducting them.
Budget PlanningBudget Planning1.1. Personnel CostsPersonnel Costs
2.2. Non-Personnel CostsNon-Personnel Costs
Factors that May Impact Factors that May Impact Personnel BudgetPersonnel Budget
Labor costs in the countryLabor costs in the country Whether it is necessary to use data collection Whether it is necessary to use data collection
staff from outside the hospitalsstaff from outside the hospitals The method of data entryThe method of data entry The need for technical assistanceThe need for technical assistance The size of the sample needed and the size of the The size of the sample needed and the size of the
country across which that sample will be spreadcountry across which that sample will be spread– influence both staff and travel costsinfluence both staff and travel costs
Amount of the personnel assigned to plan and Amount of the personnel assigned to plan and conduct the ER survey who are already on the conduct the ER survey who are already on the payroll of a participating agency.payroll of a participating agency.
Sample Budget OutlineSample Budget Outline
PERSONNEL COSTSPERSONNEL COSTS
Lead investigatorsLead investigators
Field staff (if applicable) for __ weeksField staff (if applicable) for __ weeks–Supervisors/Site ManagerSupervisors/Site Manager–Facilitators/MDs, Nurse, Med Rec.Facilitators/MDs, Nurse, Med Rec.
Core support staffCore support staffSecretarial/ClericalSecretarial/Clerical
Subcontractor CostsSubcontractor Costs–For data entry (if applicable)For data entry (if applicable)–For any other services being purchasedFor any other services being purchased
ConsultantsConsultants–Fringe benefitsFringe benefits
NON-SALARY COSTSNON-SALARY COSTS
OfficeOffice–RRental (if applicable)ental (if applicable)–FFurniture (if applicable)urniture (if applicable)–EEquipment (as needed)quipment (as needed)–PhonesPhones, , FaxFax, , Copy machineCopy machine, , ComputersComputers–SSuppliesupplies–Telephone serviceTelephone service
Advertising/Recruiting CostsAdvertising/Recruiting CostsPrintingPrinting
–QuestionnairesQuestionnaires–ManualsManuals–Brochures, instruction sheets, etc.Brochures, instruction sheets, etc.–Final reportFinal report
Shipping and postageShipping and postage–Questionnaires to hospitalsQuestionnaires to hospitals–Questionnaires back from hospitalsQuestionnaires back from hospitals–Other Other
Travel CostsTravel Costs–For investigatorsFor investigators–For field staff (if applicable)For field staff (if applicable)
Overhead charges (if any)Overhead charges (if any)
Estimated Budget for Bahamas Emergency Estimated Budget for Bahamas Emergency Room Drug SurveyRoom Drug Survey
Phase Specific Tasks Time Frame
Cost ($US)
Planning
Data Collection Training
Payment to Facilitators
Other (Travel etc)
Data Handling Coding and Editing
Data Entry
Post Cleaning
Data Analysis
Estimated Budget for Bahamas Emergency Estimated Budget for Bahamas Emergency Room Drug Survey (Cont’d)Room Drug Survey (Cont’d)
Report Writing Tabulation and Graphics
Draft Report
Editing of Report
Printing and Reproduction
Administrative Costs
?
Coordinator Fees
Supervisors Fees
Supplies
Questionnaire and other Printing
General administrative costs
Total Estimated Cost
Survey SchedulingSurvey Scheduling
Scheduling TipsScheduling Tips Considerable time and expense can be saved by Considerable time and expense can be saved by
undertaking several streams of activity undertaking several streams of activity simultaneouslysimultaneously
Anticipating which efforts need to be completed Anticipating which efforts need to be completed before the next steps can proceed before the next steps can proceed
Schedule should not be too abbreviated as there Schedule should not be too abbreviated as there may be some unexpected developmentsmay be some unexpected developments
Effort should be made to make realistic estimates Effort should be made to make realistic estimates of the time necessary to complete each line of of the time necessary to complete each line of activityactivity
Field staff should not be promised work until the Field staff should not be promised work until the investigators actually expect to proceed with the investigators actually expect to proceed with the data collection (increased costs)data collection (increased costs)
Survey SchedulingSurvey Scheduling
Hospital RecruitmentHospital Recruitment Data CollectionData Collection Analysis and reportingAnalysis and reporting Dissemination and utilization Dissemination and utilization
activitiesactivities
Scheduling: Hospital Scheduling: Hospital RecruitmentRecruitment
If participation is decided by central edict If participation is decided by central edict (Ministry of Health), the process may be fairly (Ministry of Health), the process may be fairly rapid.rapid.– If involved in the planning, this assures their willingness If involved in the planning, this assures their willingness
to cooperateto cooperate If the individual hospitals or hospital authorities If the individual hospitals or hospital authorities
have the authority to decline cooperation, the have the authority to decline cooperation, the process of securing cooperation can be a process of securing cooperation can be a substantial and time-consuming one.substantial and time-consuming one.– Investigator(s) may have to write to each hospital Investigator(s) may have to write to each hospital
administrator inviting participationadministrator inviting participation– Conduct a follow-up call (or possibly a series of calls) to Conduct a follow-up call (or possibly a series of calls) to
urge the hospital’s participation and answer questionsurge the hospital’s participation and answer questions– Communicate with higher authorities, if their approval is Communicate with higher authorities, if their approval is
also required. also required.
Scheduling: Data CollectionScheduling: Data Collection
Once the main data collection is proceeding, the Once the main data collection is proceeding, the responsible investigators should be monitoring responsible investigators should be monitoring carefully the quality of the data being collected, carefully the quality of the data being collected, to be sure that those collecting it in the field are to be sure that those collecting it in the field are following instructions, and to identify problems following instructions, and to identify problems early that might be rectified.early that might be rectified.
Plans can also be underway for how the data is to Plans can also be underway for how the data is to be collected and forwarded and how the returned be collected and forwarded and how the returned data are to be coded and/or editeddata are to be coded and/or edited
Also, the investigators can begin planning the Also, the investigators can begin planning the analyses that they would like to conduct at the analyses that they would like to conduct at the completion of data collection and data cleaning.completion of data collection and data cleaning.
Scheduling: Analysis and Scheduling: Analysis and ReportingReporting
Often not accorded the attention that they Often not accorded the attention that they deserve because not enough time and resources deserve because not enough time and resources were set aside for them at the outset of the study.were set aside for them at the outset of the study.
Be sure to leave a significant interval for the Be sure to leave a significant interval for the analysis, interpretation, and writing of results.analysis, interpretation, and writing of results.
At this late stage, normally only the lead At this late stage, normally only the lead investigator(s) and an analyst or two are still investigator(s) and an analyst or two are still needed on the study staff, perhaps in addition to needed on the study staff, perhaps in addition to a secretary.a secretary.
Scheduling: Dissemination Scheduling: Dissemination and Utilization Activitiesand Utilization Activities
Once the report is completed, arrangements Once the report is completed, arrangements should be made to get it into the hands of should be made to get it into the hands of people likely to be influenced by its results.people likely to be influenced by its results.
The lead investigators may wish to meet The lead investigators may wish to meet with certain groups, or make presentations with certain groups, or make presentations to particular audiences to whom the work to particular audiences to whom the work has relevance. has relevance. – Policy and programmatic levelPolicy and programmatic level
Flow Chart of Activities for the Flow Chart of Activities for the Conduct of a School SurveyConduct of a School Survey
INSTRUMENT INSTRUMENT DEVELOPMENT AND DEVELOPMENT AND
REFINEMENTREFINEMENT
Develop questionnaireDevelop questionnaire
↓↓
Pretest questionnairePretest questionnaire
↓↓
Pilot test questionnaire Pilot test questionnaire & &
field proceduresfield procedures
↓↓
Make final revisions to Make final revisions to questionnaire & field questionnaire & field
proceduresprocedures
PERSONNELPERSONNEL
Select lead Select lead investigatorsinvestigators
↓↓
Hire core staffHire core staff
FIELD FIELD PREPARATIONSPREPARATIONS
Develop field Develop field proceduresprocedures
↓↓
Hire and train field Hire and train field staff staff
(if applicable)(if applicable)
↓↓
Develop instruction Develop instruction manual manual
for survey leadersfor survey leaders
SAMPLE & SAMPLE & HOSPITAL HOSPITAL
RECRUITMENTRECRUITMENT
Design sampling planDesign sampling plan
↓↓
Secure data for Secure data for
selecting hospital selecting hospital sample sample
↓↓
Draw sample of Draw sample of hospitalshospitals
↓↓
Recruit the sampled Recruit the sampled hospitalshospitals
↓↓
Schedule the data Schedule the data collection collection
in each hospitalin each hospital
Flow Chart of Activities for the Flow Chart of Activities for the Conduct of a E.R. SurveyConduct of a E.R. Survey
CONDUCTING CONDUCTING
THE MAIN SURVEYTHE MAIN SURVEY
Conduct full data collectionConduct full data collection
↓↓
Code/clean/edit questionnairesCode/clean/edit questionnaires
↓↓
Enter dataEnter data
↓↓
Build computer fileBuild computer file
↓↓
Conduct data analysesConduct data analyses
↓↓
Write reportsWrite reports
↓↓
Disseminate/utilize resultsDisseminate/utilize results
Ethical ConsiderationsEthical Considerations Data must remain completely confidential or Data must remain completely confidential or
completely anonymouscompletely anonymous– no identifying information on a patient’s questionnaireno identifying information on a patient’s questionnaire
If specific identifying information for the If specific identifying information for the individual is contained on the questionnaires de-individual is contained on the questionnaires de-identify at the earliest possible stageidentify at the earliest possible stage– Store the personally identifying information separately Store the personally identifying information separately
from the answers to the rest of the questionnaire with from the answers to the rest of the questionnaire with some type of link systemsome type of link system
Avoid publicly identifying individual hospitals if Avoid publicly identifying individual hospitals if possiblepossible
Permission to participate at all levelsPermission to participate at all levels Commitment to use resultsCommitment to use results
Emergency Room Emergency Room SurveySurvey
QUESTIONNAIRE QUESTIONNAIRE DEVELOPMENTDEVELOPMENT
Factors That Impact Final Factors That Impact Final ContentContent
1.1. Amount of time the medical staff can afford to Amount of time the medical staff can afford to make available in an A&E settingmake available in an A&E setting
– Quality, completenessQuality, completeness– Treatment Vs PreventionTreatment Vs Prevention
2.2. SIDUC RulesSIDUC Rules3.3. Possibility of including other topics due to cost Possibility of including other topics due to cost
sharing with other agenciessharing with other agencies4.4. Ability of respondents to complete the Ability of respondents to complete the
questionnairequestionnaire– Complexity should not exceed respondents’ Complexity should not exceed respondents’
capabilitiescapabilities
Levels of PriorityLevels of Priority
Highly Highly RecommendedRecommended
Necessary to achieve Necessary to achieve objectives. Questions likely objectives. Questions likely to be important to almost to be important to almost any epidemiology study of any epidemiology study of substance usesubstance use
RecommendedRecommended Those that should be given Those that should be given very serious consideration very serious consideration Risk and Protective FactorsRisk and Protective Factors
–Other sources– Injury Other sources– Injury Surv.Surv.
Questions from Questions from StakeholdersStakeholders
OptionalOptional May not measure concepts May not measure concepts of vital interest in every of vital interest in every studystudy
Sequence and Priority of Sequence and Priority of Elements In The Elements In The QuestionnaireQuestionnaire
IntroductionIntroduction Background and demographic Background and demographic
characteristicscharacteristics– Age, Gender, Employment, OccupationAge, Gender, Employment, Occupation
Use of alcoholUse of alcohol Use of illicit substancesUse of illicit substances Use of controlled substancesUse of controlled substances
List of DrugsList of Drugs
AlcoholAlcohol Cocaine HCl.Cocaine HCl. Coca PasteCoca Paste
MarijuanaMarijuana TranquilizersTranquilizers MethamphetaminMethamphetamineses
Solvents andSolvents and
InhalantsInhalantsSedativesSedatives Flunitrazepam Flunitrazepam
(Rohypnol, etc)(Rohypnol, etc)
HallucinogensHallucinogens AmphetaminesAmphetamines Crack CocaineCrack Cocaine
HeroineHeroine AnticholinergicAnticholinergicss
EcstasyEcstasy
Opium/Opium/MorphineMorphine
Anti-Anti-depressantsdepressants
OtherOther
Defining Drugs For The Defining Drugs For The RespondentRespondent
Names and descriptions must be reviewed to see if they Names and descriptions must be reviewed to see if they are are appropriate in the cultural setting in which they will be askedappropriate in the cultural setting in which they will be asked..
The main point is that the names used accurately The main point is that the names used accurately communicate to respondents which substance(s) communicate to respondents which substance(s) shouldshould be be included in what they report, and which substances included in what they report, and which substances should should notnot, and under what circumstances., and under what circumstances.
To determine a list of appropriate slang or street names for To determine a list of appropriate slang or street names for various drugs, the investigators may want to speak with various drugs, the investigators may want to speak with treatment professionals and known drug users in the age treatment professionals and known drug users in the age group under study.group under study.
If a longer question stem is required to get the definition If a longer question stem is required to get the definition clarified for your respondents, then that may be justified. (See clarified for your respondents, then that may be justified. (See SIDUC Manual)SIDUC Manual)
For legally prescribed drugs, it is important that the For legally prescribed drugs, it is important that the respondents understand what occasions of use they should respondents understand what occasions of use they should and should not report in answering the questions.and should not report in answering the questions.
Definition of DrugsDefinition of Drugs
Formal NameFormal Name Street NameStreet Name
CocaineCocaine Coke, crack, white Coke, crack, white lady, blow, etc.lady, blow, etc.
MarijuanaMarijuana Weed, herb, grass, Weed, herb, grass, refer, pot, ganja, etc.refer, pot, ganja, etc.
Example of A Stem Example of A Stem ChangeChange
Concern: General question Vs Concern: General question Vs specific questions that will ensure specific questions that will ensure that all questions are answered by that all questions are answered by all respondentsall respondents
Original (OAS)Original (OAS) ChangeChangeHave you taken any Have you taken any drugs within the 6 drugs within the 6 hours preceding your hours preceding your injuryinjury
Have you taken any of Have you taken any of the following within the the following within the 6 hours preceding your 6 hours preceding your injuryinjury
–AlcoholAlcohol–Marajuana etcMarajuana etc
Layout In The Event of Lab Layout In The Event of Lab ConfirmationConfirmation
Blind Vs Not BlindBlind Vs Not Blind If Not, questions and Lab results on same If Not, questions and Lab results on same
form form – No means to connect necessaryNo means to connect necessary
IF blind, questions and lab results are IF blind, questions and lab results are separatedseparated– Unique identifiers are needed to re-uniteUnique identifiers are needed to re-unite
NB: identifiers can be added after A&E but NB: identifiers can be added after A&E but before transfer to labbefore transfer to lab
Other Useful SuggestionsOther Useful Suggestions Skip patternsSkip patterns are more difficult for are more difficult for
respondents to follow correctly than a simple, respondents to follow correctly than a simple, uninterrupted series of questions; thereforeuninterrupted series of questions; therefore, , it it is advised that they be minimized.is advised that they be minimized.
Pre-testPre-test by getting a limited number of “like” by getting a limited number of “like” respondents to complete the questionnaire.respondents to complete the questionnaire.– Determine average time to complete.Determine average time to complete.– individually interview them about whether the individually interview them about whether the
instructions were clear and whether there were any instructions were clear and whether there were any questions or answers that they had difficulty questions or answers that they had difficulty understanding or using.understanding or using.
– Determine if they understood each class of drugDetermine if they understood each class of drug
Other Useful Suggestions Other Useful Suggestions Cont’dCont’d
Pilot testing to:Pilot testing to:– see how the actual administration see how the actual administration
procedures in the ER will go,procedures in the ER will go,– see how long it takes the patients to see how long it takes the patients to
answer the questions andanswer the questions and– identify remaining problems in the identify remaining problems in the
content and clarity of the questionnaire. content and clarity of the questionnaire.
Emergency Room Emergency Room SurveySurvey
Data Collection ProcedureData Collection Procedure
Decisions Prior to CollectionDecisions Prior to Collection
WHATWHAT What data will be collected What data will be collected WHOWHOWho will be responsible for data Who will be responsible for data
collectioncollection WHENWHEN When to collect data to ensure When to collect data to ensure
a “normal” samplea “normal” sample WHEREWHERE What hospitals are to be What hospitals are to be
includedincluded HOWHOWHow to conduct all aspects of How to conduct all aspects of
survey administrationsurvey administration
What data will be collectedWhat data will be collected
QuestionnairesQuestionnaires– SIDUCSIDUC– BahamasBahamas
Lab confirmation dataLab confirmation data Site reports on:Site reports on:
– # refusals# refusals– # ineligibles# ineligibles
Who will be responsible for Who will be responsible for data collectiondata collection
Conducting the interview of each section of the Conducting the interview of each section of the questionnairequestionnaire– How to select survey leadersHow to select survey leaders
A&E Data FlowA&E Data Flow– Lab sample flowLab sample flow– ““routinely” collected informationroutinely” collected information
Ensuring that a continuous supply of Ensuring that a continuous supply of questionnaires are availablequestionnaires are available
Ensuring a continuous supply of lab sampling Ensuring a continuous supply of lab sampling equipment are available (?)equipment are available (?)
Ensuring that completed forms and lab samples Ensuring that completed forms and lab samples are routed properlyare routed properly
Selection of Survey LeaderSelection of Survey LeaderER StaffER Staff
ProsPros::– already in the already in the ERER– know the know the ERER and are and are
familiar with familiar with ERER routines routines– Patients may feel more Patients may feel more
comfortable with MD or comfortable with MD or NurseNurse
– least expensive least expensive ConsCons::
– Competing interestsCompeting interests – ER Services Vs survey ER Services Vs survey
completioncompletion
Research AssistantResearch Assistant ProsPros::
– More consistency More consistency across hospitalsacross hospitals
ConsCons– More expensiveMore expensive– doubts regarding the doubts regarding the
ability to ability to convince patients convince patients to participateto participate
E.g., Social Workers in E.g., Social Workers in HaitiHaiti
When to collect dataWhen to collect data
SIDUC - SampleSIDUC - Sample
Vs Vs
Institutionalized – routinely collected Institutionalized – routinely collected as part of diagnosisas part of diagnosis
Timing of Data CollectionTiming of Data Collection IIt is important to choose a period which should not be preceded t is important to choose a period which should not be preceded
by any holiday, ensuring that the by any holiday, ensuring that the patientspatients refer to a “normal” refer to a “normal” week or month when answering the questionnaireweek or month when answering the questionnaire
If more than one If more than one hospitalhospital is included, it is a must to administer is included, it is a must to administer the data collection at the same the data collection at the same timetime in all in all hospitals hospitals
In the event of inter-country comparisons (SIDUC), In the event of inter-country comparisons (SIDUC), the time of the time of data collection must be as similar as possibledata collection must be as similar as possible since the use of since the use of alcohol and other drugs alcohol and other drugs vary at certain identifiable periods.vary at certain identifiable periods.
e.g. e.g. December (Christmas) ↑December (Christmas) ↑
January (Recuperation) ↓January (Recuperation) ↓
Holidays ↑Holidays ↑
Lent ↓Lent ↓
Where will data collection take Where will data collection take placeplace
Geographic locationGeographic location– IslandsIslands
HospitalsHospitals– PublicPublic
Princess Margaret (Nassau)Princess Margaret (Nassau) Rand Memorial (Freeport)Rand Memorial (Freeport)
– PrivatePrivate Doctors (Nassau)Doctors (Nassau)
Activities During Activities During AdministrationAdministration
IInstructions to the Surveynstructions to the Survey Data Data Collection StaffCollection Staff
Instructions to the patientsInstructions to the patients SStress anonymity and confidentialitytress anonymity and confidentiality
Who Should Complete Who Should Complete SurveySurvey
1.1. Inclusion CriteriaInclusion Criteria
2.2. Exclusion CriteriaExclusion Criteria• Persons < 12 years of agePersons < 12 years of age• Gynae patientsGynae patients• Language barriers (Non-English speaking)Language barriers (Non-English speaking)• Unconscious patients (?)Unconscious patients (?)
• If hospitalized and no identifiers to follow-upIf hospitalized and no identifiers to follow-up
Patients who RefusePatients who Refuse
Concern is dConcern is due to the possible ue to the possible association between refusal and drug association between refusal and drug use:use:
It is important that the number of refusals It is important that the number of refusals be documentedbe documented
Could compare % refusals from low and Could compare % refusals from low and high prevalence areashigh prevalence areas
It is important that It is important that refusalsrefusals are treated in are treated in the same way in all participating the same way in all participating hospitals hospitals and countries.and countries.
Survey Administration:Survey Administration:Instructions to PatientsInstructions to Patients
Survey Instructions can Survey Instructions can be written on the front be written on the front pagepage of the questionnaire and should include of the questionnaire and should include information information on:on:
– The The purpose of the study (If not incorporated into the purpose of the study (If not incorporated into the system) system)
– TThe selection of he selection of patients (All selected during study period)patients (All selected during study period)– TThe study is anonymous andhe study is anonymous and/or /or confidentialconfidential– Steps to ensure anonymitySteps to ensure anonymity– Participation is important but if not willing, that is their Participation is important but if not willing, that is their
choicechoice
To Ensure To Ensure AnonymAnonymityity
Following SIDUC MethodFollowing SIDUC Method QuestionnairesQuestionnaires should not contain any should not contain any
identifiersidentifiers– NameName– hospital record #, etc.hospital record #, etc.
Survey Administration:Survey Administration:ReportReport
AA report should be completed (periodically) by report should be completed (periodically) by the survey leader the survey leader – Total number of refusalsTotal number of refusals– If the study only targets a specific group, the If the study only targets a specific group, the
number of those excludednumber of those excluded
Checklist Checklist For Data For Data CollectionCollection
Chose a survey leader trusted by the Chose a survey leader trusted by the patientspatients Instructions to the survey leaderInstructions to the survey leader Describe how to treat Describe how to treat patientspatients not belonging to the target not belonging to the target
populationpopulation Describe how to treat Describe how to treat refusalsrefusals Careful planning of the contacts with selected Careful planning of the contacts with selected institutionsinstitutions Safe transportation of materialSafe transportation of material (lab component) (lab component) Detailed planning of the survey administration, including Detailed planning of the survey administration, including
– stress anonymity and confidentialitystress anonymity and confidentiality– instructions to the survey leaderinstructions to the survey leader– instructions to the instructions to the patientspatients– Shift Shift reportreport
Overview of Overview of Methodological Methodological
Issues:Issues:
Interpretation Of DataInterpretation Of Data
Overview of SessionOverview of Session
Representativeness – Representativeness – the extent to which the extent to which a sample mirrors the population of interesta sample mirrors the population of interest
Reliability – Reliability – the extent to which repeated the extent to which repeated measurements used under the same conditions measurements used under the same conditions produce the same resultproduce the same result
Validity – Validity – the extent to which answers are the extent to which answers are accurate representations of the underlying reality accurate representations of the underlying reality that they are intended to measurethat they are intended to measure
Factors that Influence Factors that Influence RepresentativenessRepresentativeness
The sampling methodThe sampling method The size of the sampleThe size of the sample Response rate: The number of Response rate: The number of
eligible non-gynae emergency room eligible non-gynae emergency room patients that agree to participate in patients that agree to participate in the surveythe survey
The Method of Sampling:The Method of Sampling: Target Population Target Population
The target population in The target population in the Emergency Room the Emergency Room survey is, by definition, survey is, by definition, persons persons who who chose to chose to come to the emergency roomcome to the emergency room and excluding and excluding other other individualsindividuals who, for whatever reason, chose not who, for whatever reason, chose not to attend.to attend.
– Beware that all injured persons do not come in due to a Beware that all injured persons do not come in due to a number of reasons (choice, inaccessibility, etc.)number of reasons (choice, inaccessibility, etc.)
– Less-severely injured persons Less-severely injured persons who who do not come in may be the do not come in may be the result of reasons related to result of reasons related to substancesubstance use use
– Large Large differences in differences in emergency roomemergency room attendance (access, attendance (access, availability) between countries may make it difficult/impossible availability) between countries may make it difficult/impossible to make meaningful international comparisons to make meaningful international comparisons
Method of Achieving Method of Achieving RepresentativenessRepresentativeness
Random sampling is fundamental to Random sampling is fundamental to obtaining a sample that is obtaining a sample that is representative of the populationrepresentative of the population
OROR Sample the entire target groupSample the entire target group
Sample Size and Sample Size and RepresentativenessRepresentativeness
the number of sampled the number of sampled patientspatients must must be of sufficient sizebe of sufficient size
considerations considerations must be given to the must be given to the extent of the extent of the analysis of drug habits in analysis of drug habits in different subgroupsdifferent subgroups– always important to sample enough always important to sample enough patientspatients to be to be
able to analyse data separately for able to analyse data separately for males and males and femalesfemales
Response rate must be sufficiently high Response rate must be sufficiently high to enable representative data to be to enable representative data to be obtainedobtained
Categories of Response Categories of Response RatesRates
Institutional CooperationInstitutional Cooperation– Public and Private HospitalsPublic and Private Hospitals
Patient Cooperation For InterviewPatient Cooperation For Interview Proportion Participating in Lab Proportion Participating in Lab
AnalysisAnalysis
Issues Impacting Issues Impacting Institutional CooperationInstitutional Cooperation
AAn increased number of n increased number of proposedproposed surveys has made surveys has made hospitalshospitals in some in some countries somewhat reluctant to countries somewhat reluctant to allow allow patients topatients to participate participate directly directly in such in such surveyssurveys– highlightshighlights the necessity of approaching the necessity of approaching
institutions with emergency rooms formally institutions with emergency rooms formally and and in a way that makes them feel that in a way that makes them feel that they are part of an important studythey are part of an important study
Issues Impacting Patient Issues Impacting Patient ParticipationParticipation
PParticipation should always be voluntaryarticipation should always be voluntary Suspicions regarding “True” objectivesSuspicions regarding “True” objectives has made has made
patientspatients somewhat reluctant to take time to somewhat reluctant to take time to participate in such participate in such drug drug surveyssurveys– highlightshighlights the necessity of approaching the necessity of approaching patientspatients in a way in a way
that makes them feel that they are part of an important that makes them feel that they are part of an important studystudy
AAll questionnaires ll questionnaires shouldshould be treated confidential be treated confidentiallyly– TThe use of questionnaires without names or other kinds he use of questionnaires without names or other kinds
of identificationof identification– GGuaranties of confidential treatment of questionnaires uaranties of confidential treatment of questionnaires
and dataand data– PPromises not to report data for individual romises not to report data for individual patientspatients
Decisions Regarding Non-Decisions Regarding Non-ParticipationParticipation
If If hospitals with a large percentage of hospitals with a large percentage of overall ER visits overall ER visits refuse to participate it is refuse to participate it is essential to do a careful analysis of the essential to do a careful analysis of the reasonsreasons
IfIf systematic errors systematic errors isis suspected, suspected, interpretation might be difficult and interpretation might be difficult and international comparisons may be international comparisons may be jeopardised.jeopardised.E.g. E.g. – refusing refusing patientspatients come from areas where drug come from areas where drug and/or and/or
alcoholalcohol consumption is known to be high consumption is known to be high
ReliabilityReliability
The extent to which repeated The extent to which repeated measurements used under the same measurements used under the same conditions produce the same resultconditions produce the same result
OrOr
Whether an indicator is consistent across Whether an indicator is consistent across time and observerstime and observers
Methods of Measuring Methods of Measuring Survey ReliabilitySurvey Reliability
To conduct repeated studies using To conduct repeated studies using same methodologysame methodology
BBy using data from different y using data from different questions within a questionnairequestions within a questionnaire
Laboratory ConfirmationLaboratory Confirmation
Factors Influencing Study ReliabilityFactors Influencing Study Reliability
CompletenessCompleteness - - proportion of total proportion of total information information knownknown– All respondents answer questionsAll respondents answer questions– All targeted respondents take surveyAll targeted respondents take survey
Sources of ErrorSources of Error– InstrumentInstrument– PersonPerson
MD, Nurse, Med. Rec ClerkMD, Nurse, Med. Rec Clerk Study Subject or Respondent Study Subject or Respondent
– May occur at May occur at subject selectionsubject selection data collection (Subjective)data collection (Subjective)
– Kappa StatisticKappa Statistic analysis and interpretationanalysis and interpretation
Sources of Error: Sources of Error: Instrument and ProceduresInstrument and Procedures
Poor selection of questions and variables Poor selection of questions and variables (SIDUC)(SIDUC)
Ambiguous questions (additional questions)Ambiguous questions (additional questions)
Non-response Non-response
Poor selection of study subjects (friendliest)Poor selection of study subjects (friendliest)
Failure to validate measurements, recordingsFailure to validate measurements, recordings
– lablab
Sources of Error:Sources of Error:PersonPerson
InterviewerInterviewer Inter-observer variation Inter-observer variation
– the way medical staff explain and/or ask the way medical staff explain and/or ask questionsquestions
– Opinions on extent drug/alcohol use contributed Opinions on extent drug/alcohol use contributed to injuryto injury
SubjectSubject Inter-subject variationInter-subject variation
– Different Interpretation of questions (what drugs Different Interpretation of questions (what drugs were thought of without prompting)were thought of without prompting)
– willingness, survey environmentwillingness, survey environment
To Achieve Reliable DataTo Achieve Reliable Data
Thorough knowledge of study populationThorough knowledge of study population Unbiased selection of study subjectsUnbiased selection of study subjects Standardized, calibrated and consistent Standardized, calibrated and consistent
instrument(s)instrument(s) Clear, unambiguous questionsClear, unambiguous questions Minimal personal discomfort or Minimal personal discomfort or
discomfiture discomfiture Interviewers, recorders thoroughly and Interviewers, recorders thoroughly and
equally well trained equally well trained Need to validate Need to validate
ValidityValidity
The extent to which answers are accurate The extent to which answers are accurate representations of the underlying reality representations of the underlying reality that they are intended to measure.that they are intended to measure.
In the context of emergency room surveys, In the context of emergency room surveys, the degree to which the questionnaire the degree to which the questionnaire measures the aspects of patients’ drug measures the aspects of patients’ drug consumption that it was intended to consumption that it was intended to measure.measure.
Strategies to Enhance Strategies to Enhance ValidityValidity
Studies must guarantee anonymity and Studies must guarantee anonymity and confidentialityconfidentiality– One is to use a data collection leader trusted by the One is to use a data collection leader trusted by the
patientspatients– Stress Stress anonymity anonymity during the during the introduction introduction – no names or other identification marks should be on the no names or other identification marks should be on the
questionnaire (SIDUC)questionnaire (SIDUC) The patients must also have enough time to The patients must also have enough time to
answer the questionnaireanswer the questionnaire– important that the questionnaire is not too longimportant that the questionnaire is not too long
TThey must understand the questionshey must understand the questions TThey must be willing to answer the questions hey must be willing to answer the questions
honestlyhonestly
Strategies to Assess ValidityStrategies to Assess Validity
PatientsPatients’’ willingness to co-operate willingness to co-operate Patients’Patients’ comprehension comprehension MMissing data ratesissing data rates LLogical consistencyogical consistency RReported willingness to answer honestlyeported willingness to answer honestly TThe cultural context in which a survey is he cultural context in which a survey is
conductedconducted
Strategies to Assess Strategies to Assess Validity:Validity:
C Cultural ultural CContext ontext QQuestions uestions must be must be culturally or locally appropriateculturally or locally appropriate– Use Use the appropriate “street-names” or “nicknames” used the appropriate “street-names” or “nicknames” used
for different drugsfor different drugs WWillingness to admit drug use may be influenced by illingness to admit drug use may be influenced by
the attitudes towards drugs in a given societythe attitudes towards drugs in a given society– perceived risk of substance useperceived risk of substance use, , disapproval of different disapproval of different
kinds of substance usekinds of substance use and and the availability of different the availability of different drugs differ between countries.drugs differ between countries.
– Low availability + Negative attitude = Low availability + Negative attitude = less willing less willing to admitto admit– High availability + Positive attitudes = more willing to High availability + Positive attitudes = more willing to
admitadmit Survey TraditionSurvey Tradition
– PersonPersons in countries where surveys are less common s in countries where surveys are less common may may feel less comfortable answering questions about sensitive feel less comfortable answering questions about sensitive behaviours. behaviours.
Checklist For Survey Checklist For Survey MethodologyMethodology
RepresentativenessRepresentativeness– Define the target populationDefine the target population– Assess the importance of non-students in the same age Assess the importance of non-students in the same age
groups as the target populationgroups as the target population– Decide a proper time for the data collection (if Decide a proper time for the data collection (if
international comparisons are planned)international comparisons are planned)– Assess the importance of non-participating Assess the importance of non-participating
schools/classesschools/classes– Assess the importance of non-participating studentsAssess the importance of non-participating students
ReliabilityReliability– Assess reliability (whenever possible by using data in Assess reliability (whenever possible by using data in
the questionnaire)the questionnaire)
Checklist For Survey Checklist For Survey Methodology Cont’dMethodology Cont’d
ValidityValidity– Anonymous and confidential data collectionAnonymous and confidential data collection– Measure and reportMeasure and report– Number of eliminated questionnairesNumber of eliminated questionnaires– Survey leader information (from the classroom Survey leader information (from the classroom
report)report)– Time to answer the questionnaireTime to answer the questionnaire– Proportion of unanswered questionsProportion of unanswered questions– Logical consistencyLogical consistency– Possible use of a “willingness question”Possible use of a “willingness question”– Possible reported use of a dummy drugPossible reported use of a dummy drug– Construct validityConstruct validity
The EndThe End
ReallyReally
The EndThe End