session 3: assessment & evaluation lisa jacques-carroll, msw ncird, cdc
Post on 12-Jan-2016
226 Views
Preview:
TRANSCRIPT
Session 3: Session 3: Assessment & EvaluationAssessment & Evaluation
Lisa Jacques-Carroll, MSWLisa Jacques-Carroll, MSW
NCIRD, CDCNCIRD, CDC
Learning ObjectivesLearning Objectives
Describe key components of evaluation of Describe key components of evaluation of a perinatal hepatitis B prevention program a perinatal hepatitis B prevention program describe different methods for assessing and describe different methods for assessing and
evaluating a perinatal hepatitis B prevention evaluating a perinatal hepatitis B prevention programprogram
identify several tools that are availableidentify several tools that are available
Assessment & EvaluationAssessment & Evaluation
Program evaluation vs. program Program evaluation vs. program monitoring or assessmentmonitoring or assessment
The focus of this session is on assessing The focus of this session is on assessing performance measures, which is a performance measures, which is a component of program evaluation component of program evaluation
For more information on program For more information on program evaluation see: evaluation see:
www.cdc.gov/vaccines/programs/progeval/www.cdc.gov/vaccines/programs/progeval/
Activities Required by CDCActivities Required by CDC
Requirements for assessment of the Requirements for assessment of the perinatal hepatitis B include:perinatal hepatitis B include: frequency of assessmentfrequency of assessment performance measuresperformance measures target levels target levels
For more information: For more information: http://www.cdc.gov/vaccines/vac-gen/policies/ http://www.cdc.gov/vaccines/vac-gen/policies/ ipom/downloads/chp-05-perinatal-hepb.pdfipom/downloads/chp-05-perinatal-hepb.pdf
OverviewOverview
Laboratory reporting auditsLaboratory reporting audits Hospital record reviewsHospital record reviews Hospital policy surveysHospital policy surveys Expected births to HBsAg+ womenExpected births to HBsAg+ women Assessing case management Assessing case management Monitoring birth doseMonitoring birth dose
Laboratory Reporting AuditsLaboratory Reporting Audits
Use to assess:Use to assess: completeness of reporting of HBsAg+ resultscompleteness of reporting of HBsAg+ results timeliness of reportingtimeliness of reporting
Include delivery hospital labsInclude delivery hospital labs Collaborate with communicable disease Collaborate with communicable disease
and state laboratory staff to help plan and and state laboratory staff to help plan and execute reviewsexecute reviews
Prioritize Labs to EvaluatePrioritize Labs to Evaluate
Priority labs:Priority labs: labs serving high-morbidity areas or populationslabs serving high-morbidity areas or populations labs reporting large volume of hepatitis serologylabs reporting large volume of hepatitis serology labs serving prenatal clinicslabs serving prenatal clinics delivery hospital labsdelivery hospital labs
How often?How often? ideally, once/year for priority labsideally, once/year for priority labs every 2–3 years for othersevery 2–3 years for others minimally every 5 yearsminimally every 5 years
Laboratory Audit ToolLaboratory Audit Tool
A CDC tool available at:A CDC tool available at:
http://www.cdc.gov/ncidod/diseases/hepatitis/resource/LabReportingAssessment.doc
Completeness of Lab ReportingCompleteness of Lab Reporting
Pick a time periodPick a time period (at least 3 months) (at least 3 months) Calculate:Calculate:
Total # HBsAg+ test results reported
Total # specimens that were HBsAg+
=Proportion of HBsAg+
tests reported, or completeness of
reporting
Timeliness of Lab ReportingTimeliness of Lab Reporting
Date of HBsAg Date of HBsAg serologyserology
Date HBsAg+ Date HBsAg+ report rec’d by report rec’d by
health depthealth dept
# Days between # Days between serology and serology and
reportreport
8-18-20078-18-2007 8-23-20078-23-2007 55
8-23-20078-23-2007 8-29-20078-29-2007 66
9-7-20079-7-2007 9-12-20079-12-2007 55
11-18-200711-18-2007 11-27-200711-27-2007 99
SUMSUM 2525
Timeliness of Lab ReportingTimeliness of Lab Reporting - cont’d - cont’d
=Mean reporting time for
the laboratory, or timeliness of reporting
Time period for reporting is typically set by state statute
Sum of column “# days Sum of column “# days between serology and between serology and
report”report”
Total # of serologies Total # of serologies evaluatedevaluated
25
4= 6.25 dayse.g. from the
previous slide
Tips for Working with LaboratoriesTips for Working with Laboratories
Contact other areas of health department Contact other areas of health department to see if activity is already being doneto see if activity is already being done
See if an existing lab report can be used See if an existing lab report can be used for the audit—ensure it is source datafor the audit—ensure it is source data
When making contact with labs, include:When making contact with labs, include: lab directorlab director techniciantechnician IT staffIT staff
Hospital Record ReviewsHospital Record Reviews Use to assess:Use to assess:
maternal HBsAg screeningmaternal HBsAg screening birth dose administrationbirth dose administration post-exposure prophylaxis (PEP) for infants post-exposure prophylaxis (PEP) for infants
born to HBsAg+ and unknown-status womenborn to HBsAg+ and unknown-status women Audit paired Audit paired maternalmaternal and and infantinfant records records Partner with health dept programs to collect Partner with health dept programs to collect
other perinatal data other perinatal data rubella antibodyrubella antibody • HIV• HIV group B strepgroup B strep • • early hearingearly hearing syphilissyphilis
Determining Sample SizeDetermining Sample Size
Tool available in Tool available in Guide to Life (Table 2.1)Guide to Life (Table 2.1)
http://www.cdc.gov/ncidod/diseases/hepatitis/resource/perinatalhepB.htm
Using the Using the Guide to LifeGuide to Life table table
Determine hospital’s annual number of Determine hospital’s annual number of deliveries deliveries
Decide where it falls in the “Birth Cohort Decide where it falls in the “Birth Cohort Size” categorySize” category
Estimate HBsAg screening and hepatitis B Estimate HBsAg screening and hepatitis B birth dose coverage for the hospitalbirth dose coverage for the hospital
Use the Use the lowerlower coverage to determine the coverage to determine the sample sizesample size
Sample Size ExampleSample Size Example
Hospital X had 648 deliveries last yearHospital X had 648 deliveries last year
The expected HBsAg screening rate is 95%The expected HBsAg screening rate is 95%
The state National Immunization Survey The state National Immunization Survey (NIS) birth dose rate is 65%(NIS) birth dose rate is 65%
Using the 65% coverage level, the sample Using the 65% coverage level, the sample size for Hospital X would be 111 recordssize for Hospital X would be 111 records
Hospital Data to CollectHospital Data to Collect
Delivery date/timeDelivery date/time Mother’s HBsAg test date/result/type in:Mother’s HBsAg test date/result/type in:
maternal recordmaternal record infant recordinfant record
Infant hepatitis B vaccination (yes/no)Infant hepatitis B vaccination (yes/no) Date/time of vaccinationDate/time of vaccination Infant HBIG administration (yes/no)Infant HBIG administration (yes/no)
Hospital Data to CollectHospital Data to Collect - cont’d- cont’d
Date/time of HBIGDate/time of HBIG Other variables as appropriateOther variables as appropriate
Hospital Policy SurveysHospital Policy Surveys
Use to assess whether hospitals have Use to assess whether hospitals have written policies/standing orders in place:written policies/standing orders in place: universal birth dose of hep Buniversal birth dose of hep B HBsAg screening of pregnant womenHBsAg screening of pregnant women PEP to infants born to HBsAg+ and unknown-PEP to infants born to HBsAg+ and unknown-
status womenstatus women documentation of maternal HBsAg status, documentation of maternal HBsAg status,
birth dose, and HBIG (if appropriate) in birth dose, and HBIG (if appropriate) in medical recordsmedical records
Conducting Policy SurveysConducting Policy Surveys
Mail surveyMail survey Telephone surveyTelephone survey In-person survey In-person survey
during hospitalduring hospital
medical record medical record
reviewsreviews
Keep in mind, policies do not always match practice, Keep in mind, policies do not always match practice,
so medical record reviews are essentialso medical record reviews are essential
Expected Births to HBsAg+ WomenExpected Births to HBsAg+ Women
CDC expected birth CDC expected birth estimates are a benchmark estimates are a benchmark for programsfor programs
CDC provides estimates for CDC provides estimates for states and some cities states and some cities
Estimates are calculated Estimates are calculated using:using: birth databirth data prevalence of chronic prevalence of chronic HBV infection by HBV infection by race/ethnicityrace/ethnicity
Expected BirthsExpected Births - cont’d- cont’d
Nationally, about 24,000 births expected Nationally, about 24,000 births expected annually, only 50% identified in 2005annually, only 50% identified in 2005
CDC encourages programs to improve CDC encourages programs to improve estimates of births to HBsAg+ womenestimates of births to HBsAg+ women
More information in presentation at:More information in presentation at:
http://www.cdc.gov/ncidod/diseases/hepatitis/http://www.cdc.gov/ncidod/diseases/hepatitis/resource/presentations/2007-04PeriGrantees/PDFs/resource/presentations/2007-04PeriGrantees/PDFs/May1-p0130-1LJ_2007PeriCoordMtg-May1-p0130-1LJ_2007PeriCoordMtg-
ExpectedBirths.pdfExpectedBirths.pdf
Assessing Case ManagementAssessing Case Management
Number of infants born to HBsAg+ womenNumber of infants born to HBsAg+ women Proportion of infants receiving Proportion of infants receiving
hep B vaccine & HBIG within 12 hours of birthhep B vaccine & HBIG within 12 hours of birth on-time completion of hep B vaccine serieson-time completion of hep B vaccine series post-vaccination testingpost-vaccination testing
Reasons for cases lost to follow-upReasons for cases lost to follow-up Proportion of screened and vaccinated Proportion of screened and vaccinated
household and sexual contactshousehold and sexual contacts
Assessing Case Mgmt - cont’dAssessing Case Mgmt - cont’d
Monitor case management outcomes:Monitor case management outcomes: compare program-wide rates (entire state or compare program-wide rates (entire state or
city) to national rates (CDC peritable)city) to national rates (CDC peritable) regionallyregionally locally (county level)locally (county level) by case workerby case worker quarterly or monthlyquarterly or monthly over time (trends)over time (trends)
Monitoring Birth DoseMonitoring Birth Dose
National Immunization Survey (NIS) data National Immunization Survey (NIS) data measures hepatitis B birth dose ratesmeasures hepatitis B birth dose rates
Hospital medical record reviewsHospital medical record reviews Immunization Information Systems (IIS)Immunization Information Systems (IIS)
hep B vaccine birth certificate data hep B vaccine birth certificate data automatically uploaded into IIS automatically uploaded into IIS
calculate birth dose coverage for cities, calculate birth dose coverage for cities, regions, and individual hospitalsregions, and individual hospitals
Continuous MonitoringContinuous Monitoring
Some aspects of the Some aspects of the program require program require continuous monitoringcontinuous monitoring case management case management
outcomes outcomes laboratory reportinglaboratory reporting
Plan ahead for Plan ahead for assessments your assessments your program will conduct program will conduct over the next year/sover the next year/s
Using Your ResultsUsing Your Results
Determine future direction and focus of Determine future direction and focus of program effortsprogram efforts
Provide feedback to:Provide feedback to: hospitalshospitals laboratorieslaboratories providersproviders health departmentshealth departments case workerscase workers
top related