the big picture the role of evaluation in public health luann d’ambrosio, med
TRANSCRIPT
![Page 1: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/1.jpg)
The Big PictureThe Role of Evaluation in
Public Health
Luann D’Ambrosio, MEd
![Page 2: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/2.jpg)
Activity
Think of your favorite quote or sage advice that helps guide you through your professional work.
![Page 3: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/3.jpg)
Wise Advice
I hear and I forget.
I see and I believe.
I do and I understand.
—Confucius (551–479 BC)
![Page 4: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/4.jpg)
Standards
Utility
Accuracy
Propriety
Feasibility
CDC Evaluation Framework
![Page 5: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/5.jpg)
• Describe the relationship between performance management, program planning and evaluation
• Describe the steps of program planning and evaluation process
• Identify key stakeholders in an evaluation
• Develop the components and elements of a program logic model
• Create a basic evaluation plan including data collection and analyzing methods
• List three ways to use evaluation data to draw conclusions about a program
• Define ROI, cost benefit ratios and social return on investment.
Objectives
![Page 6: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/6.jpg)
Accreditation
![Page 7: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/7.jpg)
CDC GrantsCDC awards grants for public health performance management systems
•$42.5 million allocated for 94 projects in state, local, tribal, and territories health departments
•Distributed in 49 states, 16 tribes and territories and nine local jurisdictions
•Five-year cooperative agreement awards are being funded by the health care reform law
![Page 8: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/8.jpg)
Benefits of Accreditation
Accountability & credibility
Tool for improvement
Greater collaboration
Recognition & validationBetter
understanding of public health
Team building
Highlights HD strengths
Accreditation+
![Page 9: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/9.jpg)
Phase 1
EvaluationMonitoring
Implementation
BudgetsAction Plans
Strategies
Strategic Issues Goals & Objectives
Mission ValuesVision
SWOCAnalysis &
Environmental Scan
Mandate Analysis
StakeholderAssessment
Plan the
Plan
Readiness Assessment
Phase 2
Phase 3
Phase 4
Strategic Planning
![Page 10: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/10.jpg)
Performance Management (PM)
A systematic process aimed at helping achieve an
organization’s mission and strategic goals by
improving effectiveness, empowering employees,
and streamlining the decision-making process.
PerformancePerformance BehaviorBehavior ResultsResults= +
![Page 11: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/11.jpg)
They Are Not the Same
Quality assurance Quality improvement
Reactive Proactive
Works on problems after they occur
Works on processes
Regulatory usually by state or federal law
Seeks to improve (culture shift)
Led by management Led by staff
Periodic look-back Continuous
Responds to a mandate or crisis or fixed schedule
Proactively selects a process to improve
Meets a standard (pass/fail) Exceeds expectations
![Page 12: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/12.jpg)
They Are Not the Same
Evaluation Quality improvement
Assess a program at a moment in time
Understand the process that is in place
Static Ongoing
Doesn’t include identification of the source of a problem or potential solutions
Entails finding the root cause of a problem and interventions targeted to address it
Doesn’t measure improvements
Focused on making measurable improvements
Program-focused Customer-focused
A step in the QI process Includes evaluation
![Page 13: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/13.jpg)
Value of Performance Management
• Disciplined approach
• Systemic
• Systematic
• Engaged customers and staff
• Results focused
• Proven method
![Page 14: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/14.jpg)
Culture of QITopic Organization-wide Program/unit
Improvement System focus Specific project focus
QI planning Tied to the strategic plan
Program/unit level
QI goals Strategic plan Individual program/unit level plans
Approaches Baldrige ProgramOrganization QI Council
Lean Six SigmaIndividual QI TeamsRapid Cycle PDCA
![Page 15: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/15.jpg)
Performance Management Frameworks
• Baldrige
• Six Sigma
• Lean
![Page 16: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/16.jpg)
PDSA Cycle
What are we trying to accomplish?What are we trying to accomplish?
How will we know that a change is an improvement?How will we know that a change is an improvement?
What changes can we make that will result in improvement?What changes can we make that will result in improvement?
![Page 17: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/17.jpg)
Memory Jogger
![Page 18: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/18.jpg)
ToolsLogic models and work flow charts
Data and analysis tools
• Customer-supplier relationships
• Client flow, information flow
• Root cause tools: fishbone diagram, Pareto chart
• Force field analysis• Interrelationship digraph
![Page 19: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/19.jpg)
QI PROCESS
to improve policies, programs, & outcomes•Manage changes•Create learning organization
Turning Point Performance Management Collaborative, 2003
PERFORMANCE MANAGEMENT
SYSTEM
PERFORMANCE STANDARDS•Identify relevant standards•Select indicators•Set goals & targets•Communicate expectations
PERFORMANCE MEASUREMENT•Refine indicators & define measures•Develop data systems•Collect data
REPORTING OF PROGRESS•Analyze data•Feed data back to managers, staff, policy makers, constituents•Develop a regular reporting cycle
QI PROCESS
•Use data for decisions
![Page 20: The Big Picture The Role of Evaluation in Public Health Luann D’Ambrosio, MEd](https://reader030.vdocuments.site/reader030/viewer/2022032805/56649ee75503460f94bf7942/html5/thumbnails/20.jpg)
Questions?