model of health and wellbeing evaluation framework · 2018-04-01 · performance management, the...

ModelofHealthandWellbeing

EvaluationFrameworkVersion3.0

PerformanceManagementCommittee

February 2015

2 | P a g e

VersioncontrolinformationDocument version

Author(s) Approved on: Approved by: Summary of changes for this version:

1.0 RDSSs Aug 2014 PMC Initial version

2.0 Arron Service February 2015 PMC PMC Feedback

3.0 Arron Service Final Version PMC Final Edits

TableofContents

Model of Health and Wellbeing Evaluation Framework ............................................................................... 1

Version control information ......................................................................................................................... 2

Table of Contents .......................................................................................................................................... 2

Background ................................................................................................................................................... 3

Purpose & Framework Components ............................................................................................................. 4

The Results Based Logic Model …………………………………………………………………………………………………...…………6

MHWB Domain #1: Team Based, Integrated and Coordinated .................................................................... 7

MHWB Domain #2: Population Needs‐based ............................................................................................. 10

MHWB Domain #3: Community Governed ................................................................................................. 12

MHWB Domain 4: Accountable and Efficient ............................................................................................. 13

MHWB Domain 5: Community Development Approach ............................................................................ 16

MHWB Domain 6: Anti‐oppressive and Culturally Safe .............................................................................. 18

MHWB Domain 7: Determinants of Health ................................................................................................ 19

MHWB Domain 8: Accessible ...................................................................................................................... 22

Glossary ...................................................................................................................................................... 24

References .................................................................................................................................................. 29

3 | P a g e

BackgroundWith evaluation and performance management playing a greater role in health care decision making,

health service organizations need a method to demonstrate the value of their service models to their

stakeholders. In the past, healthcare reforms were not always based on evidence and progress was

often driven by political arguments or the interests of specific professional groups rather than by the

results of sound evaluations (Watson, Broemeling, Reid & Black, 2004). Before we can demonstrate the

value of our services, we need a common approach to describing the services we provide. At the

provincial level, the common conceptual framework we use to describe our services is called the Model

of Health and Wellbeing (MHWB).

Figure 1. Model of Health and Wellbeing

The role of the MHWB is to provide a common conceptual framework from which all services can be

understood. Broadly speaking, the MHWB is based on a shared vision for the future: the best possible

health and wellbeing for everyone. To achieve this, the MHWB revolves around eight domains that

centres agree are critical components of the community governed primary health care, health

promotion and community development programming we offer. Specifically, clients of our centres can

expect that their care will be:

1. Interprofessional, integrated and coordinated

2. Anti‐oppressive and culturally safe

3. Accountable and efficient

4. Grounded in a community development approach

4 | P a g e

5. Community governed

6. Based on the determinants of heath (DOH)

7. Population and needs based

8. Accessible

With agreement and commitment to the common vision and the eight domains of the MHWB, the

sector can move toward enhancing the evaluability of our model by developing an evaluation

framework (the Framework). Although it is challenging to distill the services delivered by more than 85

unique primary health care organizations across the province into, first a single conceptual framework

and then into a single measurement framework, this effort is critical in helping us understand what we

are collectively attempting to achieve as a sector and will guide measurement of our progress along the

way.

Purpose&FrameworkComponentsWith the recent revision of the MHWB, the Performance Management Committee (PMC) commissioned

the revision of the Evaluation Framework (the Framework). The Framework is intended to support

ongoing assessment and evaluation of our programs and services by providing a common starting point

for more focused investigations. In order to conduct program evaluations on specific programs,

investigators should use this sector‐wide evaluation framework as a first step in the development of

more focused and detailed program based conceptual models and evaluation frameworks. The sector‐

wide evaluation framework is designed to be sufficiently generic enough to apply broadly across all

programs and services. As a result, the Framework does not get into sufficient detail around any one

single program for the purposes of evaluation and therefore should not be viewed as a program‐specific

evaluation guide. For programs looking to evaluate their services, additional work will need to be

completed at the program level to first, ensure programming is evaluable and anchored in a common

conceptual framework and second, to identify appropriate indicators that might inform program

outputs and outcomes. The current sector‐wide Framework will support these more focused evaluation

efforts by providing information on our collective outputs and outcomes that we are all working

towards, regardless of which program you support. All program‐specific evaluations should be able to

identify outputs and outcomes from the sector‐wide Framework and use these concepts to drill‐down

further to uncover specific indicators of outputs and outcomes that are meaningful within the context of

their program.

The Framework contains a series of discrete but associated components that can be used to evaluate

the unique aspects of our MHWB. These components include the results based logic model (RBLM),

evaluation questions, indicators (measures) and data sources appropriate to each indicator.

5 | P a g e

TheResults‐BasedLogicModelResults based logic models linearly link resource inputs to activities performed, services delivered and

outcomes achieved. In doing so, they identify the critical areas requiring monitoring (e.g., QI), evaluation

and reporting. Deriving evaluation and performance measures from a common conceptual model of

service (i.e., MHWB) improves the relevance of the (proxy) indicators and ensures good coverage of

domains identified in the model. The results‐based logic model (RBLM) therefore serves an important

function as a bridge between the conceptual (MHWB) and the operational (performance indicators and

data entry manual).

The RBLM was developed using previous logic models developed by the sector, consultation with sector

decision makers, decision support specialists and centre staff with expertise in evaluation and

performance management, the Treasury Board of Canada results‐based management accountability

framework (2010), the results‐based logic model for primary care developed by the Centre for Health

Services and Policy Research at the University of British Columbia (Watson, Broemeling, Reid & Black,

2004) and the Ontario Ministry of Health and Long‐Term Care primary care performance measurement

framework (2013).

TheEvaluationQuestions&IndicatorsA critical component of the Framework is the identification of strategic and overarching questions

arising from the RBLM that are intended to guide evaluation activities and approaches. The following

eight tables will provide potential evaluation questions and indicators for each of the eight domains of

the MHWB. The evaluation questions and indicators were developed based on previous Community

Health Centre (CHC) evaluation frameworks and the primary care evaluation literature. Each question

will be associated with a number of indicators that are described in terms of indicator type (outcome=O

and process=P), source of data, current use (at time of report) and a description of what is success?

Processevaluation/measuresProcess evaluation examines the extent to which program implementation has taken place, the nature

of the people being served and the degree to which the program operates as expected. Indicators of

process are identified with a “P” in the following tables.

Outcome(orImpact)evaluation/measuresEvaluations of outcome can take on several levels of complexity. The most elementary level involves the

assessment of the condition of those who have received the service; For example, are clients healthier?

More challenging evaluations might attempt to demonstrate that receiving program services caused a

positive change for the better (Posavac & Carey, 2010). Indicators of outcome are identified with an “O”

in the following tables.

IndicatorUseEach indicator is identified as being in current use for particular reporting such as M‐SAA, others are

categorized as “possible” if the data is available and others are “future” meaning that there may be

potential to get this data and these provide ideas for continued work on assessing the outcomes of

programs and services.

6 | P a g e

Purpose Statement

Values

Model of Health and Wellbeing Attributes

Inputs

Activities

Outputs

Direct Outcomes

Intermediate Outcomes

Longer term Outcomes

Commitment to health through the lens of determinants, community vitality and belonging, health equity and social justice

Increased community capacity‐building

with empowered clients to address the

determinants of health elements of their

health needs

Reduced risk, incidence,

duration and effects of

acute and episodic

physical, social or

psychological conditions

Increased civic engagement and social capital

Improved level and distribution of population health and wellness

Improved capacity of communities to be involved in decision‐making about

their health

Increased seamless delivery of services,

appropriateness of time, place and

interprofessional team through

integration and coordination

Improved functioning, health, resilience and wellbeing of Individuals,

families and communities

Improved Health Equity across Sectors

Reduced risk, incidence and

effects of chronic diseases

(e.g., diabetes, mental

health & addictions) through

health promotion

Increased access for

people who experience

the greatest barriers

Resources - Financial, Material and Human Community Knowledge Synthesis - Community and client input, Needs assessments, Environmental scans

Client and community driven health care programs, services and initiatives with particular focus on those who face barriers to health

Highest Quality, People and Community Centred Health and Wellbeing

Improved equity in access to CENTRE services by

eliminating barriers and advocating for healthy public

Reduced negative impact of SDOH on health and

wellbeing of clients

How Many? (Volumes)

What services do we deliver? (e.g., primary health care with

interprofessional teams, health promotion and community development)

How do we deliver services? (i.e., 8 MoHWB Attributes)

With Whom? (priority populations ‐ e.g., seniors, homeless, racialized)

Increased community

partnerships

Accessible

Interprofessional,

integrated and

coordinated

Community governed

Based upon the

Determinants of HealthAnti‐oppressive

and Culturally

Safe

Accountable and

Efficient

Community

Development

Approach

Population and

Needs‐based

7 | P a g e

MHWBDomain#1:TeamBased,IntegratedandCoordinatedDefinition: The provision of comprehensive primary healthcare services to clients by multiple healthcare professionals who work collaboratively

to deliver care. The “team” is a collection of individuals who are interdependent in their tasks and share responsibility for outcomes. Team based

care will be integrated and coordinated by ensuring that information flows easily both within the team but also as care is transitioned outside of

the team to other community‐based agencies, secondary (specialists), tertiary (hospitals) and long term care services.

Potential Questions Source Indicator Current Use What is success?

1.1 Is client information shared efficiently within Centres?

Org or Staff Survey CIHI

1.1.1 % of Centres reporting use of EHR to exchange health information with other providers over the past 12 months (P)

Accreditation

All health care providers share access to necessary client information

Org or Staff Survey CIHI

1.1.2 % of Centres who report bi‐directional electronic communication links with other HSOs over the past 12 months (P)

Future

Health professionals are either co‐located or protocols / processes are developed to ensure a high level of integration Improved Client safety

1.2 What factors facilitate health care providers working together to provide comprehensive primary care (scope of practice regulations, funding, training)?

Staff Survey CIHI

1.2.1 % of Centre providers who report that their interprofessional team exhibits the characteristics of a highly effective interprofessional practice (O)

Accreditation

Formalized processes are developed to improve assessment and referral Increased access to teams by clients who are psychosocially complex

BIRT Client Experience

1.2.2 Number of case conferences (P) Possible Improved provider communications Improved client experience with accessing different team members

Practice Profile

1.2.3 Effective management of clients with diabetes (e.g., HA1c, retinal exam, etc.) (O)

Possible Improvement in management of clients with diabetes within the primary healthcare team

1.3 Is there evidence of increased or appropriate use and support for interprofDOHessional teams?

BIRT 1.3.1 Number of clients accessing non‐physician/NP provided services, that is, receiving more holistic care (P)

MSAA (Diabetes)

Physicians and other health professionals better understand and appreciate the complementary role they can play in client care

BIRT 1.3.2 Number of clients that access interprofessional teams, by provider type

MSAA (Diabetes)

Client needs direct access to appropriate within team referrals

8 | P a g e

(P) Org Survey

CIHI 1.3.3 Provider FTE per 100,000 population by type of provider (P)

Future Increased access to interprofessional team providers

1.4 Has the centre established practices that effectively utilize interprofessional teams?

Client Exp. Survey

1.4.1 % of clients reporting that they (P): a) have access to a team b) are being managed by a team c) are satisfied with the team

Future Increased levels of satisfaction in both clients and providers

Health Links

1.4.2 % of complex centre clients with coordinated care plans (P)

Future Greater continuity of care for complex clients More seamless transitions for clients

Org Survey CIHI

1.4.3 % of centres reporting that they are making efforts to ensure providers are working to full scope of practice (O) (as appropriate)

Future Increased number of providers working to full scope of practice

1.5 Are team members working to full scope of practice (as per training and regulation)?

Provider Survey

1.5.1 % of health care providers reporting that they are satisfied with the use of their skills by provider type (O)

Future Increased retention of staff

1.6. Is quality of work‐life acceptable to staff and providers?

Provider / staff Survey

1.6.1 % of providers who report they were satisfied with the overall quality of work life over the past 12 months (O)

Future Increased staff work‐life satisfaction

Provider Survey

1.6.2 % of providers who were satisfied with the duration of client visits over the past 12 months by provider (O)

Future Improved provider satisfaction

1.7 Do clients experience continuity of care (e.g., coordinated and integrated)?

Provider Survey CIHI

1.7.1 % of providers who had complete information (essential demographics and clinical) at the point of care, most of the time, over the past 12 months (O)

Future Health information is complete and follows the client throughout the health care system

Client Experience Survey CIHI

1.7.2 % of clients who felt that unnecessary medical tests were ordered because the test had already been done, over the past 12 months (P)

Future Decreased inefficiency of health system services

Provider Survey

1.7.3 % of providers who repeated tests because results were unavailable, over the

Future Reduction in redundant testing due to lack of info availability

9 | P a g e

CIHI past month (P) Practice

Profile 1.7.4 Reduced number of 30 day re‐admissions (P)

Future Improved community‐based health care

Practice Profile

1.7.5 Primary care follow‐up within 7‐days post‐hospital discharge (P / O)

MSAA / QIP Seamless transition between health services and sectors Improved client outcomes

Health Links

1.7.6 Reduced time from PHC referral to specialist consult (P)

Future Improved timely access to specialty care

Practice Profile

1.7.7 # of specialist visits (P) Future Increased access to appropriate specialty care

Health Links

1.7.8 Reduced time from referral to home care visit (P)

Future Improved timely access to specialty care

Practice Profile

1.7.9 % of ALC days (P) MSAA Improved access to appropriate care

1.8 Is there evidence that Centres improve communication between different levels of care?

Provider Survey

1.8.1 Provider satisfaction with coordination of care (O)

Future Improved communication to and from the centre and the ER, hospital and LTC

Client survey

1.8.2 % of Centres that can access reporting about the client following (P): a) consult b) emergency visit c) hospitalization

Future Information technology used to facilitate the sharing of information

CIHI 1.8.3 % of eligible clients who report that they received case management services over the past 12 months include but not limited to (P): a) mental health b) cardiac care c) cancer care

Future Decrease duplication of services Reduced wait times for specialist services

10 | P a g e

MHWBDomain#2:PopulationNeeds‐basedDefinition: Clients and caregivers participate fully in their own care by goal setting and providing direction to services and programming.

Communities are involved in directing, planning and governing centre services. Centres plan services and programs based on population and

community needs.


2.1 Are clients actively engaged in their care?

Centre specific data

2.1.1 % of centres who report that they received or collected information over the past 12 months about the characteristics of their catchment population that was used in planning (O)

Future Increased population‐needs based planning

HQO Client Survey

2.1.2 % of clients reporting involvement in care decisions

HQO‐QIP Relative success can be assessed through comparisons to other HQO‐QIP involved centres

2.2 Are there process to obtain client, community and caregiver input regarding health care services and community programming?

Chart Review

2.2.2 # of policies in place that involve client consultation (P)

Future Policies that contribute to a culture of client‐centeredness

Program Review

2.2.3 # of program charters that demonstrate evidence of community needs assessment (O)

Future Community needs informed planning and increase in service appropriateness

Org. Survey BIRT

2.2.4 # of PDG developed based on community input (P)

Future Improved community engagement for planning

Org. Survey CI Tool

2.2.5 # of CIs developed based on community input (P)

Future Improved community involvement in initiatives

2.3 How equitable are screening services to clients?

BIRT 2.3.1 % of eligible clients who received influenza vaccinations by DOH (P)

MSAA Increased prevention of influenza

BIRT 2.3.2 % of eligible clients who received breast cancer screening by DOH (P)

MSAA Increased secondary prevention of breast cancer

BIRT 2.3.3 % of eligible clients who received colorectal cancer screening by DOH (P)

MSAA Increased secondary prevention of colorectal cancer

BIRT 2.3.4 % of eligible clients who received cervical cancer screening by DOH (P)

MSAA Increased secondary prevention of cervical cancer

BIRT 2.3.5 % of eligible clients who participated Future Improved services that address DOH of

11 | P a g e

in self‐management programs or received resources from the centre by DOH (P)

particular communities

BIRT 2.3.6 % of female clients who are pregnant or postpartum who have been screened for depression

Future 100% of eligible clients screened for depression

BIRT 2.3.7 % of clients with depression who were asked by a provider if they had thoughts about committing suicide or hurting others

Future 100% of clients living with depression screened for suicidal ideation

12 | P a g e

MHWBDomain#3:CommunityGovernedDefinition: A method of community engagement that ensures effective involvement and empowerment of local community representatives in

the planning, direction‐setting and monitoring of health organizations to address the health and wellbeing needs and priorities of populations

within local neighbourhood communities. Centres are not‐for‐profit organizations, governed by community boards made of up members of the

local community. Community boards and committees provide a mechanism for centres to represent and be responsive to the needs of their

local communities, and for communities to develop democratic ownership over “their” centres. Community governance contributes to the

health of local communities through engaged participation contributing to social capital and community leadership.


3.1 How well do centre boards understand and represent local community needs?

Org. Survey 3.1.1 % of board members that reflect socio‐demographic characteristics of centre priority populations (O)

Possible Centre boards that reflect the diversity and complexity of the community

CI Tool / social planning councils

3.1.2 # of public meetings or other community engagement activities where centre governance is the focus (P)

Future Greater alignment between centre programs and services and community needs

3.2 How do centre community boards ensure local accountability?

Org. Survey 3.2.1 % of centres with current organizational accreditation (P)

Possible Improved accountability to communities, boards and public in general

BIRT 3.2.2 % of MSAA indicators at or above target (P)

Possible Improved capacity to benchmark

Org. Survey 3.2.3 Clear role and responsibility for board involvement in QIPs (P)

Possible Increased governance accountability in quality improvement programs (QIPs)

Org. Survey 3.2.4 % of board members satisfied with their ability to direct centre programs and services around community needs (O)

Future Increased community board involvement with population needs‐based planning

AOHC 3.2.5 % of centres that sign accountability agreements with LHINs (O)

Possible Demonstrated accountability to local funder

13 | P a g e

MHWBDomain4:AccountableandEfficientDefinition: Centres are high performing efficient organizations that are accountable to their funders and the local communities served. Centres

strive to provide fair, equitable compensation and benefits for their staff. Capturing and measuring their work are essential parts of delivering

Primary Health Care. Developing and implementing meaningful indicators based on our Model of Health and Wellbeing allows for reporting to all

funders about services and programs delivered as well as the outcomes that follow.


4.1 To what extent is there provincial adoption of the MHWB?

AOHC / org. survey

4.1.1 % of Centres that have signed the MHWB charter

Possible Increased ownership by organizations to the Model

Org. Survey CI Tool BIRT

4.1.2 % of Centres that report evidence of activities that address at least 4 of the 8 domains in the MHWB (P)

Possible Improved comprehensive programming and services

4.2 To what extent is the MHWB evaluable?

PMC 4.2.1 Measurement of at least one process and one outcome indicator from each of the 8 domains of the MHWB reported annually by centre (P)

Future Improved committee governance over accountability of sector

4.3 To what extent do Centres participate in research and evaluation (including QI) activity?

Research Inventory

4.3.1 # of centres participating in formal research projects annually (P)

Possible Increased understanding of sector’s work on DOH Improved data sharing capacity Increased community‐based participatory research among sector & academic partners

Research Inventory

4.3.2 # of published peer reviewed research articles involving Centres (P)

Future Improved visibility of sector within research community

PMC 4.3.3 # of AOHC Letters of Support to researchers seeking grant funding (P)

Possible Increased recognition by researchers to engage sector

Org. Survey 4.3.5 % of centre staff that report participation in at least one formal research or evaluation (incl. QI) project annually

Future Increased participation in research and evaluation projects

4.4 What is the per capita operational cost of providing PHC in Centres?

Economic analysis OHRS

4.4.1 Average annual cost per client for PHC services delivered by (O): a) Physicians

Possible Improved efficiency of care for clients

14 | P a g e

b) Nurse Practitioners c) All other services

OHRS 4.4.2 % administrative costs (P) MSAA Increased efficiency of services for clients BIRT 4.4.3 % of Centres over 12 months that

increased their panel size without additional PCP funding (P)

MSAA Increased access to primary health care services

4.5 How well do Centres reduce non‐value added activity (waste)?

Client survey HQO

4.5.1 % of clients that report their appointments start on time or clinic wait times (P)

Future Improved client services

BIRT 4.5.2 % of Centres that have active registries of clients with multiple chronic conditions (P)

Possible Improved primary care for clients with multiple co‐morbidities

4.6 Safety BIRT 4.6.1 % of clients in last 12 months who have had their medications reviewed and discussed, including those from other physicians (P)

Future Clients with complex med profiles receive annual med reviews

Org survey 4.6.2 Centre has a process in place to ensure current medication and problem list is recorded in the EHR (P)

Future Increase in the identification of actual or potential drug interaction problems leading to a reduction in adverse events. Potential for reduction in client medications by reducing unnecessary ones

Client survey

4.6.3 % of clients indicating that they had a side effect from a medication that required a visit to a physician or emergency room (O)

Future Reduction over time in the number of clients reporting such events

Org survey 4.6.4 Is there an incident reporting system to identify and address serious or potentially serious adverse events? (P)

Future Indicates centre capacity to systematically identify and prevent serious adverse drug events

4.7 How appropriate are mental health services?

BIRT 4.7.1 % of clients living with depression offered an effective/appropriate pharmacological or non‐pharmacological treatment

Future Effective and appropriate treatment options provided to clients living with depression

BIRT 4.7.2 % of clients living with panic disorder or generalized anxiety disorder who are

Future 100% of clients living with panic disorder or generalized anxiety disorder offered effective

15 | P a g e

offered one of the following:

Psychological services / referral

Pharmacological therapy

Psychiatry referral or;

A psychosocial support group

and appropriate therapy

16 | P a g e

MHWBDomain5:CommunityDevelopmentApproachDefinition: Services and programs are driven by community initiatives and community needs; the community development approach builds on

community leadership, knowledge, and the lived life experiences of community members and partners to contribute to the health and wellbeing

of their communities. Centres increase the capacity of local communities to address their community‐wide needs and improve their community

and individual health and wellbeing outcomes.


5.1 To what extent is community development and health promotion activity informed by community needs?

CIHI Alberta Health Services

5.1.1 % of Centres that report they used info on the composition of their catchment population over the past 12 months to plan CD or HP activities (P)

Future Increased number of HP and CD activities developed based on population based need. Increased appropriateness.

Community Initiative Resource (CIR)

5.1.2 Evidence demonstrating that community development activity is informed by community needs (P)

Possible Increased number of HP and CD activities developed based on population based need. Increased appropriateness.

CIR 5.1.3 How does the approach and design of the CI account for the history and conditions in the community? (P)

Future CD activity that is designed to address barriers to access within specific communities

CIR 5.1.4 Which DOH do centre CIs address? (P)

Possible CIs that work toward improving social conditions and thereby health in our communities

CIR 5.1.5 Is the CI reaching the appropriate audience? (P)

Possible Identification of audience allows for assessment of impact with that audience

CIR 5.1.6 Is the CI addressing a priority issue for the centre’s clients or community? (P)

Future CD activity that is strategically aligned with that of the centre or sector

5.2 What is the impact of community development activity?

CIR Org. Survey

5.2.1 # of community partnerships (P) Possible Increase in the # of relationships among partners sharing similar CD goals

CIR 5.2.2 # of CIs led by the community (P) Possible Increased sustainability of CD CIR 5.2.3 # of staff involved in community

development (P) Possible Increase in interprofessional involvement in

CD Community survey

5.2.4 Increase in % of community members reporting participation in organized activities (O)

CIW Greater level of community engagement and reduction in social isolation

17 | P a g e

Community survey

5.2.5 Increase in % of community members with a sense of belonging to the community (O)

CIW Greater level of community engagement and reduction in social isolation

Org survey 5.2.6 Increase in # of centre volunteers (P) CIW Larger volunteer group Client

survey 5.2.7 Increase in % of clients that vote (O) CIW Increase in community civic engagement

CIR 5.2.8 # of community initiatives that involve a community impact evaluation (P)

Future Greater understanding of the impact of CD activity

CIR 5.2.9 What new/useful community relationships were developed? (O)

Possible Increase in strategic partnerships

5.2.10 Were any new policies or community structures developed as a result of the CI? (O)

Future Change informed by CD activity. CD becomes an approach to test community based social initiatives

CIR 5.2.11 Did community leadership or identity emerge? (O)

Possible Increase in community leadership capacity and potential for CD sustainability

5.2.12 Did community stakeholders perceive initial goals/impacts of the CI were achieved? (O)

Future Greater level of engagement by community in determining impact of CD activity (participatory and empowerment eval)

5.2.13 Has the CI developed a sufficiently broad base of supports and resources for it to be sustainable? (O)

Future Increase in likelihood for project sustainability

5.3 Evaluability CIR 5.3.1 Has a clear goal(s) been identified for the CI? (P)

Possible Increase in the use and development of evaluation indicators tailored for specific CD activity that are based on CD goals

CIR 5.3.2 Is there an opportunity for staff and community members to reflect on the CI’s development and progress (i.e., midstream eval)? (P)

Possible Increase in participatory and empowerment eval approaches

CIR 5.3.3 What evidence is being used to support the development and design of the CI? (P)

Possible Evidence based planning and design approaches can lead to greater program success

18 | P a g e

MHWBDomain6:Anti‐oppressiveandCulturallySafeDefinition: The notion that healthcare services will be provided in an anti‐oppressive and cultural safe environment is an idea that moves beyond

the traditional concept of tolerance or cultural sensitivity (acceptable to differences) to an activist orientation that seeks to eliminate the root

causes of social inequity such as historic power imbalances and systematic discrimination. It is particularly important for those involved in

systems that deliver healthcare services to understand the role that western medicine has and can play in the oppression of various ethnic,

gender, sexual orientation, economic, religious and political groups to ensure our organizations do not perpetuate such discrimination (e.g.,

eugenics).


6.1 Do centre staff reflect the social diversity of the community?

Org survey 6.1.1 % of staff that reflect centre priority populations (e.g., culturally, linguistically, etc.) (O)

Future A staff complement that reflects the diversity of the community thereby reducing cultural barriers of access

Org survey 6.1.2 Evidence of culturally‐specific programming (P)

Possible Program offerings that reflect the cultural values of the communities we serve

Practice profile

6.1.3 Disproportionate increase in % of clients from vulnerable groups (O)

Possible Fewer ethno‐cultural barriers to accessing care

6.2 Is the centre organized to support a culturally safe environment (socio‐cultural competency)?

Client survey OHRS BIRT

6.2.1 Increase # of clients being offered services in their language of choice / use of translation services (P)

Possible Increase in efficiency and effectiveness of practices (also an indicator of access)

Chart review

6.2.2 Increase in % of encounters that involve discussion of a social issue (rather than only medical) (P)

Future Demonstrates commitment to the DOH

Org survey 6.2.3 Evidence of staff education on social inequity or cultural safety (P)

Future Majority of clinical staff with this training suggests greater capacity to effectively address client concerns in these areas

Client Exp. Survey

6.2.4 % of clients reporting that they receive services that align with their beliefs and values stratified by DOH (O)

Future Equitable consideration of clients’ beliefs and values

Client Exp. Survey

6.2.5 Client satisfaction stratified by DOH (O)

Possible Equitable treatment of clients regardless of client socio‐demographic conditions

19 | P a g e

MHWBDomain7:DeterminantsofHealthDefinition: The living conditions we experience through our lifetime that are shaped by the distribution of wealth, power and resources at global,

national and local levels. The determinants of health are mostly responsible for health inequities which can be seen in the unfair and avoidable

differences in health status seen between people. Examples of determinants of health include income, education, employment, working

conditions, early childhood development, food insecurity, housing, social exclusion, social safety network, health services, gender, race, culture

and disability. In most cases these living conditions are imposed upon us by the quality of the communities, housing situations, work settings,

health and social service agencies and educational institutions with which we interact.


7.1 How well do we understand our client’s determinants of health?

BIRT 7.1.1 % of clients that had the following updated in the EHR in last 12 months: (P)

Ethnicity

Language most comfortable speaking with provider

Self‐identified Francophone

Gender

Sexual orientation

Newcomer (Year of arrival in Canada)

Employment status

Household Income

# of persons supported by Income

Chronic conditions / disabilities

Housing status

DQAT Improved data quality Ability to accurately describe client population in terms of the DOH

Org survey 7.1.2 Evidence that community needs assessments are being used to inform program planning (P)

Possible Dynamic programming that addresses emerging community needs

BIRT 7.1.3 Increase in the ratio of social v medical issues addressed during encounter (P)

Future Equitable consideration of social issues, particularly within medical practice

7.2 How well do our services address the determinants of health?

Org survey CIR

7.2.1 # of formal partnerships that deal specifically with the DOH (P)

Possible Increased community capacity to address the DOH

NOD – BIRT 7.2.2 % of PDGs focused on DOH (P) Possible Evidence of importance of DOH within health

20 | P a g e

Org. Survey promotion practice CIR Org. Survey

7.2.3 % of CIs focused on DOH (P) Possible Evidence that community capacity building efforts incorporate the DOH

Practice profile (ICES) NOD‐ BIRT

7.2.4 Rate of ODSP participation (P) Possible Evidence that we serve those we intend to serve

Centre specific data

7.2.5 # of centre policies that address DOH (P)

Future Demonstration of organizational commitment to addressing the DOH

Centre specific data Org. Survey CIR

7.2.6 Evidence of advocacy for healthy public policy (P)

Possible Evidence that organizations are working to change systems to address the needs of our clients

CIW BIRT

7.2.7 Reduction of % of homeless / precariously housed clients (O)

Future Evidence that organizations are working towards addressing social issues which correlate with poor health outcomes

BIRT Practice Profile (ICES)

7.2.9 % of uninsured clients who received a new health card number over the past 12 months (O)

Future Reduction in the % of clients without insurance, except for new clients

Client survey

7.2.10 % of vulnerable clients who rate their health positively (O)

Possible Increase in clients’ positive perceptions of their own health despite the social challenges they face

7.3 Does health promotion and prevention address the DOH?

BIRT 7.3.1 % of clients screened for: (P)

Diabetes

Asthma

Congestive Heart Failure

Coronary Artery Disease

Mental Illness Addictions

Future Increase in screening rates over time or maintenance of high screening rates

Client survey

7.3.2 % of eligible clients that report receiving specific help or info on: (P)

Future 100% of eligible clients are being provided with health promotion support

21 | P a g e

Tobacco use

Eating habits

Physical activity

Alcohol use

High‐risk sexual practices

Unmanaged stress NOD –BIRT 7.3.3 # of internal referrals to health

promotion education sessions over 12 months (P)

Possible Demonstrates effective health promotion referral and recruitment within centre

NOD – BIRT 7.3.4 # and attendance to health promotion education sessions over 12 months (P)

Possible Demonstrates effective health promotion activities

Centre specific data BIRT

7.3.5 # of health promotion programs that involve an outcome evaluation (P)

Future 51% of health promotion programs involve outcome evaluation

22 | P a g e

MHWBDomain8:AccessibleDefinition: Clients should be able to get timely and appropriate healthcare services to achieve the best possible health outcomes. Access is multi‐

dimensional: affordability, availability (i.e., getting care when a person needs it), geographic accessibility (i.e., location of the care provider

relative to where the client lives), accommodation (e.g., expanded hours of operation; ability to obtain an appointment in a reasonable time

frame) and acceptability (e.g., physical access to the clinic, culturally appropriate services, receiving care from the appropriate health

professional). Access is only achieved if all its components are ensured.


8.1 How accessible are Physicians and Nurse Practitioners?

ICES 8.1.1 Reduced number of avoidable ED visits (O)

QIP Reduction in CTAS scores of 4 or 5 (non‐urgent)

ICES 8.1.2 Reduced unnecessary hospital admissions (O)

QIP Reduction in unnecessary hospital admissions

Org Data 8.1.3 Increase # of complex clients with regular and timely access (O)

QIP Reduction in barriers to access for those who generally experience barriers to care

Org Data 8.1.4 Acceptable 3rd next available appointment (P)

QIP Reduction in average length of time between request for an appointment and the 3rd next available appointment

Org survey NOD – BIRT

8.1.5 % of MDs and NPs that report they are accepting new clients (P)

MSAA Increase in panel size

HQO survey

8.1.6 % of clients who report same day access (O)

QIP 100% of clients who need same day access get it

8.2 How accessible is the interprofessional team?

BIRT 8.2.1 % of clients accessing interprofessional teams by type of providers (O)

MSAA (diabetes only)

Equitable accessibility to the entire interprofessional team

Org survey 8.2.2 # of extended or on‐call hours (P) Possible Increase in availability of services Client

survey 8.2.6 % of clients that report easy access during regular hours using phone or email (P)

Potential Increase in availability of services

Org survey BIRT

8.2.3 % of centres that offer: (P) a) liaison with home care or; b) the provision of home care services

Potential Demonstration of the navigator function by organizations

Org survey 8.2.4 % of centres that report they provide the following: (P)

Potential 100% of clients can access a comprehensive primary healthcare team

23 | P a g e

Acute episodic care

Non‐urgent care (e.g., well baby care, prenatal care, chronic disease management)

Prevention and health promotion services

Primary mental health care

Psychosocial services (e.g., counselling advice for physical / emotional / financial concerns)

Case management for vulnerable populations

Referral to and follow‐up care from specialized services

Rehabilitation and reintegration services

Nutrition counselling

Dental services

End‐of‐life care

24 | P a g e

Glossary

ActiveClientA registered client who has had either an individual service encounter or was involved in a personal development group session within the last

three years.

AccessPointAn Access Point refers to the health services that are defined by:‐ less than 21 hours per week health services delivery, perhaps in conjunction

with a partner‐ delivery of a specific health service or bundle of services‐ staffing originate from the main centre or satellite‐ no reception or

administrative support staff‐ administration and infrastructure support by the main centre.

Baselineinformation

Information collected at the beginning of a project that serves as the basis for comparison with information collected later

BIRTBusiness Intelligence Report Tool is a bilingual, centrally hosted system that consolidates data and enables data analysis. Cognos is the reporting

tool used to access, create and/or run reports on CHC data in BIRT.

CommunityInitiativeA community initiative is a set of activities aimed at strengthening the capacity of the community to address factors affecting its collective

health.

DataMinerData Miner is a report/decision support tool that is being used to access individual centre NOD data primarily being used for reporting. The data

management coordinator will primarily use this tool.

ElectronicHealthRecord(EHR)An EHR is a full electronic patient record, with a variety of data input capabilities, health maintenace tracking, clinical decision support with

alerts interoperable with internal and external systems including interfaces to multiple practice management systems that complies with

25 | P a g e

principles of documentation. The elements of an HER are health information and data, results management, order entry/management, decision

support, electronic communication and connectivity, patient support, administrative processes, and reporting/population health management.

IndicatorsIndicators are specific measures indicating the point at which goals and/or objectives have been achieved.

LogicModelA program logic model is a diagram that shows what a program is supposed to do, with whom and why. Logic models typically include

information on a program’s target population, intended activities, intended inputs and intended outputs, and intended outcomes.

TargetpopulationsTarget populations include the individuals, groups, organizations or communities for and with whom a program’s services are designed.

They are a program’s priority population or its intended reach.

InputsInclude resources dedicated to or consumed by the program. Examples are money, staff, and staff time, volunteers and volunteer time,

facilities, equipment, and supplies.

OutputsThe direct products of program activities and are usually measured in terms of the volume of work accomplished (eg. # of counseling

sessions conducted, # of people served)

OutcomesOutcomes are a change that occurs as a result of a project or program and show the benefits or changes in people or groups

participating in a program. Outcomes are often associated with impact evaluations. There are different levels of outcomes: short term,

intermediate and long term.

MandatoryDataThis data is necessary for a record to be valid. The field cannot be blank. The validation rules will check to ensure that all mandatory data is

recorded. If there is missing data, an error message will appear on the screen after attempting to save.

26 | P a g e

NODNOD is “Nightingale‐on‐Demand”, the current Electronic Health Record that is being utilized by NPLCs, CHCs, and AHACs.

NORANORA is the Non‐Operational Reporting and Analytics work stream; it is a group of interrelated projects to integrate data from multiple systems

and provide participating organizations with a holistic view of operations and clients served.

OngoingPrimaryCareClientOngoing Primary Care Clients (OPCC) are registered clients who receive ongoing primary care from a GP/NP. This group forms the denominator

for the MSAA accountability reporting.

Outcome(orimpact)evaluation:assesses what your project has achieved

PerformanceManagementCommitteeThe Performance Management Committee (PMC) is the authoritative source for performance management activities that support and enhance

the full scope of the CHC Model of Health and Wellbeing. PMC makes recommendations and provide strategic guidance to the Strategy Group

and CHC /AHAC Provincial ED Network on provincial level initiatives related to the setting of performance and data standards, sector‐wide

reporting, decision‐support, quality improvement initiatives, research, and accountability agreements to support the provision of better quality

of care that improves the health of individuals and their communities. In order to complete the variety of work plan deliverables, PMC has four

sub‐committees: Standards; Research; Clinical Advisory; and Health Promotion and Community Development.

PersonalDevelopmentGroupA Personal Development Group (PDG) is defined as a series of time‐limited or on‐going sessions conducted, facilitated or supported by internal

or external staff, whose purpose is to effect changes in participating individuals’ behaviour, knowledge or attitudes. Groups may have a specific

purpose (i.e.

Processevaluation

Assesses what activities were implemented, the quality of implementation and the strengths and weaknesses of the implementation

Referrals

27 | P a g e

A referral directs a client from a source health provider to a target health provider, recommending the type and and length of care required by

the client in a secure and efficient manner. The referral management process includes creating, sending, revising, updating and responding to a

referral.

RequiredDataThese data fields must be completed for Ministry reporting, where possible. Missing data in the required fields will result in

incomplete/inaccurate Ministry/CHCs local MIS reports. There will be no error message on the screen if field left blank.

SatelliteSatellite is a permanent location where health services are delivered outside of the main centre, characterized by:‐ regular operations (minimum

of 21h/week);‐ fixed, accessible location, secured through ownership, a lease or a written agreement‐ ongoing staff‐ its own administrative and

infrastructure support

ServiceEventThe term used to describe an encounter with an individual client, a session for a personal development group and a monthly report for a

community initiative. The term “encounter” is often also used to signify a service event.

ServicesProvidedServices provided are those list of actions undertaken:

CaseConferenceThis is the attendance at a scheduled meeting with other providers and/or family members and/or the client to discuss and develop a

plan of care to benefit the client. Encounter under Services Provided “case conference”.

CaseManagement/CoordinationThis is the provision of support, counselling, coordination, advocacy and/or life skills instruction to long term clients with complex issues.

Encounter under Services Provided “Case management/coordination”.

ConsultationA consultation is when two providers discuss the service(s) or quality of care being provided to the client, and the details of the

discussion is significant enough to be charted.

28 | P a g e

ExternalExternal consultation means to request or provide an opinion of/for an external provider. Encounter under Services Provided

“consultation – external consultation”.

InternalInternal consultation means to request or provide an opinion of/for an internal provider. Encounter under Services Provided

“consultation – internal consultation”.

Referral

ExternalExternal referral is defined as a referral of a client made to a provider/service outside the centre. This includes written referrals.

Encounter under Services Provided “external referral” AND Referrals.

InternalInternal referral is defined as a referral of a client made to a provider/service within the centre. This includes written referrals.

Encounter under Services Provided “internal referral” AND Referrals.

CulturalInterpretationA service provided by a CHC – an interpreter (volunteer or paid by the CHC), is used to provide language interpretation during a contact

between a provider and a client. This is a service and should be collected as such. The interpreter is not captured as a staff involved.

Cultural interpretation alone does not warrant an encounter. Note: the language of contact is always the language the provider spoke

while delivering the service.

TargetPopulationsTarget populations are the priority populations as identified by broad organizational, CI and groups objectives. They are often related to

sociodemographic characteristics of the target population.

ValidationRulesA set of rules applied to each encounter to ensure that the minimum mandatory data set has been completed. An encounter should not be

saved until all of the validation rules have been met and the encounter is complete.

29 | P a g e

ReferencesA Primary Health Care Evaluation System for Nova Scotia. Nova Scotia Department of Health [Internet] 2006 October;[cited 2012 August 23]

Available from: http://www.gov.ns.ca/health/reports/pubs/Primary‐Health‐Care‐Evaluation‐Report‐2006.pdf

Alberta Quality Matrix for Health: User Guide. [Internet] September 2005;[cited 2012 June 4] Available from:

http://www.hqca.ca/assets/pdf/Matrix/User_Guide_R290506.pdf

Canadian Institute for Health Information. Pan‐Canadian Primary Health Care Indicator Development Project [Internet];[cited 2012 June 4]

Available from: https://secure.cihi.ca/estore/productSeries.htm?pc=PCC330

CIHI & Statistics Canada. The 10th Anniversary. The Indicators Project. Report from the Third Consensus Conference on Health

Indicators.[Internet] 2009; [cited 2012 June 4] Available from: https://secure.cihi.ca/free_products/82‐230‐XWE_e.PDF

Government of Canada, Treasury Board of Canada results‐based management accountability framework (2010). Available from: http://www.tbs‐

sct.gc.ca/cee/tools‐outils/polrmaf‐polcgrr‐eng.asp

Hoffman, K. (2009). A Decision Support System for Community Initiatives: Background and Recommendations for Action: Final Report. Submitted

to the AOHC.

Ontario Primary Care Performance Measurement Summit. Proceedings Report. (2012). Ontario Ministry of Health and Long Term Care & the

Canadian Institute for Health Information.

Posavac, E.J. & Carey, R. G. (2010). Program Evaluation Methods and Case Studies (8th ed). Prentice Hall, Upper Saddle River, New Jersey.

Primary Health Care Evaluation Framework. Primary Care Branch, Alberta Health, November 2013:

http://www.health.alberta.ca/documents/PHC‐Evaluation‐Framework‐2013.pdf

Program Evaluation Framework. Primary care initiative [Internet];[cited 2012 Aug 23] Available from:

http://www.albertapci.ca/OperatingPCN/Evaluation/Pages/ProgramEvaluationFramework.aspx

Raphael, D. (Ed.). (2009). Social Determinants of Health: Canadian Perspectives. 2nd edition. Toronto: Canadian Scholars’ Press Incorporated.

The IHI Triple aim [Internet]; [cited 2012 July 11] Available from: http://www.ihi.org/offerings/Initiatives/TripleAim/Pages/default.aspx

30 | P a g e

Watson, D.E., Broemeling, A.‐M. , Reid, R. and Black, C. (2004). “A Results‐Based Logic Model for Primary Health Care: Laying an Evidence‐Based

foundation to Guide Performance Measurement, Monitoring and Evaluation.” Vancouver: Centre for Health services and Policy Research.

Watson, D.E. 2009. “for Discussion: A Roadmap for Population‐Based Information systems to Enhance Primary Healthcare in Canada.”

Healthcare Policy 5(Sp): 105–20.

Weinstein, S. Bramalea Community Health Centre / Four Corners Health Centre Community Initiative Evaluation Guide (2014). Consultant

Report.

World Health Organization. (2008). Closing the Gap in a Generation: Health Equity through Action on the Social Determinants of Health. Geneva:

World Health Organization.

WHO. Primary Care Evaluation Tool. [Internet]; [cited 2012 May 28] Available from:

http://www.euro.who.int/__data/assets/pdf_file/0004/107851/PrimaryCareEvalTool.pdf

model of health and wellbeing evaluation framework · 2018-04-01 · performance management, the...

Documents