determining & demonstrating value with the logic model

Determining & Demonstrating Value with the Logic Model

Adapted from a Presentation by

Mohamed Hosseini-Ara of City of Markham

Rebecca Jones of Dysart & Jones

The logic of the old model …

Determining & Demonstrating Value With the Logic Model - Paul Burry

2

Demonstrate to Whom?


3

The Framework

Understand the Context

Align Strategies & Objectives

Identify Services & Programs

Define Measures

Manage Measurement

Data

Translate Data Into Outcomes

& Impacts

Communicate the Results


4


How is value defined, measured & communicated?

Know your stakeholders—who are they?

How do they measure their progress?


5

Stakeholders

Identifying and illustrating value requires relationships/conversations with stakeholders


6


7

Start With Stakeholders


8

Start With Stakeholders


9


10


11


12


13

Clear Identity & Pride, Supportive/Engaged Community, Equity & Inclusion, Cultural Richness


14


Enhance Community Well-Being & Engagement, Create Welcoming Physical & Virtual Spaces


15


Enhance Community Well-Being & Engagement, Create Welcoming Physical & Virtual Spaces

Local History Collections & Speaker Series, Teen Queer/Straight Alliance Program, Locally Sourced Food/Farming Programs, Accessibility Initiatives


16

Defining Measures

What Measures Do We Currently Use?

Operations Customer Satisfaction

Value or Impact


17

Defining Measures



Value or Impact

CirculationOpen HoursPeople CountProgram AttendanceInternet UseReference Queries


18

Defining Measures



Value or Impact

Circulation Focus GroupsOpen Hours SurveysPeople Count Program

EvaluationsProgram AttendanceInternet UseReference Queries


19

Defining Measures



Value or Impact

Circulation Focus Groups ?Open Hours Surveys ?People Count Program

EvaluationsProgram AttendanceInternet UseReference Queries


20

Meaningful Measures…

Matter to you AND your stakeholders

Are conveyed in stakeholders’ language

Focus attention on what is most important for the organization

Are critical for managing, planning, & decision-making

Are organization-dependent


21

Problems with Conventional Approaches to Measurement

Not a “culture of assessment”

Many measures convey only past performance

Often measures don’t align with stakeholder goals and concerns

Measuring impact/value can be complex

What does success look like?


22

Measuring Impact/Value

You’re measuring a change in the human condition:

Behaviour Awareness

Skill Motivation

Knowledge Condition

Attitude Status

Circumstance


23

The Logic Model

Goes beyond documenting what you did, and measures what difference you made in the life of your target audience. How have they changed?


24

The Logic Model

InputResource

Perspective

OutputOperational Perspective

OutcomeUser

Perspective

ImpactStakeholder Perspective


25

The Logic Model

InputResource

Perspective


OutcomeUser

Perspective


• Equipment• Space• Supplies• Funding• Staff


26

The Logic Model

InputResource

Perspective


OutcomeUser

Perspective


• Training Module

• Program• Report• # of Programs• # of Attendees


27

The Logic Model

InputResource

Perspective


OutcomeUser

Perspective


• Increased Skill• Increased Knowledge• Change in Status• Change in Behaviour

or Attitude• Satisfaction level


28

The Logic Model

InputResource

Perspective


OutcomeUser

Perspective


• Long-term Effects

• Lower Dropout Rate

• Increased Literacy Level

• Increased Recovery Rate


29

If we do this

InputResource

Perspective


OutcomeUser

Perspective


The Logic Model


30

If we do this This should happen

InputResource

Perspective


OutcomeUser

Perspective


The Logic Model


31

Begin With the End in Mind


32

STEM Program for GirlsInput(resources)

Output(operations)

Outcome(user)

Impact(stakeholder)


33


Output(operations)

Outcome(user)

Impact(stakeholder)

• Girls ages 10-12 are interested in pursuing STEM careers (short-term)

• Girls take math, science, computer courses in high school and pursue STEM university degrees (mid-term)

• Girls pursue careers in STEM and become scientists programmers, doctors (long-term)


34


Output(operations)

Outcome(user)

• Girls ages 10-12 increase understanding of STEM careers• Increased interest in STEM course options• Positive response to questionnaire regarding

appropriate careers for girls (engineers, scientists)Impact(stakeholder)





35


Output(operations)

• 6 week program offered twice per year• 25 girls ages 10-12 complete each session• Pre and Post session questionnaires (permission to follow

up in 3 years)Outcome(user)







36


• Staffing; program design; marketing materials, facilities, equipment, speakers (6 female STEM professionals)

• $1,500 to pay for honorariums and supplies

Output(operations)

• 6 week program offered twice per year• 25 girls ages 10-12 complete each session• Pre and Post session questionnaires (permission to follow

up in 3 years)Outcome(user)







37

The Framework




Define Measures

Manage Measurement

Data


& Impacts



38

The Framework




Define Measures

Manage Measurement

Data


& Impacts



39

Manage Measurement Data

Plan/schedule data Collection

Who is responsible? What other work areas need to be involved?

Are you currently collecting data that is no longer relevant?

How long do you need to keep your data?


40

Translate Data into Outcomes & Impacts

Define Success: what will it look like? How willyou get there?

Remember: You’re measuring social impact, not library performance/success

Be aware of possible confounding variables

Impact can be inferred, even when your data/measurement isn’t perfect


41

Translate Data into Outcomes & Impacts

What does the data say? What does it not say?

No one organization is going to solve any major social issue: partner with other community organizations

Nobody said it would be easy!

Totally killing it!


42


Begins at the “Understand the Context” stage

Dear stakeholder: we contribute X toward your goals

You DID build that!


43

What Should You Stop Doing?


44

Thank You!

Paul Burry: [email protected]

More at:http://www.metricsandlibraries.org/program.html


45

determining & demonstrating value with the logic model

Documents