using the logic model for impact & success; #sla2017

Using the Logic Model for Impact & Success

SLA 2017 Monday @ 10:30Rebecca Jones, Director, Branches & Neighbourhood Services @ Brampton [email protected]

2017

My lens•must use evidence for decision-making

• Healthy blend of evidence, emotion & risk

•key questions: what, so what, now what• crucial to start with where you are, plan small

progressive steps that won’t kill your Achilles’ heel •everyone’s lens looks through their mental models

Logically…...

Our logic isn’t everyone’s logic

The Logic Model

Where we hope many miracles occur

Good work, but we may need a few more details right here….

Impact & Success…...

Impact according to who

?

Board ChairKim Kardforyour Mony • Socially connected

• Lobbying to have library budget reallocated to other projects• Has never used the library• Buys all her books from

Amazon• Why does the library need so

many staff? And why are they paid so much?

VP ResearchDr. Ural Doomed

• Libraries days are numbered • Libraries aren’t relevant• Everyone has computers and internet• Need information? It’s all online• Research happens in faculties not

libraries

• With edocs, ebooks, open data and research online, believes facilities can be made smaller – no need for print material

CFO Roi Bottomline

• All about the bottomline• Revenue and investments• How do your services support

revenue generation?• Services that don’t generate

revenue are a drain on the system• If it’s not efficient, outsource

it!

Dean of HumanitiesKee Paiper

• Influential in many circles• Has deep seated beliefs in traditional

library services and importance of paper• Libraries are institutions for archives

& research• Needs to demonstrate to college

that humanities still contributes to reputation

Logic model in action for Dr. Papier

Input(resources)

• One liaison librarian

Output(operations)

• Faculty research support for a Grant Application• Completed application submitted on time

Outcome(user)

• Grant dollars received by faculty• Faculty member is able to hire 2 grad students to conduct

research

Impact(stakeholder)

• Increased university visibility in research area• Enhanced university reputation• A number of articles are submitted and accepted by academic

journals, increased citations for faculty member

Project Outcomes: Jones & Raven 14

Resource Perspective

What we put in

Operational Perspective

What we deliver

User Perspective

What comes out

of it for customers

Stakeholder Perspective

What changes for customers

input

outcomeoutput

impact

Customer

Outcome for Client

Stakeholders

17

Impact for Stakeholder

“”

BEGINWITH

THEIN MIND

ENDCovey 1989

Stakeholders

19

Impact for Stakeholder

Customer

Outcome for Client

start with where you are

Our “frames” & mental modelsOur “frames” & mental models

How should we frame this issue?

How should we frame impact & success?

Outcomes Change Growth

It’s hard

What measures do you keep now?

• No one magic measure

• “Impact” or value is dynamic, economic, psychological & relative to alternatives

• Identifying outcomes & especially impact takes a long time & heavy duty analysis

• Successful organizations:

• clarity of purpose

• understand their culture

• performance measurement system that fits that culture

Operational/Usage Measures• We’ve long tracked “how much”• We also need to look at differences:• Peaks? Dips? Switches?

• What are these statistics really telling us? Or, not?• Who needs to see these statistics?• Are they telling a story that you want to

tell?

Operational

Satisfaction Measures• Tends to focus on existing products

and/or services• Usually measured through surveys• Are we also measuring importance?• Survey burnout, plus the

expectation is to score all“excellent” or “10 out of 10”

Satisfaction

Value Measures

• What do you have?• Qualitative?• Testimonials, attributes

• Quantitative?• Budget

Value

Value

SatisfactionOperational

Focus on the sweet spot 72 % of First-year Students engaged with Library staff services in Fall 2017 semester

85% of these Students rated these services as very important in addressing their workload & assignment completion

70% of all Students used the Library in Fall 2017

83% of all these Students rated the Library as the 2nd most important service related to academic success on campus, 2nd only to their faculty

Measurement Framework

Understand the Context

Align Strategies & Objectives

Identify Services & Programs

Define Measures

Manage Measurement

Data

Translate Data into

Outcomes & Impacts

Convey Results

Know the Context (& the Contexters)

How do theymeasure theirprogress?

Know your Stakeholders – who are they?

How is value or impactdefined,measured &communicated?

Know the language of your context

Metrics

Value

Customer

Growth

StudentSustain

ResidentsConstituent

We can't come to terms with new concepts until we learn new language

Horseless CarriagesCar Phones

ATM’s or PTB

Start with stakeholders

Name Position How do they measure their progress?

Goals & objectives

What do we know about them? What type of stakeholder are they?

What do they know about our services?

Critical that you understand the context in which the measures are considered. 1. In your organization, how is value defined, measured and conveyed?

2. Who are the stakeholders who must understand the value of your services?

The Logic Model

…goes beyond documenting what you did and measures what difference you made in the life (work, reputation, etc. )of your target audience…What has been the positive change in their sphere… as a result of what you do?

It’s logical that:

INPUT

resource perspective

OUTPUT

operational perspective

OUTCOME

user perspective

IMPACT

stakeholder perspective

IMPACT

Equipment

INPUTResource Perspective

• Equipment• Space• Supplies• Funding• Staff

IMPACT

Equipment

OUTPUTOperational Perspective

• Training sessions completed• # of attendees• Grant application• Reports

IMPACT

OUTCOMEUser Perspective

• Increased skill to do• More or better know-how or know-that• Change in status• Change in behavior or attitude (confidence)

IMPACT

IMPACTStakeholder Perspective

• Long term results• Lower dropout rates• Increased contract engagements• Increased recovery rates


Stakeholder’s Goals & Objectives Your Goals & Objectives: How they Contribute

Accelerate product to market Catalyze product development

Improve market share in specific industry segments

Customize segment dashboards for business development

Increase research grants in xxx Underpin the research process and assist with submission and completion of grants

Identify Services

What services, programs or products contribute to your realizing an objective?

Stakeholder Goals & Objectives

Your Goals & Objectives

Your Services





Define Measures





Define Measures

Manage Measurement

Data

Manage Measurement Data

• What will success look like?• How will you track progress towards meeting goals?• What indicators or measures will you use?• Qualitative? Quantitative?• What data needs to be collected?• Keep it key• Keep it simple• Be practical• Don’t get mired – or

overwork frontline staff

Manage collection

• Determine a collection schedule•Work out agreements with other departments that

have needed data• Are you currently collecting data or indicators that

are no longer relevant?• How long do you need to keep data?•Who is responsible?





Define Measures

Manage Measurement

Data

Translate Data into

Outcomes & Impacts

Define outcomes & impact

•What will success look like?

• And how will you “get” there?

OUTCOME IMPACT

INPUT OUTPUT

Government Library Example

Input(resources)

• $100K resource budget• Staff to advise on development of policies

Output(operations)

• 2500 resources utilized by 75% of department employees• Library has contributed to the development of 6 policies in past

12 months

Outcome(user)

• 6 policies approved and in place• Eliminate department-specific resource purchases saving the

organization $35K

Impact(stakeholder)

• Credible policies• Reduce policy duplication, “blind-sides” or contradictory policies

by at least 25% (as reported during policy debriefings) • Reduced costs to run department

Corporate Library Example

Input(resources)

• $80K electronic content contract• Tech and applications to integrate library’s products and resources into the

departments applications• Skilled info pro to develop research portfolio for business development

department

Output(operations)

• Customized research portfolio in response to request for background on deal between XYZ company & ABC government agency, including specific data on terms how ABC likes deal structured

• Library resources are integrated into business development applications

Outcome(user)

• Client extrapolated how ABC agency likes to see deals structured and structured deal appropriately

Impact(stakeholder)

• Organization successfully completed a deal with ABC government agency for $10M, the 1st in 5 years





Define Measures

Manage Measurement

Data

Translate Data into

Outcomes & Impacts

Convey Results

Communicate to…

Remember us

?

Convey results

• Actually begins when you are understanding the context

• Your message: “we have contributed towards your goals by……”

• If your goals & objectives are meaningful for your planning & decision-making, your measures will be indispensable

c

Value

SatisfactionOperational

Sweet Spot

Logically, we need to reallocate our time and attention

[email protected]

REBECCAJONES

Thank you!

using the logic model for impact & success; #sla2017

Data & Analytics