framing(&(defining(...

Framing  &  Defining  Metrics  for  Libraries

Rebecca Jones rebecca@dysartjones.com

HOT topic

Know the language of your context

Metrics  Value  

Evalua:on   Assessment  

Measures  

Measure: info about what is

Assess: test against a target

Evaluate: judge the findings

h=p://goo.gl/kJrzVw  

Symptoms  

Underlying causes

What’s below the water causing the symptoms?

•  We don’t tend to have a “culture of assessment”…

•  Most “measures” convey past performance

•  Difficult and complex

•  Our “measures” don’t align with decision-makers

We’ve learned that:

•  Identifying & illustrating value depends on conversations with your stakeholders –  …the first conversation shouldn’t be when measures are

presented

•  “More” isn’t necessarily better –  …in fact, “more” clouds the issue and the message

Our frames & mental models

h=ps://goo.gl/WOjS7N  

Why should we frame this issue?

•  No one magic measure

•  Successful organizations:

•  clarity of purpose

•  understand their culture

•  performance measurement system that fits that culture

•  “Value” is dynamic, economic, psychological & relative to alternatives

•  Identifying outcomes & especially impact takes a long time & heavy duty analysis

IT’S HARD

Center for Investigative Reporting’s Frame

Non-profit media & their funders have been working on impact for years 4 “strands of the conversation”:

ü Standards ü Engagement ü Online analytics ü Offline activities

Oh, those offline activities

“Because offline activities are messy and irregular and require staff and time to track, they are cast aside as, at best, qualitative data and, at worst, anecdotes. But qualitative data is data, and it is as valuable as quantitative data if it is collected and analyzed using rigorous methods.”!

Impact CIR’s definition: Change in status quo @ macro, meso or micro levels

“we want to understand the relationship of convertibility among these outcomes. We are only beginning to think about how to quantify these interactions; this is not alchemy, but science – and maybe a little art. “

•  Dissection •  App in Podio

Your frame

Your assumptions & beliefs about measurements now:

Ideas, discomfort, provoking

Ideas, discomfort, provoking

Your assumptions & beliefs about measurements at noon tomorrow: