meaningful measurement how do we articulate the value of public libraries in a contemporary context?...

Meaningful Measurement

How do we articulate the value of public libraries in a contemporary context?

SWITCH Conference 2015

Jackie Bailey, Principal BYP Group

http://bypgroup.com

http://bypgroup.com/

(c) BYP Group 2015 http://bypgroup.com

Exercise:

Jot down words or doodle pictures in answer to the following question:

When I think about ‘evaluation’ and libraries, I feel….

What is evaluation?

• What, if anything, has changed because of us?

• Who would miss us if we were gone?

• Why would they they miss us?

• What value do we create for society, which no one else can (or at least, not as efficiently as we do)?


Society and culture

Library users

You and library staff

Why evaluate?


Improve•Use your findings to continuously improve•Challenge your own or others’ assumptions•Discover unexpected outcomes

Communicate

•Tell people about the value you create in ways they can understand

Advocate•Use your findings to advocate for what libraries do, which no one else can

The unexamined life is not worth living - Socrates

Formative evaluation

•Establish the baseline•Identify a ‘theory of change,’ linked to the research•Identify goals and indicators or success

Process evaluation

•Look at how you work e.g. partnerships, operations

Summative evaluation

•Measure the impact of your activity over the short, medium and long term, and on various people or groups

Stages of evaluation


What do I evaluate?


• Who is this evaluation for?

• How will this evaluation be used?

• What do I need to know?

• What is a nice to have, and what is a must have?

• What can actually be measured, given my time and resource constraints?

Evaluation is…

for us (e.g. program improvement)

Process evaluation

Impact evaluation

for funders (e.g. lobbying for funding)

Impact evaluation

Return on investment

Funders’ objectives

For example…

What are ‘impacts’ and how (on earth) do I measure them?


Warning: these ‘impacts’ are off the top of my head and should just be

read as an example!

What are the impacts?•Access to resources they would not otherwise easily afford•Opportunity for social bonding•Opportunity for self-expression and identity•Sense of community connectedness, belonging•Personal capacity development as a result of using library resources

How do I measure them?

•Number of users and frequency of usage•User demographics, including access to like resources•Survey of user experience, benefits, suggestions, uses, self-assessment of the impact of the library on them•Survey of user capacity development (preferably referring to benchmark data)•Gather stories of ‘change’ amongst library users – e.g. how people’s lives are different BECAUSE of the library•Creative tools e.g. before and after pictures, ‘if there was no library,’ vox pops, chalkboards

Library Users




read as an example!

What are the impacts?

•Sense of community identity, public spaces, civil society•‘Institutional value’ of the existence of libraries (John Holden)•Sense of having resources if needed


•Survey of wider community perceptions and attitudes towards libraries (e.g. through local council surveys)•Contingent valuation survey

Community members




read as an example!

What are the impacts?•Social and cultural capital•Economic outcomes e.g. support for small businesses, further education and employment

How do I measure them?•Theoretical assertions based on your findings, and drawing sound conclusions about your contribution to the determinants of social and cultural capital as outlined in the research literature•Study of business, education and job seeker usage

Society, economy and culture




read as an example!

What are the impacts?•Contribution to the ongoing development of library culture and information systems•Capacity development of staff


•Staff surveys•Internal team reflection and discussion•Conversations with respected peers or mentors•Awards

You and the art of ‘librarianism’

Possible methods


User impact surveys

• Demographics of users• Level of access to like resources• Suggestions and needs• Qualitative impacts e.g. sense of connection, social bonding, confidence, self-efficacy

Stories of change

• Vox pops• Writing exercise• Pictures of before and after, or the community with/without a library• Chalkboard words• Interviews and group discussions

Wider community value survey

• Contingent valuation• Public perception of libraries

Return on investment studies

• Social ROI• Economic ROI

How some organisations use their evaluations

Penrith City Council Booklet & DVD Awards – Australian National

Disability Award for Excellence in Social Participation; Arts and Health Award

Arts Centre Melbourne: Published report and summary

report (flipbook) Online video for donors

MCA Funding reports Peer to peer learning

Improve, Communicate, Advocate



Exercise:

Jot down words or doodle pictures in answer to the following question:

Now, when I think about ‘evaluation’ and libraries, I feel….


What, if anything, changed?

Discussion

Jackie Bailey, PrincipalBYP Group

http://bypgroup.comM: 0428 576 372

E: [email protected]

http://bypgroup.com/

mailto:[email protected]

meaningful measurement how do we articulate the value of public libraries in a contemporary context?...

Documents

advocate c byp group

principalbyp group http

comc byp group

principal byp grouphttp

doodle pictures

following question

value of public libraries

social participation