jose marie griffiths evaluating social and economic impact
DESCRIPTION
Public library: Evaluating social and economic impactTRANSCRIPT
Work Stream Two
Evaluating Social and Economic
Impact
José-Marie Griffiths
School of Information and Library Science
University of North Carolina at Chapel Hill
European Congress on E-Inclusion: ECEI09
Technology and Beyond in Public Libraries
Brussels
October 22-23, 2009
Introduction
• Scope of Work Stream
– How to evaluate social and economic
impact of accessible technologies in
public librariespublic libraries
– How libraries can demonstrate the
impact of the investments made
– Which criteria are needed to evaluate
social and economic impact
Key Questions
• What are current approaches to the
evaluation of public library impact
and how are they being applied?
• What studies have been undertaken to • What studies have been undertaken to
assess specifically the performance of
e-inclusion services within public
libraries?
Questions cont’d• How have these evaluative tools
affected the wider municipal, regional
or national policy agendas?
• What should be the future approaches
to the evaluation of public libraries to the evaluation of public libraries
within the context of increasing direct
access to networked services within
communities
Potential Outcomes• Feedback on the range of current
evaluation activities within public
libraries
• Examples of evaluation that has
demonstrated the social and economic demonstrated the social and economic
value of e-inclusion in public libraries
• Future trends in terms of how public
service can be evaluated successfully
within the networked society
Measuring Return-on-
Investment
in Public Librariesin Public Libraries
Return-on-Investment (ROI) for
Public Libraries
� has been applied to many different types of
organizations and community resources
� application of cost/benefit, cost-
effectiveness, impact and return-on-effectiveness, impact and return-on-
investment measures
� for-profit sector - common
� to libraries, museums, schools and
colleges, parks, etc. – not common
Importance of ROI for Public
Libraries
• Tight budgets
• Competing national, regional and local interests
• Must make the case in quantitative terms, not just anecdotal evidencenot just anecdotal evidence
• Economic valuation is a powerful tool for advocacy
• Data must be collected and analyzed in the context of what is important to the communities within which the libraries operate
Why ROI is More Difficult to
Calculate for Libraries• Difficulty of quantifying benefits that vary
from
� user to user
� use to use
� from library to library (as service mixes � from library to library (as service mixes vary)
• The push for public libraries to develop services relevant to the needs of their local communities, has made the evaluation process more difficult.
• As a result, libraries tended to focus on user satisfaction and other attitudinal measures.
Early Efforts -
Value Assessment
• Example: Griffiths/King study - 1982, Office of Scientific and Technical Information of the U.S. Department of Energy — develop approaches to assessing/measuring the value of the assessing/measuring the value of the Energy Database
• Three Levels of Value Assessment
�Willingness-to-pay or exchange value
�Use value
�Consequential value
Early Public Library
Evaluations
• Example: Griffiths/King ROI studies–
1989, U.K public libraries; 1991,
Massachusetts public libraries; 1993 U.K
policy briefing
• Need to create an aggregate picture of
library value
• Compare this value to the total investment
in libraries - return-on-investment
Costs, Outputs and Outcomes
Need to consider:
• The costs (investments) of the libraries
and their services
• The outputs produced
• The use of the outputs• The use of the outputs
• Outcomes resulting from that use
– Improved quality of life
– Support for lifelong learning
– Support for the community’s economy
Total Economic Value*
• Use value - net willingness to pay
• Option value - willingness to pay for the option to use in the future
• Existence value - willingness to pay for the good/service to exist even though no future good/service to exist even though no future use is contemplated
• Bequest value - willingess to pay for the endowment of the good or service for future generations
* North Carolina Blue Ridge Parkway Study, 1999-2002*
Jobs and Income
• Center for Economic Development
Research at the University of South Florida
- economic contribution or impact of
various corporations and institutions to the
statestate
• Measured:
– Jobs and jobs created (both paid and volunteer)
– Personal income (wages and disposable
income)
– Local output (value of goods and services
resulting from jobs created)
Contingent Valuation• economic method of evaluation for non-
priced goods and services
• looks at the implications of not having the goods/services.
• includes • includes
– added cost to use alternatives sources of information, should people choose to do so (also called net benefit);
– portion of direct economic contribution public libraries make to their communities that would be lost;
– portion of economic benefits to the library users that would be lost.
Case Study -
State of FloridaState of Florida
Methods
Used a variety of data collection and
analytic methods including – data reported to the state by the libraries
– a statewide household telephone survey of
adults
– in-library user surveys of adults
– a follow-up survey of the libraries
– surveys of organizations
– an input-output econometric model
Household Interviews/
In-Library Survey
Collected information about:– cost to use the library
– services used
– reasons for using the services
– importance of the services; ways the services – importance of the services; ways the services were important
– what visitors would do to obtain the needed information if there were no public library
– estimated cost in time and money to use the alternatives
– used critical incident approach
Organization Survey
Determined:
– use of public library services
– cost to use these services
– savings resulting from service use– savings resulting from service use
Follow-up Survey
Obtained some information about:
– use by tourists and school age children
– interlibrary lending and borrowing
– expenditures and income
– business-like operations run by the library,
outside persons or vendors.
REMI
• Addresses the economic effect a public organization or resource has on other economic sectors over time
• Econometric input-output model• Econometric input-output model
• Extends the economic contribution of libraries beyond the actual users of the libraries to yield a set of direct, indirect and induced effects to the served communities
Results and Presentation
Results
• Figured on lower bound - so results were at
least as good as reported
• Emphasis on “bottom line” results (with
detail available for those who wanted it)
• Correlation with interests important to state • Correlation with interests important to state
and community leadership and decision-
makers
• Provision of effective graphics in reports
and presentations
Florida’s Public Libraries
Return $6.54
for every $1.00 Invested
+ + =Net Benefit Lost Community
Benefits
Lost Community Spending
Community BenefitsUse Benefits
Lost Uses
Lost Use Benefit
Economic
Return
$2,993.660
million
Economic
Return
$2,993.660 million
Investments$448.903 million
Return on
Investment
6.54 to 1.0÷=
User Investment to Use the
Investments (costs)
Federal
Funding
State
Funding
Local
Funding
Other
Funding Multitype
Benefits (of having the library)
To
individualusers:
availability
To
organizations as users
of the
library:
availabilitty
To community/
state
Pass through
Halo
Staff
Compensation
In-state spendingTo
individualusers:
use
Community BenefitsUse Benefits
Expenditures by the
library
Staff
Compensation
Expenditures
In-state
out of stateIn-state
In-state
out of state
Electronic
Expenditures
Media
Expenditures
Other
Operating
Expenditur
es
Capital
Outlay
In-state
out of state
In-state
out of state
In-state
out of state
For
IndividualsFor
Organizations
Cost to use alternatives
User Investment to Use the
Library $1.7 billion
Individuals
$1,721 million$1,721 million
Organizations
$2.384 million$2.384 million
Revenue Investment
Economic Return:
$2.93 Billion
Economic Returns ResultingFrom Florida's Public Libraries
(total $2.93 billion)
80%Added cost to use80%Added cost to usealternatives
15%Lost direct communityeconomic benefits
5%Lost direct user
economic benefits
+
New Jobs —
1 for every $6,448 invested
Florida’s Public Libraries
Increase GRP
Florida Public Libraries
Increase Income in the State
$12.66$14.00
Income Increases from Public Support of Florida Public
Libraries
$1.00
$0.00
$2.00
$4.00
$6.00
$8.00
$10.00
$12.00
Publicsupport tolibraries
Floridaincomeincrease
Economic Ripple Effect
• Effect of public investment in public
libraries:
– GRP $4.0 billion increase
– Wages $5.6 billion increase in– Wages $5.6 billion increase in
personal income
- Jobs 68,700 jobs created
Net benefit: $4.9 billion or 3.7 to 1
Florida’s Public Libraries are
Extensively Used —
94 million In Person and Remote Visits
68 million
in-person
visits
25.2 million
remote
Internet
connections (not including
children or
tourists)
Florida’s Public Libraries are Used
by 11.8 Million People Annually
Communicating Results
to Stakeholdersto Stakeholders
Variety of Report Formats
• Executive summary/overview report
• Detailed methodology and analysis
document
• Additional analysis and survey • Additional analysis and survey
instruments
Variety of Presentations
Based on target audiences
• Results-oriented summary
• More detailed utilization data as well
as resultsas results
• Methodologies and detailed analysis
Variety of Publications -General taxpayer audience
Variety of Publications -Community/State leaders audience
Variety of Publications -Library directors/leadership audience
Presentations and Press
From Economic to Social Impact
Social Return-on-Investment
• Economic value of cultural, social and
environmental impacts
– How to value the cultural wealth created – How to value the cultural wealth created
by public libraries?
– How to value the contribution public
libraries make to larger policy
objectives?
– How to value the social worth of public
libraries?
Public Library Valuation:
Needs & Opportunities
• Study conducted in 2006-07
• Americans for Libraries Council
• Support from The Bill & Melinda
Gates FoundationGates Foundation
• Excellent overview of economic
valuation studies
• Call for broader valuation to include
social impact valuation
Social Responsibility Models
• Balanced scorecard
• Triple-Bottom-Line Accounting
• Corporate Social Responsibility
Reports
Recommendations
• Improve coordination and communication
among stakeholders
• Develop a comprehensive research agenda
that promotes systematic valuation of
libraries’ contribution to education, civic libraries’ contribution to education, civic
participation and quality of life
• Create a varied set of innovative tools
• Take advantage of valuation and evaluation
reporting lessons from other sectors and
fields
Recommendations cont’d
• Define a national agenda for library
valuation research as part of research
coordination infrastructure
• Seek support for impact assessments
enabling libraries, advocates and
researchers to work together to evaluate the
impact of different studies in different
contexts
Economic Impact in the Arts -
Matarasso
• Voluntary labor and donations
• Consumer spending
• Employment and training for work
• Investing in local communities• Investing in local communities
• Savings in public expenditures
Social Impact in the Arts -
Matarasso
• Personal development
• Social cohesion
• Community empowerment and self-
determinationdetermination
• Local image and identity
• Imagination and vision
• Health and well-being
Social Impact Measurement
Areas - Matarasso
• Making life better
• Creating public arts projects
• Involving local people
• Developing people’s skills and resources• Developing people’s skills and resources
• Strengthening partnertships
IMLS Interconnections Study
• Conduct national survey of information
needs of users and potential users of online
information
• Primary focus on museums, public libraries
and the Internet as sources
• Telephone surveys of adults (18 and over)
The Power of Trust
Conclusion 1:
Libraries and museums evoke
consistent, extraordinary public
trust among diverse adult users.trust among diverse adult users.
Libraries and museums are the most trusted
sources of information according to a survey
of over 1,700 adults.
Conclusion #2:
The Internet Does Not Kill
Libraries and Museums
Conclusion #2:
Internet use is positively related to
in-person visits to museums and
libraries.
Adults Who Use the Internet are
More Likely to Visit Libraries and
Museums
71.0%
Proportion of Public Library and Museum Visitors Who Use or Do Not Use the Internet
66.4%
38.3%
73.3%
66.7%
47.4%
0% 20% 40% 60% 80% 100%
Proportion of Adult Visitors
Museums
Public
Libraries
Internet users
Non-users of the Internet
All adults
Adults Who Use the Internet Visit
Libraries and Museums More Often
3.36
3.44
Public
Average Number of Public Library and Museum Visits by Those Who
Use and Do Not Use the Internet
3.14
1.34
3.46
3.42
3.36
1 2 3 4 5
Number of visits per adult
Museums
Public
Libraries
Internet users
Non-users of the Internet
All adults
Conclusion #3:
Q. Why do we need
museums and libraries
if we have the Internet?
A. Interconnections
Conclusion #3:Museums and public libraries
in-person and online serve important and
complementary roles in supporting a
wide variety of information needs.
To fulfill their need for information, most adults use
museums, public libraries, and the Internet. Museums
and public libraries are used by 70%, the Internet is
used by 83%, and nearly half (47%) use all three.
Only 7% of adults do not use any of the three sources.
Interconnections
Use of Public Library
Workstations
• 49 % of 149 million in-person visitors
to public libraries used library-
provided workstations a total of 294
million times during the previous 12 million times during the previous 12
months
• 69% or 203 million of those 294
million uses were for Internet and
other online resources and services
Services Used from Public
Library Workstations
• Search engines 70%
• Obtaining info from libn or library 53%
• Looking at other website 46%
• Used e-mail 36%• Used e-mail 36%
• Viewing/downloading articles 36%
• Viewed blog 18%
• Viewed/downloaded e-books 9%
• Used chat or IM 7%
• Other 5%
Reasons for Using Public
Library Workstations
• Convenience/ease of use 87%
• Best source of information 74%
• Information could be trusted 58%
• Low cost (time and $$) 58%• Low cost (time and $$) 58%
• Don’t own computer or currently
unavailable 6%
• Other 4%
Time Spent Using Public
Library Workstations
• 1 - 10 minutes 31%
• 11 – 20 minutes 19%
• 21 – 30 minutes 18%
• 31 – 60 minutes 29%• 31 – 60 minutes 29%
• 1 -2 hours 2%
• Over 2 hours 2%
Average time:
Per in-library online visit: 29 minutes
Per remote online visit: 63 minutes
Purpose for Using Public
Library Workstations
• Meet educational needs 53%
• Meet personal or family needs 41%
• Meet work-related needs 33%
• Recreation or entertainment 27%• Recreation or entertainment 27%
Satisfaction with Attributes of
Public Library Workstations(scale: 1 – 5 – 5 is high)
• Hours of availability 4.14
• Software available 3.92
• Hardware available 3.89• Hardware available 3.89
• Number of workstations
and their accessibility 3.87
• Amount of time allowed in
a single session 3.67
Outcomes of Using Public
Library Workstations
• Obtained needed information
– Got all information sought 50%
– Got some information sought 46%
• Addressed reason for use• Addressed reason for use
– Completely 62%
– Somewhat
• Led to favorable outcome 94%
• Led to negative outcome 4%
• Too much irrelevant information 24%