Exploring the Evidence in Evidence-Based Acquisition

Exploring the Evidence in Evidence-Based AcquisitionSpeakerStephanie SprattElectronic Resources and Serials LibrarianUniversity of Colorado Colorado Springs sspratt@uccs.edu

Co-AuthorGabby WiersmaHead of Collection Development University of Colorado Bouldergwiersma@colorado.edu

Co-AuthorRhonda GlazierDirector of Collections ManagementUniversity of Colorado Colorado Springs rglazier@uccs.edu

Co-AuthorDenise PanAssociate Director for Technical ServicesAuraria Library (University of Colorado Denver)denise.pan@ucdenver.edu

Shared E-resource Licensing and Assessment at CU

Longstanding culture of shared e-resource licensing among CU libraries

CU libraries are now building a culture of shared assessment of e-resources

Research Questions for Streaming Video EBA with Alexander Street• Initial Questions• What types of usage metrics are provided?• What can libraries interpret from usage statistics?• What are the issues with the data provided in usage reports?

• Research Questions• How does EBA compare to PDA?• How do the workflows compare to other resources or models?• Do usage statistics provide sufficient evidence to inform purchase decisions?• How will the consortia decide which titles to purchase?

EBA v. PDA Considerations

• Requires an upfront financial commitment• Cost known at start of program

• Mediated - library decides what to purchase by the end of the contracted time period (up to 1 year) through usage analysis• EBA program has a set title list

• Free MARC records for discovery

• Platform provides accessibility features; LMS integration; clip/playlist creation; citations

• Library pays on quarterly invoices as videos are used• Less control on amount of money spent• Deposit account is available but not required

• Unmediated by library – patrons’ use triggers licenses (default 1 or 3 years)• Library can choose which subject areas/producers to

activate

• Free MARC records for discovery

• Platform provides accessibility features; LMS integration; clip/playlist creation; PPR

Alexander Street EBA Model Workflow

SetupMaintenanceAssessment

• Financial commitment and time period set at start

• Track balance in Admin Portal • Establish access points,

MARC records, discovery layer

• Patrons trigger metrics (30 seconds = 1 playback)

• Add desired videos to cart and submit • Perpetual access; varied prices

• Spend entire commitment by program's end

• Can continue program for another period

Kanopy PDA Model Workflow

SetupMaintenanceAssessment

• Library determines budget, license default (1- or 3-year)

• Library can create a deposit account, if desired

• Access points: MARC records, platform link, discovery layer

• Patrons trigger metrics (4 views within one year of activation triggers a license)

• Patrons must click to begin playback• Flat 1-year license price for all videos

• Invoices are sent out quarterly

• Unused budget amount is rolled forward

Workflow Comparison Deciding on access points into platform (i.e. main

landing page and/or Community pages) could take time and communication with subject specialists

De-duplication may be necessary

Staff must monitor usage and make purchasing decisions for consortium by end of program

Individual title selection could cause double payment if subject collection is purchased in the future

Library must decide if second period of EBA is warranted

Selecting subject area or producer collections takes more time than activating entire catalog

De-duplication may be necessary

Staff do not have to monitor usage for video licensing to trigger

Quarterly invoices mean more staff time spent paying

Deposit option would require tracking

Licenses would need to be reviewed for renewal before expiration

Available Usage Metrics

EBA Administrative Portal:

http://adminportal.alexanderstreet.com• Current EBA commitment and remaining balance $• Playbacks

Usage Statistics Portal: http://admin.alexanderstreet.com• COUNTER reports MR1, MR2, DB1, PR1, CR2, CR3• Playbacks by subject area• Playbacks by collection

• Dashboard and Analytics• Visits• Pages• Plays, PDA Plays• Minutes• Subjects• Pages/visit• Plays/visit• Plays by Device

Alexander Street-Usage Statistics Portal

Challenges with EBA program• Usage stats • Two statistics portals• Lack of unique identifier to connect data • Usage stats included collection and subject information that was disconnected from AS subject

collections • Data quality hiccup

• MARC record quality – non-Library of Congress defined subject headings

• Denver campus already owned some content that was heavily used on other campuses

• Desired subject collection purchases but EBA was designed to lead to title-by-title purchases

Kanopy DashboardSummary

Daily Activity

Minutes Viewed

Kanopy Dashboard, Pt.2Top 10 VideosPlays by Device

Engagement

Top 10 CountriesTop 10 Suppliers

Kanopy-PDA Performance

Assessment Metrics Average cost per title used = Total commitment / Number of titles used

Average cost per playback = Total commitment / Number of playbacks

Benefit = Actual cost of all titles used

Cost benefit analysis (CBA) = Benefit / Total commitment

Return on investment (ROI) = (Benefit – Total commitment) / Total commitment * 100

Assessment Metrics - Results

Evidence-Based Decision MakingEBA

• Campus leads reviewed title use• Titles viewed at all campuses = 0• Found common top subject areas across

campuses: Counseling & Therapy, Ethnography

• Asking AS to provide title lists for these subject collections and add all to cart

• Remaining individual title purchases being selected by use count and subject specialist input

• Cost splitting still to be determined

• Significant usage automatically triggers a purchase

• Useful for scoping or expanding PDA• Top 10 Suppliers• Subject Analysis

What We Learned• Promote programs

• Cost of piloting a new model• Admin Portal training• MARC record management

• Managing expectations for faculty

• Assessment• EBA metrics cannot be compared to PDA Metrics (purchase vs. license)

Which Model Would You Select?• EBA is best for

• Libraries with room in their budgets for perpetual access streaming videos

• Institutions with broad program offerings

• Libraries needing patron input on which subjects have need for streaming video resources

• PDA is best for

• Libraries with more limited budgets

• Libraries valuing access over ownership of streaming videos

• Libraries receiving requests for streaming videos in specific subject areas

Next Steps• Alexander Street—Boulder, Denver, Colorado Springs campuses replacing EBA

with individual AVON subscriptions• On EBA purchases – will not duplicate Denver-owned content; will select titles with subject

interest across all campuses (Counseling & Therapy / Anthropology)• Communication from vendor indicates that commitment fee does not decrease for a second

period of EBA• AVON to own option provides opportunity to buy titles that have ongoing popularity• AVON is more cost effective than EBA in terms of cost per title, setup, maintenance• Monitor improvements to consortial report options

Next Steps• Kanopy—all campuses will continue PDA for another year • Will devise license management workflow, ERM tracking• Video licenses are triggered throughout the year, so license renewal is not one-time

process• Budgeting for license renewal in addition to continuing PDA• Will monitor consortial opportunity development• No consortial discount offered• EBA program – no titles viewed at each campus• Central management might be easier, but workflows not in place

Exploring the Evidence in Evidence-Based AcquisitionThank you for attending!Any questions?SpeakerStephanie SprattElectronic Resources and Serials LibrarianUniversity of Colorado Colorado Springs sspratt@uccs.edu

Co-AuthorGabby WiersmaHead of Collection Development University of Colorado Bouldergwiersma@colorado.edu

Co-AuthorRhonda GlazierDirector of Collections ManagementUniversity of Colorado Colorado Springs rglazier@uccs.edu

Co-AuthorDenise PanAssociate Director for Technical ServicesAuraria Library (University of Colorado Denver)denise.pan@ucdenver.edu