Usage data in the publishing world and an update on COUNTER

Peter Shepherd

DirectorCOUNTERJune 2007

If I may be so bold as to inquire, to what degree do you wish to interact with Franz Kafka?

Average Time Spent and Number of Articles Read Per Year Per Scientist(Tenopir et al)

0

20

40

60

80

100

120

140

1977 1978-1983

1984 1985-1989

1990-1993

1994-1998

2000-2001

Number ReadHours Spent

Scholarly Article Reading

Work Field Articles Reading (Per Year)

Time Spent (Hours)

Time Per Article (Min)

University Medical Faculty

~322 118 22

Chemists ~276 198 43

Physicists ~204 153 45

Engineers ~72 97 81

Browsing Searching

Core titles Current issues Background Current

awareness

New topics Old articles Primary research For writing

Sources of Reading

0%10%20%30%40%50%60%70%80%90%

100%

AAS ORNL UTK

Paper

Other e-

E-prints

E-journals

Publishers need usage statistics

To demonstrate that reduced usage of print issues has been compensated by increased online usage

To assess the relative importance of the various channels via which its content reaches the market

To experiment with new pricing models To support editorial policy To obtain improved market analysis/demographics To inform authors where and how articles are used To improve site design and navigation To plan infrastructure, e.g. mirror sites/caches

So how are we getting there?

ICOLC Guidelines for statistical measurement of usage of web-based information resources

National Commission on Libraries and Information Science (NCLIS) Electronic access and use-related measures

NISO – Z39.7 (Library Statistics) ISO – 2789 (library statistics) and 11563

(library performance measures) MESUR – investigate metrics derived from the

network-based usage of scholarly information COUNTER (Counting Online Usage of

NeTworked Electronic Resources)

Background

Understanding usage Different approaches Role of usage statistics

Usage statistics Should enlighten rather than obscure Should be practical Should be reliable Are only part of the story Should be used in context

COUNTER Achievements Current status Future challenges

COUNTERCodes of Practice

Definitions of terms used Specifications for Usage Reports

What they should include What they should look like How and when they should be delivered

Data processing guidelines Auditing Compliance Maintenance and development of the

Code of Practice Governance of COUNTER

COUNTER: current Codes of Practice

1) Journals and databases

Release 1 Code of Practice launched January 2003 Release 2 published April 2005 replacing Release 1 in

January 2006 Now a widely adopted standard by publishers and librarians 60 vendors now compliant 10000+ journals now covered Librarians use it in collection development decisions Publishers use it in marketing to prove ‘value’

2) Books and reference works

Release 1 Code of Practice launched March 2006 6 vendors now compliant Relevant usage metrics less clear than for journals Different issues than for journals

Direct comparisons between books less relevant Understanding how different categories of book are used is

more relevant

Journal and Database Code of Practice

Usage Reports

Journal Report 1 Full text article requests by month and journal

Journal Report 2 Turnaways by month and journal

Database Report 1 Total searches and sessions by month and database

Database Report 2 Turnaways by month and database

Database Report 3 Searches and sessions by month and service

Code of Practice for books

Book Report 1 Number of successful requests by month and title

Book Report 2 Number of successful section requests by month and

title Book Report 3

Turnaways by month and title Book Report 4

Turnaways by month and service Book Report 5

Total searches and sessions by month and title Book Report 6

Total searches and sessions by month and service

Journal Report 1Full text article requests by journal

Html and PDF totals reported separately

COUNTER Audit

Independent audit required within 18 months of compliance, and annually thereafter

Audit is online, using scripts provided in the Code of Practice Auditor can be:

Any Chartered Accountant Another COUNTER-approved auditor

ABCE is the first COUNTER-approved auditor Industry-owned Not-for-profit Independent and impartial Part of ABC (Audit Bureau of Circulations) Providing website traffic audits for over 150 companies and

certifying over 1400 domains Have successfully completed test audits on COUNTER usage

reports

ABCE Audit fees

FEES CHARGEABLE Standard Fee COUNTER Members

Year 1 Year 2 Year 1 Year 2

[A] Registration Fee £1,556   £612  

[B] Annual Subscription £852 £852 £621 £621

Journal Report 1 £1,480 £1,480 £1,480 £1,480

Journal Report 2 £740 £740 £740 £740

Database Report 1 £1,110 £1,110 £1,110 £1,110

Database Report 2 £740 £740 £740 £740

Database Report 3 £370 £370 £370 £370

Total cost of Journal Report 1 audit £3,888 £2,332 £2,713 £2,101

ABCE audit fees

Year 1 (first audit) £2,500 Includes 1.5 man-days pre audit consultancy as well as 1.5 man-days audit

Max 20 million records (add 0.5 man-days per 10 million or part thereof) Max 50 reports (add 0.5 man-days per 50 reports or part thereof) Assumes 1 data source for all reported numbers, in correct format,

delivered to agreed timescale Ongoing support (technical, administrative & marketing) Reduced by £250 for COUNTER members

Year 2 (and ongoing per audit) £1,500 Includes 1.5 man-days audit

Max 20 million records (add 0.5 man-days per 10 million or part thereof) Max 50 reports (add 0.5 man-days per 50 reports or part thereof) Assumes 1 data source for all reported numbers, in correct format,

delivered to agreed timescale Ongoing support (technical, administrative & marketing)

At ABCE’s normal daily consultancy rate

COUNTER: deriving metrics from Journal Report 1

Local metrics For libraries and library consortia At journal, collection and publisher level To compare the cost-effectiveness of journal

subscriptions To assess the value of Big Deals

Global metrics For authors, funding agencies, libraries and

publishers At journal, collection and publisher level To compare quality and value

COUNTER: ‘local’ metrics

JISC (UK Joint Information Systems Committee) Funded by UK higher education funding councils Supports higher education in the use of information and

communications technologies Access to information and communication resources Advice on creation and preservation of digital archives Implications of using ICT Network services and support Research to develop innovative solutions

National overview of online journal usage Develop a reliable, widely applicable methodology Use COUNTER Journal Report 1 ‘article full-text requests’

Local metrics: an example

COUNTER data was analysed in relation to: usage range Price band Subject category

Metrics derived from this analysis Trend in number of full-text article downloads Full text article requests per title Full text article requests per publisher package Full text article requests per FTE user Most requested titles Usage of subscribed vs.. unsubscribed titles Cost per full-text article downloads Cost per FTE user

Summary report available at:www.ebase.uce.ac.uk/projects/NESLi2.htm

Local metrics: an example

Growth in full-text article downloads Publisher A: 12%- 208% Publisher B: 12%- 59% Publisher C: 23%- 154% Publisher D: 22%- 81%

Cost per full-text article download Publisher A: £0.97- £5.26 Publisher B: £0.70 - £2.91 Publisher C: £0.80 - £3.29 Publisher D: £0.45 - £2.26

COUNTER: ‘global’ metrics

Impact Factor Well-established, easily understood and accepted Endorsed by funding agencies and researchers Does not cover all fields of scholarship Reflects value of journals to researchers Over-emphasis on IF distorts the behaviour of authors Over-used, mis-used and over-interpreted

Usage Factor Usage-based alternative perspective Would cover all online journals Would reflect value of journals to all categories of user Would be easy to understood

Global metrics: UKSG Project

Assess the feasibility of developing and implementing journal Usage Factors

Level of support from author, librarian and publisher communities

Data from which UF would be derived COUNTER Journal Report 1? Article numbers Process for consolidation, calculation and reporting of UFs

Factors in the calculation Level of reporting Total usage Articles

Report in April 2007 Just completed set of 29 interviews with industry leaders Wider online survey will take place in February 2007

UKSG Project: feedback

Are the COUNTER usage statistics sufficiently robust? Frustration at lack of comparable, quantitative data on journals Should items covered by restricted to articles? Many journals still have significant usage in print Diversity of views on the factors in the calculation

Specified usage period Specified publication period

Usage data is more susceptible to manipulation Will the journal be a meaningful concept in the future? Two measures with different limitations are better than one,

and UF will be derived from a set of credible, understandable data

Usage data will be used as a measure of value, whether publishers like it or not

Current issues Interface effects on usage statistics

E.g. downloading HTML and PDF of the same article in one session

COUNTER has tested data filter solutions, but what does the duplicate downloading signify?

Reporting separately purchasable digital archive usage

Currently all usage for a journal is usually reported together

Separately purchasable archives mean we need separate reports for archival content, or a year of publication breakdown of usage

Usage in Institutional Repositories Growth in Institutional Repository (IR) content Need for credible IR usage statistics IR usage statistics already being collected, but no

standards SUSHI Improving consortial usage reports

Current usage reports inadequate New reports in XML format

Reporting separately purchasable digital archive usage

Increasingly requested by librarians Interim solution

Journal Report 1a:Number of Successful Full-Text Article Requests from an Archive by Month and Journal

Optional additional usage report Longer-term solution

Journals Report 1a? Include year-of-publication data in JR1?

SUSHI

Standardized Usage Statistics Harvesting Initiative (SUSHI)

No mechanism yet for automatically retrieving, combining, and storing COUNTER usage data from different sources

NISO-sponsored XML-based SUSHI aims to provide a means to do just this, via a standard model for machine to machine automation of statistics harvesting.

COUNTER and NISO have signed an agreement to work together on the development of SUSHI. More details of SUSHI can be found at:-

http://www.niso.org/committees/SUSHI/SUSHI_comm.html

Future challenges

Improving/extending the Codes of Practice Reliability ( audit, federated searches, prefetching) Usability (number of compliant vendors, XML format,

additional usage reports) Additional data (year of publication, article level

reports) Categories of content (Institutional Repository content)

Deriving metrics from the Codes of Practice Journals (cost per use, Usage Factor) Databases? Books?

Next steps…..

Release 3 of Code of Practice for Journals/Databases

Features: prioritisation on basis of demand and practicality Process: consultation via focus groups,etc; publication of

draft CoP Release 2 of Code of Practice for Books

Review R1 in practice Other categories of content ( eg Institutional

Repositories) Metrics derived from the COUNTER usage statistics

Cost per use Usage Factor

COUNTER Membership

Member Categories and Annual Fees (2007)

Publishers/intermediaries: £530 Library Consortia: £355 Libraries: £265 Industry organization: £265 Library affiliate: £106 (non-voting

member)

Benefits of full membership Owner of COUNTER with voting rights at

annual general meeting, etc. Regular bulletins on progress Opportunity to receive advice on

implementation

http://www.projectcounter.orghttp://www.projectcounter.org

Apply for COUNTER membership

Apply for COUNTER membership

For more information……….

http://www.projectcounter.org

Thank you!

Peter Shepherd, COUNTERpshepherd@projectcounter.org