Assessing User Experience of E-Books in Academic Libraries

Tao Zhang, Marlen Promann

1

Xi Niu Department of Software and Information Systems University of North Carolina at Charlotte

Purdue University Libraries

Project motivation

•  Growing e-books in library collections •  1.5 million, 41.9% of total

volumes at Purdue •  Physical space •  Cost •  Access

•  Positive attitude of faculty and students

•  Actual user experience? •  Exploratory study

2

Research goals

•  Explore how users search for e-books in the library’s discovery tool.

•  Understand how users use e-books. •  Understand how users find information in e-books.

3

Research goal

•  Explore how users search for e-books in the library’s discovery tool: •  Log analysis •  Observation in user tests

4

Discovery tool

5

Log analysis

•  Apache http logs from Sep., 2014 •  284,912 queries •  6,854,362 actions

•  Measures •  Initial query distribution •  Query level: query length, facet selection •  Session level: number of queries and http requests in search sessions

6

Initial query distribution

7

Search Tab # Search Queries

All (default) 44,502 (77.1%)

Books & Media 8,342 (14.5%)

Articles 3,532 (6.1%)

Course Reserves 1,335 (2.3%)

Total initial queries: 57,623

Query-level results

8

All searches (Sep. 1, 2014)

All searches (Sep. 12, 2014)

Book searches (Sep., 2014)

Average 3.75 3.79 3.35

Median 4 3 2

SD 3.92 3.66 3.61

Query length

Query-level results

9

0

2000

4000

6000

8000

10000

12000

14000

0 1 2 3 4 5 6 7 8 9

Facetdepthlevel

Searchquerieswithfacetselec>ons

Query-level results

10

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

Publica>onDate

ShowOnly:PeerReview

Subject ShowOnly:OnlineAccess

MaterialType:Books

Language Author

Topfacetselec>ons

Session-level results

11

All searches (Sep. 1, 2014)

All searches (Sep. 12, 2014)

Book searches (Sep., 2014)

Average 2.96 3.30 2.04

Median 1 1 1

SD 5.15 6.07 2.99

Queries per session

All searches (Sep. 1, 2014)

All searches (Sep. 12, 2014)

Book searches (Sep., 2014)

Average 41.42 41.80 102.13

Median 21 21 73

SD 69.49 73.48 131.07

HTTP requests per session

Log analysis summary

•  Users most likely use “Search All” with no more than two facets •  Book searches vs. general searches:

•  Query length (effect size ~ 0.10) •  Queries per session (effect size ~ 0.24) •  Actions per session (effect size ~ 0.33)

•  Identify relevant results from similar titles •  Examine item details

12

Research goal

•  Understand how users use e-books: •  How long do users use e-books? •  How many users choose to read online vs. download e-books? •  How many pages do users read?

13

Usage log analysis

•  EBL e-book usage logs from Jan. 1 to Nov. 25, 2014 •  29,495 individual reading sessions •  Data fields:

•  Date & time •  User ID •  E-book title, ID •  Publisher, subject category •  Page numbers browsed

14

Reading session duration

•  69% reading sessions lasted less than 10 minutes •  92% sessions lasted less than 30 minutes •  2,261 sessions (8%) had download action

15

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

0

2000

4000

6000

8000

10000

12000

100

1400

2700

4000

5300

6600

7900

9200

10500

11800

13100

14400

15700

17000

18300

19600

20900

Freq

uency

Dura,on(seconds)

Dura,on

Frequency

Cumula>ve%

Pages browsed

•  80% sessions involved no more than 30 pages browsed by users

16

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

0

2000

4000

6000

8000

10000

12000

10

50

90

130

170

210

250

290

330

370

410

450

490

530

570

610

Freq

uency

Pages

Pagesbrowsed

Frequency

Cumula>ve%

Pages browsed

17

Page numbers browsed by users across e-books and grouped by publishers and subjects https://goo.gl/EXij2l Tableau visualization:

Usage log summary

•  E-books were not read from beginning to end •  Short reading sessions, relatively small number of pages read •  Browsing patterns

•  Initial parts of book – Table of Contents – Start of a chapter •  Non-sequential reading, flipping back and forth •  Focus on certain page ranges

Users mainly use e-books to find discrete information and sections (“fact searching”)

18

Research goal

•  Understand how users find information in e-books: •  How does users’ experience level affect their information seeking behavior? •  How do users use book navigation features (table of contents, list of figures/

tables, index)? •  How do users use full text search in e-books? •  What are users’ difficulties interacting with e-books?

19

User test

•  12 participants •  Faculty, undergraduate and graduate students •  Screening survey measured library and e-book experiences •  4 randomly selected from each level: beginner, intermediate, and expert

•  Exploratory search tasks on general topics •  Own topic, Apple iMovie software, Marketing, …

•  Information finding tasks In White’s (2008) book on neuroscience, find the definition of Ideational Apraxia.

Open Norris’s book “Electoral Engineering: Voting Rules and Political Behavior” and find a table that lists the ideal functions of political institutions. In Geismar’s (2011) book Identify Basic Principles of Identity Design in the Iconic Trademarks of Chermayeff & Geismar, find the meaning behind the Chase Manhattan Bank identity design in the 1960s. …

20

User test

•  TechSmith Morae for task performance measures •  Pre- and post-test interviews

•  E-book use •  Overall experience, things to be improved

•  Observation •  Search (discovery tool, within e-books) •  Navigation (page, table of contents, index, list of tables & figures) •  Copy, annotation, download

21

E-book interface

22

E-book interface

23

E-book interface

24

Search tasks

•  Observations •  Start from default “Search All” •  Simple keyword search and browse first page of results •  Few further query formulation •  Low usage of facets •  Rare use of advanced search

•  Users’ difficulties: •  Scan large number of results •  Identify material type (book vs. article, journal, report, etc.) •  Identify format (print vs. online access) •  Determine relevance of results (only title, author, publication year in results

list)

25

Information finding task - video

26https://goo.gl/XnCVJu

Search functionality in e-books

•  Any keyword matching for multi-term queries •  Exact phrase search not supported •  Long list of search results

27

Differences among experience levels

•  Beginners searched more •  Beginners clicked more in text (page numbers, table of contents) •  Beginners scrolled more (not supported in some e-books) •  Experts relied more on index, table of contents, list of figures/

tables to navigate

28

Difficulties in interaction

29

Copy text

Difficulties in interaction

30

Copy text

Difficulties in interaction

31

Rotate, view change

Information finding tasks

•  Users expected searching in e-books similar to web searches •  E-books designed for linear reading, not for flipping/scanning •  Single page view, limited context for skimming •  Copyright restrictions (copy, download, print, special software) •  Discomfort of reading on screen

32

Design recommendations

•  Search •  More information in search results (table of contents, book summary,

reviews) •  Promote facet usage

•  Reading •  Better full text search •  More emphasis on print book features (table of contents, lists, index) •  Flexible page view options (rotate, multiple-page view) •  Less restrictions on copy, annotation, print, download

33

34

Project supported by the 2015 ER&L + EBSCO Library Fellowship

Thank you!