Team JSTOR / Group 1:

Morgan Burton - Isabela Carvalho

Stan (Tze-Hsiang) Lin - Leo (Lei) Shi

Data for Research (DfR) for JSTOR

Introduction to DFR• System that includes metadata, information

visualization, and article retrieval for JSTOR articles

• JSTOR is a major database of scholarly articles

• Provides “facets” or “selectors” that allow the user to filter their search

based on specific elements such as journal, author, and discipline

• Provides graphs that update dynamically based on search query

• User base:

• User might be a researcher such as a doctoral student in linguistics, or a more

casual researcher interested in comparing trends across disciplines (not

exhaustive)

Methods• Interaction map

• Provides a map of the sections of the site

• Personas and Scenarios• A glimpse at what the typical user and situation might be for the system• 5 Interviews conducted on potential users

• Comparative Analysis• several competitive systems including Google Scholar and NINES

• Survey• We surveyed over 20 target users

• Heuristic Evaluation• An evaluation of general usability principles

• Usability Testing of 5 target users

Finding 1• The overall purpose of DfR is clear to users at first

glance – prior to interacting with the system

• Usability testing result: we tested prior finding from heuristic analysis that purpose of site might be unclear at first glance

• We asked users to fill out pre-task assessments where we asked them to answer questions about their expectation of the system

• Form asked users about what their general idea of the site was

• Result: User expectation matched what site was about and accurately inferred relationship to JSTOR

Recommendation 1• (Contrary to prior finding) do not include an explanatory

sentence on the main page about DfR

Finding 2• lack of visual indication of interrelationship

between search and select features

• ‘Graphs’, ’Results list’, ‘key term’, and ‘references profile’ features are tightly linked to the main search

• Current layout does not give an indication that ‘results list’, ‘key term’, and ‘reference profile’ are not separate content, but are about the search query done on the main page

• There is a hierarchy

• Some users did not understand that under the article list they would see

the results of the search done on the main page

DfR Layout

RefinedData Set

WholeData SetOf DfR

Narrowing Narrowing Down by Down by USERUSER

Diff. Views: Charts, GraphResults ListKeyterm Cloud

Layout Change

Current Version

Previous Version

Location indicates incorrect hierarchy

Appearance of being in the same frame indicates closer relationship

Previous version took advantage of proximity

Recommendation 2• Put Search Bar on a higher level

(Personal Comments: The mockup us too small….

Finding 3• The cognitive model of users and design of DfR are

divergent.

• Cognitive Model & Usability• Designer v. User

• “It’s like Google Scholar”• Instances of expectations v. reality using Data for Research

• Search• Key Terms

Cognitive Model: Defined• The way people think for the purposes of comprehension and

prediction • Significance: for people to understand how to use the Data for

Research tool, designers must understand the way they already think

• Usability: After purpose, there must be positive interaction in function for repeat use

Providing Search

“It’s like Google Scholar” (but it isn’t!)

Different Searching Patterns

RefinedData Set

#1

WholeData SetOf DfR

Search #1Search #1

RefinedData Set

#2

Search #2 Search #2 IF NOT “IF NOT “Clear AllClear All””

Search in Search in DfRDfR

WholeData Set

Other Database SearchOther Database Search

RefinedData Set

#1

RefinedData Set

#2

Search #2 Search #2

SearchSearch #1#1

search terms accumulate, rather than reset on new search (EXCEPT WHEN going directly to index)

Instance: Search aggregation

DfR’s Accumulative Search

Use “Selectors”

Keep Current Data Set

Click “Clear All”

• all produce DIFFERENT search results • punctuations have different treatment in

the DfR interface

Instance: Keyword searching + punctuations

Search for “politics”

Search for “republica

n”

Search for

“witch”

Delete“witch”

Search for “politics republican witch”

SAME Results, By Diff.

Methods

BUT, you cannot do

this!!

SAME Data Set,

Instance: Search + Blank Space

Recommendation 3• Search aggregation: • Clearer path for new search vs. adjusting current search (“New Search”

button)

• Keyword punctuation:• Explain difference between “key terms” and extracted “key terms”• Clarify how search results are accumulated (using all terms? listing by

articles and journals with higher frequency?)

Finding 4• A lack of DfR system feedback left searches with

unclear meanings.

• Search record is crucial to researchers - must keep track of information they gather

• Duplication of search in results view indicates system action to users

• Instances • After-search feedback• Facets/Selectors

Lack of system feedback before and after making a search - No tracking or matching of search terms No indication that anything has happened! - Selection criteria box is not prominent enough to notice

Facets/Selectors• New version: • Not intuitive that the

NAMES are links

• Further, cannot determine what they are doing to the results (start with selection ALL included?)

• Older version:• Check and “X” boxes

• Much clearer

• intuitive as to what is happening when “checking” (adding) or “X”-ing (subtracting) aspect of information

Recommendation 4• Search Feedback

• Additional feedback after search that indicates search has been performed

• Google Scholar model: redundancy WORKS!• Header renaming to “Search Results”

• Facet/Selector Appearance:• Reinstate the "X" function for all selectors (option to eliminate from

results or from search entirely)• Reinstate "checkmark" function for facets that have been eliminated or

are not included in the results.

Summary• For (finding 1)...for marketing purposes, a description of DfR is

NOT needed on the main page - it’s intuitive to users! • For (finding 2)...take advantage of X to Y. <-- not sure what to put

here. • For (finding 3)...similar cognitive models will lead to positive

interactions between the system and new users. • For (finding 4)...clear feedback leads to discernible meaning of

search results

Thanks !!Any Questions?