federated search : context, capability, & consequence

Federated Search: Context, Capability, & Consequence

• Brenda Reeb, University of Rochester

• John D’Ignazio, Syracuse University School of Information Studies

• John Law, ProQuest

LITA National Forum 2005, San Jose, California

Federated Search – Reeb/D’Ignazio/LawLITA Forum 2005

2

Today

• What we did and why we did it – Brenda• How we did it – John D’Ignazio• What we found – John Law• Capability and Consequences – John

Law• Questions – Audience!


3

Reasons why• Animal, mineral, or

vegetable?• Bridge from

federated search results to native resource – or not?

• Find information vs. analyze information – transparent technology and resources Context

How we did it


5

Inserting a new technology into a complex system

Questions multiply!• How does it work with mass of database offerings?• How does it relate to the existing web implementation?• How do you value the additional system responses? • How does it relate to Google?

How to answer?• Keep building…• Or,

Take it to the user?


6

Inspiration: A Formative User Study

• comfort with usability testing to evaluate library web design

• contextual inquiry: put tech development in context of user’s information tasks

• compare user work practice in library’s web environment1. w/ federated search2. without


7

Elements of Contextual Inquiry

An interview technique:– Master-apprentice model BUT

user is master!– Think aloud protocol…or in our case Think

After“Tell me about the paper you last wrote.” User then dives away from description toward concrete actions taken to reach goal

– Establish easy manner to elicit practices and user relationships with technology


8

Participants• University of Rochester

Brenda Reeb -- usability program coordinator

Nancy Foster -- anthropologist/researcher

Nora Dimmock -- multimedia center

• ProQuestJohn Law -- director, product

management

• EndeavorMike Visser -- product manager

• Association for Research Libraries - ARL

John D’Ignazio -- usability consultant

• Subjects - 9– 3 first year

– 6 upper years

– 3 social sciences

– 3 humanities

– 2 sciences

– 1 undeclared

• Solicited from library usability email roster – Did you write a paper last

semester?

• Received $25 gift certificate


9

Test SetupModerator/Apprentice- sits next to subject- guides, prompts subject through

task according to script What are you doing now?,Why

did you do X instead of Y?, What would you like to do right now? Show me.

Have you ever used federated search…let’s try.

Note-taker - records interaction,- concentrates on user

attitudes/approach

Subject and subject’s actions simultaneously recorded


10

Morae Software Suite• Event recording software, made

by TechSmith

• Flexible, easy, low cost

• 3 modules:– Morae recorder: capture audio,

video and user actions– Morae remote viewer: observe

a test from a remote location– Morae manager: analyze the

results and create visual presentations


11

Room for Improvement• Test Subjects:

– not chosen randomly; one third were library employees

• Moderators: – hard to play apprentice while guiding progress

• Test Setup: – not native work environment– recreating past actions may lead to idealization or omissions– forced switchover to federated search

• Fitness of Method– adopted from HW/SW development community; libraries may need

different formative/evaluative methods


12

What we found

• Observations• New capabilities for federated search • Consequences• Conclusions


13

Observations

• Students’ search strategies are well considered (albeit more literal than informed)

• Users value their time highly and are unwilling to “fish around” long for appropriate resources


14

Video snippet #1


15

Observations (con’t)

• Everybody wants a summary/abstract– Give me enough information to decide

which items are worth pursuing– Where are my search terms in my results?

• Nobody likes complex paths to full text

• Users expect that search results are presented in order of most relevant


16

Video snippet #2


17

Observations (con’t)

• “Search all” usually not appealing in searches of library resources

• Little known distinction between different databases

• Frequently accessed target e-resources via course pages

• Context adds to value perception: “Faculty-recommended resources”


18

Video snippet #3


19

Even more observations

• Trusted resources, whether appropriate for the research project or not, win over a “search all” federated view of library resources

• Resources discovered “in context” are considered as trusted and credible (otherwise, little perceived distinction between resources)

• Fluid interaction among tools and resources (e.g. Google, Amazon, NoodleBib, etc.)


20

New capabilities for Federated Search

• Provide evaluative information in search results. Why pick this one?– abstract/summary, subject term highlight, etc.

• Provide direct access to full text – no confusing intermediate pages (e.g. link resolver

page, citation page, etc.– predictors of full text availability (i.e. don’t show a

full text link unless full text is known to be available


21

New capabilities (con’t)

• Conundrum: – Based on current technology, “Search all”

federated searching of premium scholarly resources falls short of satisfying user needs

– Web search engines’ superiority in relevancy sorting of search results is found more satisfactory although lacking in scholarly integrity


22

Consequences

• Students return to past trusted resources. Is this wrong?– This is compensatory behavior:

• When resources are not available in the context of their work

• When it’s unclear where to go for quick, reliable answers to research questions

• Student focus is course-centric– The “home page” is the course page

– Learning management systems and student personalized portals will become “information central” as these solutions evolve

– Library resources need to be available in these environments• Selective content presented in context


23

Conclusions• Context adds value and trust

– Resources associated with a course were perceived as superior

• Clearly, students are engaged in their research– an appropriate combination of resources (and tools) need to

be available in-context

• Users differentiate between authoritative scholarly resources and general web search results, however: – the latter is easier to deal with since it provides a

predictable quality of search results, while – the former is inconsistently presented and difficult to

discover in a manner consistent with user needs, expectations and current practices


24

From the user’s perspective:

How do finch hormones impact birdsong?

Finch HormonesLab ReportDecember 2004

federated search : context, capability, & consequence

Documents