metasearching: the problem, promise, principles, possibilities & perils roy tennant california...

Metasearching: The Problem, Promise,

Principles, Possibilities & Perils

Roy TennantCalifornia Digital Library

NISO Metasearching Meeting, October 2003

Purpose

• To provide an overview, to set the stage

• To help you begin thinking both about the possibilities, as well as current issues and problems

• To hopefully leave you with the impression that despite the problems, it’s still very worth doing

NISO Metasearching Meeting, October 2003

The Problem

• Most users do not care where the information they need comes from, or who provides it…nor should they have to

• But our systems presently require them to know:– How to select one or more databases– How to get to them– How to use the unique search options for each

• How can we create systems that minimize what the user needs to know to get what they want?

RemoteCatalogs

LocalCatalog Local

WebSite

RemoteWebSites

VendorDbs

RemoteDigitalContent

LocalDigitalContent

RemoteCatalogs

LocalCatalog Local

WebSite

RemoteWebSites

VendorDbs

RemoteDigitalContent

LocalDigitalContent

RemoteCatalogs

LocalCatalog Local

WebSite

RemoteWebSites

VendorDbs

RemoteDigitalContent

LocalDigitalContent

RemoteCatalogs

LocalCatalog Local

WebSite

RemoteWebSites

VendorDbs

RemoteDigitalContent

LocalDigitalContent

RemoteCatalogs

LocalCatalog Local

WebSite

RemoteWebSites

VendorDbs

RemoteDigitalContent

LocalDigitalContent

NISO Metasearching Meeting, October 2003

The Promise

• The “Holy Grail” of resource discovery: one-stop shopping

• The simplification of a formerly complex activity (put the complexity in the back end, not the front)

• Allows the user to focus on evaluating results, not figuring out where to search

NISO Metasearching Meeting, October 2003

Before we talk specifics…

…let’s start with some principles…

NISO Metasearching Meeting, October 2003

Principles

Only librarians like to search, everyone else likes to find

NISO Metasearching Meeting, October 2003

Principles

All things being equal, one place to search is better than two or more.

NISO Metasearching Meeting, October 2003

Principles

“Good enough” is often just that.

NISO Metasearching Meeting, October 2003

Principles

Our ability to create effective one-stop searching is dependent on our ability to appropriately target user needs.

NISO Metasearching Meeting, October 2003

Principles

The size of the result set doesn’t matter as much as how the results are presented.

NISO Metasearching Meeting, October 2003

Principles

Services should be placed as close to the user as possible.

NISO Metasearching Meeting, October 2003

Possibilities: Software Solutions

• Special “metasearching” software optimized for:– Creating and managing target and resource

profiles– Managing authorization and authentication– Parsing search strings and result sets– Merging, deduping, and ranking results

• Stand-alone or integrated with an ILS• Many vendors (although a number are

based on the same underlying software)

NISO Metasearching Meeting, October 2003

Possibilities: Implementation Scenarios

• One search box for all resources (only for small libraries w/few targets)

• An interface that encourages/requires the user to select a subject or task group of DBs

• Tailored portals for particular needs and user communities or purposes (e.g., “a few good things”)

• Search boxes in other systems (e.g., courseware)

Slide from Greg Van Essen, Endeavor

NISO Metasearching Meeting, October 2003

Perils

• Software Provider Concerns• Database Provider Concerns• Library Concerns• User Concerns• Note: the following lists are not

comprehensive, but illustrative

NISO Metasearching Meeting, October 2003

Software Provider Concerns

• Access management• Search mapping• Unreliability of targets• Systems that don’t support an API (that

must be screen-scraped)• Inadequate result data for good:

– Deduping– Ranking

NISO Metasearching Meeting, October 2003

Database Provider Concerns

• Access control (robust authentication and authorization)

• Load• Inappropriate searches (searching

databases that don’t apply)• Branding and “unfair” deduping

NISO Metasearching Meeting, October 2003

Library Concerns

• Selecting the right system• Cost (both upfront and ongoing)• System design and implementation• System maintenance

– Ability to add new resources/targets– Ease of interface changes– Ease of upgrades

NISO Metasearching Meeting, October 2003

User Concerns

• What must I go through before hitting the search button?

• How difficult is it to review results?• Are results ranked by relevance? (that will be my

assumption)• Will I get buried? (too many sources, too many

results?)• Do I have methods to easily focus in on what I want?• Once I find what I want, can I get to the full-text with

a click?• Can I copy a citation and put it in my paper?

NISO Metasearching Meeting, October 2003

Possible Futures

• More standards for this type of interaction (i.e., support NISO’s efforts)

• Effective results ranking• Learning from other systems (e.g.,

RedLightGreen.com’s citation displays)• Dynamic selection of sources based on

user query

NISO Metasearching Meeting, October 2003

Recap

• “One-stop shopping” for resource discovery is the Holy Grail of librarianship

• Metasearch services offer the best opportunity we’ve yet seen to attain this goal

• It ain’t perfect, but it’s well worth pursuing• If we can present solid resources next to

Google hits — and win back our users in the process — what’s not to like?