E-Valuating Virtual Viewpoints: User, Non-User, and Librarians
Perspectives on Live Chat-Based
Reference
Marie L. Radford, Ph.D.Associate Professor, Rutgers, The State University of New Jersey
Lynn Silipigni Connaway, Ph.D.Senior Research Scientist, OCLC
Panel: E-Valuating E-Reference: Transforming Digital Reference through Research and Evaluating
ASIST Annual MeetingOctober 24-29 2008Columbus, OH
$1,103,572 project funded by Institute of Museum and Library ServicesRutgers, The State University of New Jersey OCLC, Online Computer Library Center, Inc.
Four phases:Focus group interviewsAnalysis of 850 QuestionPoint live chat transcriptsOnline surveys
176 VRS librarians184 VRS non-users
137 VRS users Telephone interviews
283 Total
Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives
Critical Incident Technique
Flanagan, 1954 Qualitative technique Focuses on most memorable
event/experience Allows categories or themes to emerge
rather than be imposed
Online Survey CI Questions
Librarians & UsersThink about one experience in which you felt a
chat reference encounter achieved (or did not achieve) a positive result
Non-usersThink about one experience in which you felt you
achieved (did not achieve) a positive result after seeking library reference services in any format
Interpersonal Communication Analysis: Results
Relational FacilitatorsInterpersonal aspects of chat conversation that
have a positive impact on librarian-client interaction & that enhance communication.
Relational BarriersInterpersonal aspects of chat conversation that
have a negative impact on librarian-client interaction & that impede communication.
Relational Theory & Approach to Interpersonal Communication
Every message has dual dimensions – both content & relational
(Watzlawick, Beavin, & Jackson, 1967)
Librarians: Positive Result(CI N=142)
Number %
Both Relational & 85 60% Content Primarily Content 54 38% Primarily Relational 3 2%
Librarians: Positive Result(CI N=142)
Relational Themes*
Number %
Attitude 69 49%
Relationship quality 33 23% Familiarity 3 2%
*The percentages do not total to 100% because each CI can be coded into more than one theme
Librarians: Positive Result(CI N=142)
Content Themes* Number %
Providing information 12085%
Providing instruction 49 35% Demonstrating knowledge 14 10% Convenience/multi- 10 7% tasking/ time or money
saving*The percentages do not total to 100% because each CI can be coded into more than one theme
Librarians: Negative Result (CI N=124)
Number %
Primarily Relational 53 43% Primarily Content 40 32% Both Relational & 31 25% Content
Librarians: Negative Result (CI N=124)
Relational Themes*
Number %
Attitude 67 54% Relationship quality 28 23% Impact of technology 7 6% Approachability 1 1%
*The percentages do not total to 100% because each CI can be coded into more than one theme
Librarians: Negative Result (CI N=124)
Content Themes*
Number %
Lack of information 64 52% Lack of knowledge 15 12% Task unreasonable 1 1%
*The percentages do not total to 100% because each CI can be coded into more than one theme
Users: Positive Result(CI N=129)
Number %
Primarily Content 79 61% Both Relational & 33
26% Content Primarily Relational 17
13%
Users: Positive Result(CI N=129)
Content Themes* Number %
Providing information 91 71% Convenience/multi- 36 28%
tasking/time saving/ money saving Providing instruction 14 11%
Demonstrating knowledge 7 5%
*The percentages do not total to 100% because each CI can be coded into more than one theme
Users: Positive Result(CI N=129)
Relational Themes* Number %
Attitude 36 28%
Relationship quality 21 16% Impact of technology 1 1%
*The percentages do not total to 100% because each CI can be coded into more than one theme
Users: Negative Result(CI N=68)
Number %
Primarily Content 46 68%
Primarily Relational 15 22%
Both Relational & 7 10% Content
Users: Negative Result(CI N=68)
Content Themes* Number %
Lack of information 48 71% Lack of knowledge 8 12%
*The percentages do not total to 100% because each CI can be coded into more than one theme
Users: Negative Result(CI N=68)
Relational Themes* Number
%
Relationship quality 17 25% Attitude 13
19% Approachability 1 1% Impact of Technology 1
1%*The percentages do not total to 100% because each CI can be coded into more than one theme
Non-users: Positive Result (CI N=154)
Number %
Primarily Content 79 51% Both Relational & 48
31% Content Primarily Relational 27
18%
Non-users: Positive Result (CI N=154)
Content Themes* Number %
Providing information 75 49%
Providing instruction 35 23% Demonstrating knowledge 21 14% Convenience/multi- 18 12%
tasking/time saving/ money saving*The percentages do not total to 100% because each
CI can be coded into more than one theme
Non-users: Positive Result (CI N=154)
Relational Themes* Number %
Attitude 51 33% Impact of FtF assisting 32 21%
relationship development Relationship quality 25 16% Impact of phone/Email 5 3%
assisting information seeking process Approachability 4 3% Familiarity 1 1%
*The percentages do not total to 100% because each CI can be coded into more than one theme
Non-users: Negative Result (CI N=100)
Number %
Primarily Content 52 52%
Primarily Relational 33 33%
Both Relational & 15 15%
Content
Non-users: Negative Result (CI N=100)
Content Themes* Number %
Information 60 60% Lack of knowledge 24 24% Instruction 9 9% Task unreasonable 4 4%
*The percentages do not total to 100% because each CI can be coded into more than one theme
Non-users: Negative Result (CI N=100)
Relational Themes* Number
%
Attitude 47 47%
Relationship quality 24 24% Approachability 3 3% Impact of technology 2 2%
*The percentages do not total to 100% because each CI can be coded into more than one theme
Implications: Librarians
Value Delivery of accurate answers/
information Polite, interested users
Find rude or impatient users disruptive to chat success
Implications: Users & Non-Users
Value Accuracy of answers/information
Delivery of specific content Knowledge of sources & systems Positive attitude Good communication skills
Younger VRS users Impatient & want info delivered quickly - no
fuss Not as concerned as librarians w/ instruction
Recommendations
Provide Specific info Variety of formats Friendly & courteous service Marketing to non-users
User education needed for more realistic expectations
Do not force instruction unless wanted
Future Directions
Online survey results informed 283 telephone interviews Collected more critical incidents Analysis in progress
Special Thanks
Rutgers University & OCLC Grant Project TeamProject Managers:
Jocelyn DeAngelis Williams Timothy J. Dickey
Research Assistants:Patrick A. ConferDavid DragosJannica HeinstromVickie KozoMary Anne ReillyLisa Rose-WilesSusanna Sabolsci-BorosAndrea SimzakJulie StrangeJanet Torsney
End Notes
This is an updated version of a presentation given at ALISE 2008
This is one of the outcomes from the projectSeeking Synchronicity: Evaluating Virtual Reference Services from
User, Non-User, and Librarian Perspectives
Funded by IMLS, Rutgers University, & OCLC Online Computer Library Center, Inc.
Slides available at project web site: http://www.oclc.org/research/projects/synchronicity/