This article was downloaded by: [York University Libraries]On: 17 October 2014, At: 04:25Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Web LibrarianshipPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/wjwl20

An Analysis of Undergraduate Students'Search Behaviors in an InformationLiteracy ClassHsin-Liang Chen aa School of Library and Information Science , IndianaUniversity–Indianapolis , Indianapolis, Indiana, USAPublished online: 02 Dec 2009.

To cite this article: Hsin-Liang Chen (2009) An Analysis of Undergraduate Students' SearchBehaviors in an Information Literacy Class, Journal of Web Librarianship, 3:4, 333-347, DOI:10.1080/19322900903328807

To link to this article: http://dx.doi.org/10.1080/19322900903328807

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Journal of Web Librarianship, 3:333–347, 2009Copyright © Taylor & Francis Group, LLCISSN: 1932-2909 print / 1932-2917 onlineDOI: 10.1080/19322900903328807

An Analysis of Undergraduate Students’ SearchBehaviors in an Information Literacy Class

HSIN-LIANG CHENSchool of Library and Information Science, Indiana University–Indianapolis,

Indianapolis, Indiana, USA

The purpose of this project was to investigate how students’ searchstrategies changed over the course of a semester-long informa-tion literacy class. Data collection included four different paperquestionnaires corresponding to course content in the spring 2008semester. Seventy-seven participants completed the questionnairesand course work in the class. Results showed that participants’ de-mographics did not have any impact on their selection of searchkeywords. While participants reported better search experienceswith more search keywords, they were not able to develop moresophisticated search keywords after receiving more instruction andsearch experience. The participants who did develop more sophisti-cated search keywords had better search experiences and a higherrate of satisfaction with their research results. Future studies shouldexamine what enabled these participants to develop more sophisti-cated search keywords.

KEYWORDS information literacy, information and computer lit-eracy, higher education, educational standards, search strategies,undergraduate students, academic libraries

Library instruction has been an important part of higher education in theUnited States. Over the decades, the rapid development of information tech-nology and its various applications at academic libraries have enriched libraryinstruction with the concept of “information literacy” (Wilder 2005). Since themid-1970s, the academic community has dedicated significant time and effort

Received 29 July 2009; accepted 31 August 2009.Address correspondence to Hsin-liang Chen, Assistant Professor, School of Library and

Information Science, Indiana University–Indianapolis, 755 West Michigan Street, UL 3100N,Indianapolis, IN 46202-5195. E-mail: chenhsin@iupui.edu

333

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

334 H.-L. Chen

to defining information literacy and its implementations at higher educationinstitutions (Webber and Johnston 2000). The Association of College andResearch Libraries (ACRL) approved the Information Literacy CompetencyStandards for Higher Education in 2000, and those standards were lateradopted by several higher education agencies (ACRL 2000).

However, the decades of effort in designing and implementing informa-tion literacy curricula at higher education institutions has not achieved thedesired results, according to a study conducted by the Educational TestingService (ETS) in 2006 (Katz 2007). The ETS study suggested that the major-ity of participating students did not have the expected skills in informationliteracy (ETS 2006; Katz 2007). Moreover, academic librarians debate boththe purpose and methods of conducting information literacy training. Stan-ley Wilder argued academic librarians are applying the “wrong solution tothe wrong problem” (2005, n.p.). He advocated for academic librarians to“use their expertise to deepen students’ understanding of the disciplines theystudy” and suggested librarians should not think of “students as informationseekers in need of information-seeking training.” On the other hand, SheilaWebber and Bill Johnston recommended that information seeking should bea priority area for information literacy curriculum design (2000).

Despite these differing opinions, many standards for information literacyaddressed the importance of “developing successful search strategies” andprovided the background for this project in an information literacy coursefor undergraduate students. The purposes of this project were to (1) inves-tigate students’ search strategies for assignments in an information literacyclass, (2) study whether students changed their search strategies during thesemester-long class, and (3) examine the relationships between students’search strategies and their learning experience.

INFORMATION LITERACY STANDARDS FOR HIGHER EDUCATION

The American Library Association (ALA) defines information literacy as a “setof abilities requiring individuals to ‘recognize when information is neededand have the ability to locate, evaluate, and use effectively the needed infor-mation’” (ALA 1989). The ACRL is one of the driving forces in the develop-ment of information literacy curriculum and instruction for higher educationinstitutions. In 2000, the ACRL Board approved Information Literacy Com-petency Standards for Higher Education. According to these standards, “aninformation literate individual is able to: determine the extent of informationneeded; access the needed information effectively and efficiently; evaluateinformation and its sources critically; incorporate selected information intoone’s knowledge base; use information effectively to accomplish a specificpurpose; understand the economic, legal, and social issues surrounding theuse of information, and access and use information ethically and legally”

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

An Analysis of Undergraduate Students’ Search Behaviors 335

(ACRL 2000, n.p.). Both of these definitions suggest that the ability to findinformation is a part of information literacy.

In the year following their release of the Standards, the ACRL Board ap-proved Objectives for Information Literacy Instruction: A Model Statement forAcademic Librarians, which defined the terminology of information literacyand identifies objectives, performance indicators, and outcomes of informa-tion literacy programs (ACRL 2001). This model was intended to be adoptedin a variety of instructional environments (e.g., 50-minute workshops orself-paced online library tutorials). In 2008, the ACRL approved InformationLiteracy Standards for Anthropology and Sociology Students. The standardscommittee of the ACRL received feedback from the American SociologicalAssociation and American Anthropological Association on the developmentof these standards.

The purposes of the standards are to: provide a common ground forfaculty to work with librarians in helping students become more criti-cal researchers and to offer faculty a basis for integrating the outcomesinto their courses; help librarians design the content of instruction forstudents and plan information literacy initiatives in anthropology andsociology; and make possible an evaluation of the information literacyskills of anthropology and sociology students by providing standards andcompetencies to assess. (ACRL 2008, n.p.)

These standards demonstrate the ACRL’s efforts and long-term commit-ment to the development of information literacy policies and programs. TheInformation Literacy Standards for Anthropology and Sociology Students alsodemonstrates potential development on subject-specific information literacystandards at the ACRL.

The need for such information literacy standards at higher educationinstitutions was reinforced by a study conducted by the ETS in 2006 (ETS2006). More than 3,600 students from 63 four-year colleges and universities,community colleges, and senior high schools participated in the ETS ICT(Information and Communication Technology) Literacy Assessment, whichconsisted of fourteen tasks (e.g., define information and access to informa-tion) and one long task (i.e., evaluate information). The terms “informationliteracy” and “ICT literacy” are both used by many organizations for issues inthis general area; therefore, both terms are used interchangeably in this arti-cle. According to the preliminary results of the ETS study, many participantsdid not have the ICT literacy skills expected for college-level work. However,the participants were not randomly selected at the 63 institutions, accordingto the ETS study. As a result of this study, the National ICT Literacy Pol-icy Council was established by the National Forum on Information Literacyfor the development of national ICT standards (Katz 2007). These nationalstandards are important keystones of curricula and provide educators and

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

336 H.-L. Chen

librarians with essential guidelines for course implementation. Notableamong these standards was the Competency Standard Two, PerformanceIndicator Two of the ACRL’s Information Literacy Competency Standards forHigher Education: “The information literate student constructs and imple-ments effectively designed search strategies.” This article investigates whetherstudents in a semester-long information literacy class are able to develop so-phisticated search keywords as effectively designed search strategies.

PEDAGOGICAL CONSIDERATION AND CURRICULUM DESIGN ONINFORMATION LITERACY

Even though educational standards are available, it can be complicated toimplement them into courses. Many educators, industry leaders, and policymakers agree that college students must have information literacy skills to bequality employees, but there are different views on course implementationand pedagogical design (Bruce 1997; Webber and Johnston 2000). Manyacademic libraries provide information literacy programs, but Wilder arguedthat “librarians should not assume that college students welcome their helpin doing research online” (2005, n.p.). Instead, he urged, “librarians shoulduse their intimate knowledge of the collections and the writing process aspracticed in the disciplines to teach apprentice readers and writers.” In otherwords, Wilder suggested that student engagement in an information literacyprogram is an important factor for pedagogical design, and the engagementmust be relevant to the course assignments.

Stephen Bowen (2005) also advocated the importance of student en-gagement within the context of the subject of study. Nancy Shapiro andKatherine McAdams (2006) reported positive feedback from college sopho-mores in an information literacy course where students chose their ownresearch topics, searched for primary resources, and presented their resultsvia posters. Choosing their own topics motivated students to learn indepen-dently and be engaged in course materials.

ASSESSMENT OF STUDENT PERFORMANCE

As educators and librarians have devoted much effort to the pedagogicaldesign of information literacy courses, they also have conducted studentassessment to validate their pedagogical design (Dunn 2002; Manuel 2002;Maughan 2001; Orme 2004; Rockman 2002). Various research strategies andmethods are used at different higher education institutions, which vary fromsingle-campus institutions to multiple-campus university systems. In somecases, a longitudinal research approach is necessary to collect sufficientdata to fully analyze the changes made by information literacy instruction(Rockman 2002). The California State University System has implemented a

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

An Analysis of Undergraduate Students’ Search Behaviors 337

longitudinal assessment of student information literacy skills on its 23 cam-puses since 2000 (Dunn 2002; Rockman 2002). This longitudinal assessmentis intended to identify students’ learning outcomes, to improve informationliteracy programs, and to document the development of information literacyprograms (Dunn 2002).

The concept of the California State University assessment brought upthe question of whether students were able to develop sophisticated searchkeywords in a semester-long information literacy course and if there wererelationships between students’ learning experiences in the course and thetypes of search keywords they used. Based on Shapiro and McAdams’s rec-ommendation, students in this study were free to choose research topicsfrom their own personal interests or intellectual curiosity. Students reportedtheir search keywords on three different questionnaires provided by theinvestigator. The results were analyzed to determine if any relationship ex-isted between the pedagogical design of information literacy instruction andstudents’ formation of search strategies.

Based on the literature review, several important factors were identifiedfor this project: students must be able to develop effective search strate-gies as evidence of information literacy required by the ACRL’s educationalstandards, the pedagogical design of course assignments should motivate stu-dents for independent learning and course engagement, and assessment isneeded of students’ learning outcomes and the improvement of informationliteracy programs. Therefore, the purposes of this project are to: (1) inves-tigate students’ search strategies for assignments in the semester-long class,(2) study whether students change their search strategies during the class,and (3) examine relationships between their search strategies and learningexperience. The following research questions are addressed:

1. Do students change their search strategies after learning how to use dif-ferent search tools and gaining more experience in the class?

2. Do students’ academic status, major, gender, or computer and libraryexperience have any effect on their strategies at any time?

3. Is there any relationship between students’ search process and the numberof search keywords?

4. Is there any relationship between students’ search process and the typesof search keywords?

THE STUDIED COURSE: ISLT 1111 INFORMATION USEAND STUDENT SUCCESS

The course examined in this project, ISLT 1111 (Information Use and StudentSuccess), is a one-credit undergraduate elective course offered in fall andspring semesters at the University of Missouri in Columbia. The class’s goals

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

338 H.-L. Chen

are to educate students to “frame meaningful questions, gain knowledge andskills to succeed academically, understand the structure and content of in-formation resources, evaluate information, and use information resources asgenuine learning tools” (University of Missouri 2009). All section instructorsof the class received training before the class from Dr. John Budd, who over-saw the ISLT 1111 course before the class, and the students in all sectionsreceived the same instructions. The instructors had bi-weekly meetings tokeep all sections in sync through the whole semester. Students learned howto use different paper-based and electronic information resources in differ-ent subject areas. They chose their own topics search for information fromthese resources and wrote reflection papers on their search experiences andresults. These papers were graded by the section instructors.

DATA COLLECTION

Data collection included four different paper questionnaires correspond-ing to course content in the spring 2008 semester (see the appendix). Thefirst questionnaire was administered at the beginning of the course to col-lect students’ demographic, academic, information technology, and libraryuse information. The second questionnaire asked students to list their ownsearch topics and the keywords they planned to use. This questionnairewas given to the students in the second or third week in the semester. Inthe third questionnaire, given during the fourth week, the students startedtheir searches and reported their actual search keywords and the levels ofsatisfaction with the search results. In the fourth questionnaire, during theeighth week, students had a second search assignment and reported boththeir actual search keywords and the levels of satisfaction about the searchresults. The students were encouraged to keep the same search topic for bothsearches.

PARTICIPANTS

More than 100 students initially registered for the seven sections of the ISLT1111 class at the beginning of the spring 2008 semester. Students who with-drew or failed to attend the class or did not finish the four questionnaireswere excluded from the project. The current research included data from77 students who completed the course work and questionnaires. The ma-jority of the participants were freshmen (N = 41, 53.2 percent). Thirty-eightparticipants were male, and 39 were female. More than one-third of the par-ticipants had science majors, and one-quarter had social science majors (seeTable 1).

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

An Analysis of Undergraduate Students’ Search Behaviors 339

TABLE 1 Participants’ Characteristics (N = 77)

Number ofCharacteristics Measurement participants %

Academic status Freshman 41 53.2Sophomore 12 15.6Junior 6 7.8

Senior 17 22.1Other 1 1.3

Major Humanities 10 13.0Social science 19 24.7Science 27 35.1Interdisciplinary/ 4 5.2

multiple majorsUndeclared 17 18.1

Gender Male 38 49.4Female 39 50.6

TABLE 2 Participants’ Computer and Library Experience (N = 77)

Characteristics Measurement N %

Computer ownership None 4 5.2Laptop 57 74.0

Desktop 3 3.9Both laptop 13 16.9

and desktopTime at campus Hours

computer lab 0 15 19.51–5 44 57.16–10 9 11.711–15 6 7.815–20 2 2.620+ 1 1.3

Time on campus Hourswireless network 0 17 22.1

1–5 12 15.66–10 18 23.411–15 12 15.615–20 10 13.020+ 8 10.4

Time at campus Hourslibraries 0 15 19.5

1–5 40 51.96–10 17 22.111–15 2 2.615–20 3 3.920+ 0 0

Time spent online for Hourspleasure and study 0 0 0

1–5 7 9.16–10 19 24.711–15 15 19.515–20 17 22.120+ 19 24.7

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

340 H.-L. Chen

TABLE 3 Participants’ Search Keywords (N = 77)

Pre-search keywords First search Second search

Total number Mean = 6.45 Mean = 5.48 Mean = 5.43of search keywords Max = 13 Max = 10 Max = 10

Min = 2 Min = 2 Min = 1SD = 2.60 SD = 2.22 SD = 2.34

General Mean = 2.25 Mean = 1.27 Mean = 1.65Max = 10 Max = 7 Max = 10Min = 0 Min = 0 Min = 0SD = 2.37 SD = 1.67 SD = 1.99

Unique Mean = 1.30 Mean = 0.94 Mean = 0.65Max = 6 Max = 10 Max = 9Min = 0 Min = 0 Min = 0SD = 1.70 SD = 1.83 SD = 1.41

General with refiner Mean = 2.55 Mean = 2.83 Mean = 2.70Max = 12 Max = 9 Max = 8Min = 0 Min = 0 Min = 0SD = 2.72 SD = 2.43 SD = 2.46

Unique with refiner Mean = 0.32 Mean = 0.45 Mean = 0.42Max = 6 Max = 6 Max = 8Min = 0 Min = 0 Min = 0SD = 1.01 SD = 1.17 SD = 1.30

Ninety percent of the participants owned a laptop computer, and morethan 16 percent owned both desktop and laptop computers. One-fifth ofthe participants had never used the computer labs, wireless networks, andlibraries on campus. More than 90 percent spent more than six hours onlinefor pleasure and study each week (see Table 2).

Participants’ search keywords from the pre-search, first search, andsecond search questionnaires were classified into four categories: general,unique, general with refiner, and unique with refiner (see Table 3). The fourcategories are based on P. G. B. Enser and C. G. McGregor’s 1992 study of2,722 image requests. Examples of the four categories are: U.S. presidents(general), Bill Clinton (unique), U.S. presidents after World War II (gen-eral with refiner), and Bill Clinton in New York City (unique with refiner).The four categories were used to examine the level of effectiveness of theparticipants’ search keywords. The effectiveness of general versus uniquecategories and “with refiner” versus “without refiner” categories was com-pared. With this categorization approach, the distribution of the participants’search keywords could be observed. On average, the participants generateda total of five to six search keywords in each of the three questionnaires.Most search keywords were in the general category, with the general withrefiner category being the second-most common.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

An Analysis of Undergraduate Students’ Search Behaviors 341

TABLE 4 Results of Paired T -test and Wilcoxon Signed Rank Test (N = 77)

Paired T -test Wilcoxon testVariables two-tailed two-tailed

Total number of search keywords (pre-test) t = 3.25, z = −3.17,Total number of search keywords (first search) p = 0.002 p = 0.002

Total number of search keywords (pre-test) t = 3.52, z = −3.34,Total number of search keywords (second search) p = 0.001 p = 0.001

Number of general search keywords (pre-test) t = 4.02, z = −3.65,Number of general search keywords (first search) p = 0.000 p = 0.000

Number of general search keywords (pre-test) t = 2.25, z = −2.11,Number of general search keywords (second search) p = 0.027 p = 0.035

Number of general search keywords (first search) t = –1.63, z = −1.63,Number of general search keywords (second search) p = 0.107 p = 0.103

Number of unique search keywords (pre-test) t = 1.71, z = −1.77,Number of unique search keywords (first search) p = 0.092 p = 0.076

Number of unique search keywords (pre-test) t = 3.25, z = −3.09,Number of unique search keywords (second search) p = 0.002 p = 0.002

Number of unique search keywords (first search) t = 1.84, z = −1.72,Number of unique search keywords (second search) p = 0.070 p = 0.086

Research question #1: Do students change their search strategies afterlearning how to use different search tools and gaining more experiencein the class?

Paired T-tests and Wilcoxon signed-rank tests were used to examinehow participants changed their search strategies after learning search tools(see Table 4). Several significant relationships indicated that the participantswere able to generate more search keywords between the pre-search ques-tionnaire and the first-search questionnaire, and between the pre-searchquestionnaire and the second-search questionnaire.

Table 5 shows two examples of participants’ queries over the course ofa semester. Participant A started searches mainly with personal names andused Boolean operators in the final search. Participant B used more specificwords in the first search compared to the pre-search queries and also startedwith using Boolean operators in the final search. The same results werefound within the general and unique categories of search keywords amongthe pre-, first, and second searches. There was no significant relationshipwithin the “general with refiner” and “unique with refiner” categories.

The participants demonstrated that they were not able to generate refin-ers, such as time and geographic location, to make their search more specific.This observation echoed Webber and Johnston’s (2000) recommendation

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

TA

BLE

5Tw

oPar

ticip

ants

’Sea

rch

Quer

ies

Res

earc

hm

otiv

atio

nPre

-sea

rch

quer

ies

Firs

tse

arch

quer

ies

Seco

nd

sear

chquer

ies

Par

tici

pan

tA

’sse

arch

top

ic:C

SIIlik

eth

eTV

show

,CSI

.I

•Hora

shio

sp.H

ora

tioCan

e•C

SI•C

SIw

atch

both

CSI

Nev

ada

and

•CSI

(CB

S.CO

M/C

SI)

•Crim

esc

ene

inve

stig

atio

n•C

rim

ela

bs

CSI

Mia

mi.

•Gil

Griss

iom

(Will

iam

Pet

erse

n)

•Cat

her

ine

Will

ow

s•C

rim

esc

enes

•Cat

her

ine

Will

ow

s•S

cua

Spad

e•C

rim

era

tes

inci

ties

•TV

show

s•H

ora

cio

sp.H

ora

tioCan

e•C

rim

esan

dci

ties

•Hai

rygu

y•G

ilG

riss

iom

•Rat

esan

dci

ties,

crim

es•S

arah

•Gre

gSa

nder

s•R

ates

and

citie

san

dcr

imes

•Crim

era

tes

or

citie

s•P

opula

rci

ties

and

crim

e

Par

tici

pan

tB

’sse

arch

top

ic:h

un

tin

gIt

isone

ofm

y•H

untin

gse

asons

•Huntin

gse

asons

•Huntin

gan

dfish

ing

favo

rite

activ

ities

•Typ

esofgu

ns

•Rifl

es•H

untin

gor

fish

ing

•Bow

huntin

g•B

ow

s•D

eer

huntin

g•R

ule

s/re

gula

tions

for

regi

ons

•Dee

rhuntin

g•H

untin

gse

asons

•Gea

r•T

urk

eyhuntin

g•B

ow

huntin

g•S

easo

ns

•Sco

res

342

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

An Analysis of Undergraduate Students’ Search Behaviors 343

that college students need a fairly good knowledge of their field of studybefore they can develop meaningful refiners. These results indicate someimportant pedagogical considerations for the information literacy course: Dostudents learn from instruction about developing more sophisticated searchkeywords? Do students need to develop more sophisticated search keywords?Should librarians and instructors of the information literacy courses acceptthat students do not develop more sophisticated search keywords, even afterreceiving instruction? These questions should be addressed in future studies.

Research question #2: Do students’ academic status, major, gender, orcomputer and library experience have any impact on their strategies atany time?

Repeated ANOVA analysis was used to examine these variables versusthe students’ search keywords types. Interestingly, there were no significantcorrelations. It seemed that participants’ characteristics did not influence theirsearch behavior. This may indicate that Web-generation students use thesame search techniques when searching for information on both the Internetand databases. Future large-scale studies may confirm this observation.

Research question #3: Is there any relationship between students’ searchprocesses and the number of search keywords?

The students answered three questions in questionnaires #3 and #4regarding their search process. The three questions addressed their searchexperience, their satisfaction about the search results, and the completion oftheir search. The students used a seven-point scale (1 indicating not at allsatisfied and 7 meaning very much satisfied) to indicate their satisfaction level(see Table 6). Linear regression analyses were used, with four dependentvariables: search experience, satisfaction, completion, and overall experience(the mean score of the first three dependent variables). The independent

TABLE 6 Participants’ Satisfaction with Search Results (N = 77)

Variable Mean SD

First search:Number of search keywords 5.48 2.22Search experience 5.56 2.34Satisfaction 5.49 1.18Completion 6.00 1.29

Second search:Number of search keywords 5.43 1.09Search experience 5.35 1.30Satisfaction 5.45 1.37Completion 6.06 1.13

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

344 H.-L. Chen

variable was the number of search keywords. This is an explorative study,so all results where p ≤ 0.10 were reported.

The participants reported positive search experiences with more searchkeywords (F = 4.73, p = 0.03), more satisfaction with more search key-words (F = 3.03, p = 0.09), and better overall experiences with more searchkeywords (F = 3.55, p = 0.06) for their first search. However, only onesignificant relationship was found between the completion of the search andthe number of keywords (F = 2.77, p = 0.10) in the second search.

This indicated that the participants may have reached a plateau onlearning how to search for information after the first search assignment, asstudent performance stalled after this point. This plateau is an importantresearch area for future studies.

Research question #4: Is there any relationship between students’ searchprocess and the types of search keywords?

Finally, the relationships between the students’ search experiences, sat-isfaction about the search results, completion of the searches, and the fourtypes of search keywords were examined. Linear regression analyses wereused with the dependent variables—search experience, satisfaction, comple-tion, and overall experience (the mean score of the first three dependentvariables)—to review relationships. The independent variable (the types ofsearch keywords) also examined possible relationships. The number of par-ticipants’ general search keywords with refiners was the only independentvariable that had significant relationships with three dependent variables:search experience (F = 5.00, p = 0.03, first search), search experience (F =4.08, p = 0.05, second search), and satisfaction (F = 4.00, p = 0.05, secondsearch). This indicated that once participants successfully developed moresophisticated search keywords, they had better search experiences and weresatisfied by the search results.

This report identifies several important areas for information literacyinstruction: students must be able to develop effective search strategies asevidence of information literacy required by the ACRL’s educational stan-dards, pedagogical design of course assignments should motivate studentsfor independent learning and course engagement, assessment is needed onstudents’ learning outcome, and information literacy programs must showimprovement.

Results showed that participants’ demographics or computer experi-ence did not have any impact on their selection of search keywords. Eventhough the participants were able to generate more search keywords, theywere not able to develop more sophisticated search keywords after receivingmore instructions and search experience. The stalled student performanceneeds further investigation. Participants who developed more sophisticatedsearch keywords (general search keyword with refiner) had better search

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

An Analysis of Undergraduate Students’ Search Behaviors 345

experiences and a higher rate of satisfaction with their research results. Fu-ture studies should examine what enabled these participants to develop moresophisticated search keywords. In the meantime, the participants reportedbetter search experiences with more search keywords. These findings raisetwo important questions for future studies.

First, alternative research methods are needed to confirm the data fromparticipants’ self-evaluation in this study. Future studies may include stu-dents’ final grade as a factor for data analysis. Second, as one educationalgoal of the information literacy course is to help students access the neededinformation effectively and efficiently, the meanings of effectively and effi-ciently need to be discussed between instructors and students. This studyused four types of search keywords to examine the development of theparticipants’ search keywords. Future studies should consider whether thisapproach is sufficient for determining “effectively” and “efficiently.” Thesesuggestions will reinforce education standards for information literacy andenhance the pedagogical design of information literacy courses.

ACKNOWLEDGMENTS

The author would like to thank Dr. John Budd and the ISLT 1111 instructorsfor their assistance as well as the students for their participations in theproject.

REFERENCES

American Library Association (ALA). 1989. Presidential Committee on InformationLiteracy. Final report. http://www.acrl.org/ala/mgrps/divs/acrl/publications/whitepapers/presidential.cfm (accessed May 8, 2008).

Association of College and Research Libraries (ACRL). 2000. Information liter-acy competency for higher education. http://www.acrl.org/ala/mgrps/divs/acrl/standards/informationliteracycompetency.cfm (accessed May 8, 2008).

Association of College and Research Libraries (ACRL). 2001. Objectives for informa-tion literacy instruction: A model statement for academic librarians. http://www.acrl.org/ala/mgrps/divs/acrl/standards/objectivesinformation.cfm (accessed May 8,2008).

Association of College and Research Libraries (ACRL). 2008. Information liter-acy standards for anthropology and sociology students. http://www.acrl.org/ala/mgrps/divs/acrl/standards/anthro soc standards.cfm (accessed December15, 2008).

Bowen, Stephen. 2005. Engaged learning: Are we all on the same page? Peer Review7(2). http://www.aacu.org/peerreview/pr-wi05/pr-wi05feature1.cfm (accessedDecember 15, 2008).

Bruce, Christine. 1997. The seven faces of information literacy. Adelaide, Australia:Auslib Press.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

346 H.-L. Chen

Dunn, Kathleen. 2002. Assessing information literacy skills in California State Uni-versity: A progress report. Journal of Academic Librarianship 28(1): 26–35.

Educational Testing Service (ETS). 2006. ICT literacy assessment preliminary findings.http://www.ets.org/Media/Products/ICT Literacy/pdf/2006 Preliminary Findings.pdf (accessed January 5, 2009).

Enser, P. G. B., and C. G. McGregor. 1992. Analysis of visual information retrievalqueries. Report on Project G16412 to the British Library Research and Develop-ment Department. London: British Library.

Katz, Irvin R. 2007. ETS research finds college students fall short in demonstrat-ing ICT literacy: National Policy Council to create national standards. Col-lege and Research Libraries News 68(1). http://www.ala.org/ala/acrl/acrlpubs/crlnews/backissues2007/january07/ets.cfm (accessed June 25, 2008).

Manuel, Kate. 2002. Teaching information literacy to Generation Y. Journal of LibraryAdministration 36(1/2): 195–217.

Maughan, Patricia D. 2001. Assessing information literacy among undergraduates: Adiscussion of the literature and the University of California–Berkeley assessmentexperience. College and Research Libraries 62(1): 71–85.

Orme, William A. 2004. A study of the residual impact of the Texas InformationLiteracy Tutorial on the information-seeking ability of first-year college students.College and Research Libraries 65(3): 205–215.

Rockman, Ilene F. 2002. Strengthening connections between information literacy,general education, and assessment efforts. Library Trends 51(2): 185–198.

Shapiro, Nancy, and Katherine McAdams. 2006. Discovery projects: Contextualizedresearch experiences for college sophomores. In Student Engagement and In-formation Literacy, ed. C. Gibson. Chicago: ALA.

University of Missouri. School of Information Science and Learning Technolo-gies, University of Missouri. http://education.missouri.edu/SISLT/index.php (ac-cessed August 30, 2009).

Webber, Sheila, and Bill Johnston. 2000. Conceptions of information literacy: Newperspectives and implications. Journal of Information Science 26(6): 381–397.

Wilder, Stanley. 2005. Information literacy makes all the wrong assumptions.The Chronicle of Higher Education. http://chronicle.com/weekly/v51/i18/18b01301.htm (accessed May 8, 2008).

APPENDIX

Questionnaire #1 (Demographic Data)

Class section: Number:

1. Academic status: Freshman Sophomore Junior Senior Other2. Academic major: Humanities Social sciences Science

Interdisciplinary/multiple majors Undeclared3. Gender: Male Female4. Do you own one or more computers?

Yes, Laptop Desktop BothNo

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4

An Analysis of Undergraduate Students’ Search Behaviors 347

5. How many hours do you use MU’s computer labs each week during thesemester?

0 1–5 6–10 11–15 15–20 more than 206. How many hours do you use MU’s wireless network each week during

the semester?0 1–5 6–10 11–15 15–20 more than 20

7. How many hours do you stay at MU’s libraries each week during thesemester?

0 1–5 6–10 11–15 15–20 more than 208. How many hours do you go online (for school or fun) each week during

the semester?0 1–5 6–10 11–15 15–20 more than 20

9. When you go to MU’s libraries, what do you do there? Please mark topthree items from the following list.

StudyingDoing homeworkSeeking help from librariansReading library materialsKilling timeHanging out with friendsOther, please specify

Questionnaire #2 (Description of Search Topic)

Class section: Number:Search topic:Why you are interested in this topic?Search terms you plan to use (please fill out as many as you need):

ISLT 1111—Questionnaire #3 and #4

Class section: Number:Search topic:Search terms you actually used (please fill out as many as you need):

1. How was your search experience?Bad 1 2 3 4 5 6 7 Excellent

2. Are you satisfied with your search results?Not at all 1 2 3 4 5 6 7 Very much

3. Do you think you finished the assignment successfully?Not at all 1 2 3 4 5 6 7 Very much

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

4:25

17

Oct

ober

201

4