[ieee 2009 international conference on engineering education (iceed) - kuala lumpur, malaysia...

2009 International Conference on Engineering Education (lCEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

Information Literacy of Engineering Students: A Case Study

Rosmah Alii & Norihan Abu-Hassan2

Department of Science, College of Science and Technology Universiti Teknologi Malaysia International Campus

54100 Kuala Lumpur, Malaysia [email protected], [email protected]

Abstract- This study aims to identify the strengths and

weaknesses of engineering students' information literacy knowledge. It also identifies the common misconceptions

and mistakes students make. The respondents in the pilot

study have undergone at least three semesters at a

Malaysian college. Data are collected using a survey

instrument adapted from Mittermeyer and Quirion and

analyzed using SPSS. The results of this study show that the respondents seriously lack information skills especially

in evaluating internet information, identifying the most

efficient search strategy and using information ethically.

Keywords-assessment; information literacy; engineering; misconception; mistakes

I. INTRODUCTION

Information literacy is one of the critical digital-age literacy for higher education. Students need to be information literate to cope with the challenges in further studies and future profession. Engineering is a field that is fast developing as a result of scientific and technological advancement. Thus, engineering students need to be independent learners who can deal with vast amount of information. To be able to do this, students must have the appropriate information literacy skills.

An important step in information literacy development is to obtain data on students' abilities. These data will indicate areas of improvement and the most appropriate training and services required. Data for this study are obtained using the test adapted from Mittermeyer and Quirion [1]. Information literacy competency standards for higher education serve as guidelines to assess students' information literacy skills. Examples of such standards are the Association of College and Research Libraries (ACRL) standards [2] and the International Federation of Library Associations (lFLA) guidelines on information literacy [3].

II. DEFINITION OF INFORMATION LITERACY

UNESCO describes information literacy as part of the wide-ranging information and communication technology skills comprising knowledge, skills and attitude dimensions. The technical skills dimension includes information literacy. UNESCO [4] further defines information literacy as the ability

Identify applicable sponsorls here. If no sponsors, delete this text box. (sponsors)

978-1-4244-4844-9/09/$25.00 ©2009 IEEE 143

Mohd YusofMd Daud3

Department of Mechanical Engineering College of Science and Technology

Universiti Teknologi Malaysia International Campus 54100 Kuala Lumpur, Malaysia

[email protected]

to recognize information needs and use ICT features and applications to access, retrieve, store, manage, integrate, evaluate, create and communicate information effectively. To be information literate, an individual should be able to:

• determine the nature and extent of information required

• access and retrieve information effectively and efficiently

• evaluate information and its sources with regard to its authority, accuracy, objectivity and currency.

• create, adapt or apply information

• use information for specific purposes

• communicate information properly within its context

• understand the ethical and legal use of information

The defmition and abilities mentioned above provide a solid foundation upon which this study is developed and constructed.

III. OBJECTIVE AND SCOPE OF THE STUDY

The objective of this study is to determine which specific skills students are weak at, and to recommend an appropriate intervention program. Subjects of the study are second and third year diploma of engineering students who have undergone at least three semesters of education at a Malaysian college. These students will eventually continue their studies or join the workforce upon graduation.

IV. LITERATURE REVIEW

A. Overview of Information Literacy Development

Research and publications in information literacy have been active among librarians and information professionals since early 1970s in many industrialized and developing countries. In the United States, the National Forum on Information Literacy was established in 1989, followed by the Institute for Information Literacy in 1998. Information literacy was included in the United States education plan at various levels since 2000. Developments in information literacy have


also been reported in other parts of the world such as Australia, Canada, New Zealand, China, Japan, South Africa, Singapore, and Malaysia.

Similarly, in Europe, information literacy initiatives and programmes evolved from the work of academic librarians since 1970s. The growth of digital information, the focus on lifelong learning and the need for highly skilled workers have highlighted the need for information-related competencies. These issues were discussed in several reports by the Organisation for Economic Co-operation and Development (OECD) such as the Knowledge-based Economy, Learning to Bridge the Digital Gap, and Educational Policy Analysis 2001 [5].

The Ministry of Higher Education (MOHE) of Malaysia is committed to produce ICT and information literate graduates as part of the country's human capital development plan. The National Higher Education Action Plan 2007 -2010 stresses the importance of ICT in two of the strategic thrusts. The second strategic thrust: "Improving the Quality of Teaching and Learning" emphasizes the effective use of ICT in content delivery and the acquisition of 21 Ol_century skills. The importance of ICT skills in human capital development through self-directed learning using the electronic media, the Internet and e-learning is highlighted in the sixth strategic thrust: "Enculturation of Lifelong Learning" [6].

B. Approaches to Iriformation Literacy Education

Historically, information literacy education was initiated by academic librarians and they used several approaches to develop information literacy among students. Common pedagogical approaches are:

• Stand-alone courses

• Computer-assisted tutorials

• Course-integrated instruction

Stand-alone courses can be in the form of compulsory orientation information skills module for fIrst-year students, optional information skills courses for fInal-year students undertaking a research project, and compulsory credit-based courses. These courses are often conducted by librarians. Researchers that advocate the teaching of information literacy as an independent discipline include Webber and Johnston [7].

Computer-assisted courses such as interactive Webbased tutorials, virtual tours and OPAC tutorials are often used as a training package to develop information skills in distance learning programmes. These courses encourage the selflearning principle for individuals and specifIc user groups.

Course-integrated instruction has gained popularity and is supported by many authors such as Rafste [8] and Town [9]. An example of a subject specifIc guide is an information retrieval guide for medicine developed by Leicester University Library. Such course-integrated information literacy programmes often use problem-based learning approach [10], [11].

144

C. Assessment Methods For Information Literacy

Assessment is vital in an academic curriculum because "students learn what assessment assesses" (Laurillard, 2002). Summative assessments are performed for a variety of reasons such as to judge a learner's ability and readiness to proceed, graduate and for selection purposes. Formative assessments are used to give feedbacks on performance to improve learning. Diagnostic assessments identity students' existing skills and learning difficulties. This information can guide course coordinators and academic advisors in designing an appropriate intervention programme.

Assessment methods should match the purpose of assessment while taking into account the crucial issues of reliability, validity and feasibility. Different types of information literacy assessments include one-off multiple choice tests, essay assignments, direct observations of students actually performing an information search, group projects, and portfolio assessment of information searching skills [13]. Each of these assessment methods has its strengths and weaknesses. For instance, multiple choice questions are easy to mark, can test a wide range of knowledge quickly, and is highly reliable, but does not have high validity to indicate the learner's actual information skills in solving real life problems [14].

There are numerous online information literacy software developed by libraries, colleges and universities. An example is SAILS (Standardized Assessment of Information Literacy Skills) developed by Kent State University. The test items align with the ACRL Information Literacy Competency Standards for Higher Education. Another example is the Information Skills Survey (ISS) developed by the Council of Australian University Librarians [15]. The National Higher Education ICT Initiative collaborated with Educational Testing Service (ETS) to develop iSkills [16]. Mittermeyer and Quirion developed a survey instrument consisting of 20 multiple-choice questions based on the ACRL standards for information literacy. The results of these tests are used to identify the strengths and weaknesses of college students' information literacy skills.

v. METHODOLOGY

This study uses quantitative approach to explore engineering students' information literacy skills. Purposive sampling for the pilot study was used to select 49 second and third-year engineering students at the diploma level. These students were chosen for the belief that they must have already acquired the necessary information skills after undergoing at least three semesters of study. During the fIrst week of study at the college, students attend an inductive library talk and participate in a tour, which normally takes about two hours. However, students may request for library ICT skills courses anytime during their study programme to enhance their information skills.

Students in the study were given a take-home multiplechoice test. The test items were modifIed to make them specifIcally relevant to engineering students. The content validity of questionnaire was evaluated by a Malaysian information literacy specialist with more than twenty years

2009 International Conference on Engineering Education (ICEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

working experience in the field. In the questionnaire, information literacy skills are categorized into five broad areas consisting of several specific skill questions in each area. The five areas are: Concept Identification, Search Strategy, Document Types, Search Tools, and Use of Results.

VI. ANALYSIS OF DATA

The following data based on the questionnaire responses are analyzed using descriptive statistics:

i) Percentage of students who correctly answered each question

ii) Percentage of students who selected each of the incorrect answer to identify common mistakes and misconceptions within each skill area

Data are analyzed using SPSS version 10.0.

VII. RESULTS AND INTERPRET A nON

The results according to the broad information literacy areas and the relevant questions are as given below.

Area I: Concept Identification

40 35 30 25 20 15 10

5 o

.tt=.

,c:::. =1-- l-

I-- l-I-- I-- '--

Q4 Q8 Q13 Significant terms

o Percentage of students who correctly answered the question

For question 4, only 18.4% of the respondents selected the best answer, which includes the three main concepts in the topic. 61.2% of the respondents included the non-significant term "effect", which reduces the number of search results obtained.

For question 8, most respondents do not appear to be able to distinguish between significant and non-significant terms by including restrictive terms such as impact and depletion.

Over a third of the respondents (36.7%) selected the best answer to question 13. They seem to be able to recognize words with similar meaning. "Protective measures" and "measures to decrease damage" have the same meaning. However, most respondents (48.9%) do not seem to be able to distance themselves from the original terms in the topic.

The responses to questions 4, 8 and 13 show a common misconception that the more terms used in a search will yield the best search results. Respondents need to know not to make the search too narrow, and yet include the significant terms in a search.

145

Area 2: Search Strategy

40�---------------' 35��-

------------;

30 25 20 15 10

5 O��--��--�


For question 2, 36.9% of respondents appear to recognize the problem of using terms, which do not correspond with the ones employed and preferred by the OPAC system. However, some of these respondents might have arrived at the answer by eliminating options, which are very unlikely to be true.

Only 16.3% of respondents chose the correct Boolean operator "OR" for question 9 to get more search results. This shows that the basic concept of "OR" in searching is not well understood.

Only 12.2% responded correctly by choosing C for question 11. Most respondents (77.6%) selected A or D, which indicates that they are not be able to use the correct search index by failing to differentiate between an author, a title and the subject of search.

Most respondents (81.8%) do not know the use of a database thesaurus in searching for preferred terms for a particular database. Using controlled vocabulary is an important search strategy when using databases.

Only 26.5% of respondents know the function of Boolean operator " AND" to restrict the number of search results.

Responses to questions 2, 9, 11, 12, and 16 show that most respondents lack the strategy to perform searching efficiently by not knowing which Boolean operators to use, not being aware of the use of a database thesaurus and not knowing which search index to use in an OPAC system.

Area 3: Document Types

30 25 20 15 10

5 o

- -----""""


2009 International Conference on Engineering Education (ICEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

Only 26.5% of the respondents know the usefulness of encyclopedias in providing an overview or summary of a topic.

Only about a quarter of the respondents (24.5%) recognize that a journal is the best document type to obtain the latest information on a subject. Quite a high percentage cited "Google" for E as a way to get new information. These respondents are probably not aware of the varying quality and reliability of information found on the internet as compared to journals.

None of the respondents selected all the characteristics of a scholarly journal. 30.6% of respondents are not able to identify any of the characteristics of a journal. Most respondents know the characteristics partially.

The responses to questions 3, 15, and 20 indicate that most respondents lack the knowledge in identifying the criteria of different document types, and subsequently cannot select the most appropriate document type to use for different types of information.

Area 4: Search Tools

60 50 40 30 20 10

o LR �

..::::=I I u:


Only 8.2% of the respondents know that the most efficient search tool to find journal articles is a database. A majority (75.5%) of respondents chose Google, which can provide links to some journals but requires more time to explore the web to get the articles.

About half (51 %) of the respondents recognize that a search engine cannot locate books in the library.

A very low percentage (4.1%) know that a library catalogue cannot access an article by author or article title, but only by the journal title. Not being able to use the search indexes correctly leads to waste of time in searching.

Most students gave incorrect answers or did not know what a meta search engine does (83.7%). This lack of knowledge about internet search engines can cause inefficiency in locating information.

Most students only know partially what information can be found using the library catalogue.

Responses to questions 1, 6, and 14 show that while most respondents tend to use an Internet search engine even to search for journal articles; they seem to be unaware of what a meta search engine can do. A low score for question 7 shows

146

their lack of knowledge on how to use the search index in an OPAC system or a library catalogue.

Area 5: Use of Results

20 �------------�

15.1..:-:=--1

10

5

O ��-&�


Most respondents (85.7%) cannot interpret a citation or recognize the corresponding document type. This lack of knowledge hinders efficient searching because the respondents will not be able to gauge the relevance and currency of a reference, and subsequently unable to select the best way to search for information.

About (81.6%) do not seem to know what a bibliography is, and subsequently will not be able to use it to find relevant documents.

A very small percentage of respondents (6.1%) know the criteria to evaluate an Internet site despite it becoming the most popular search medium today.

Only 4.1% of respondents know when to include a reference to an article they cite, thus may commit plagiarism without them realizing it.

The responses for questions 5, lO, 18, and 19 show that most respondents seriously lack the knowledge for making a sound evaluation of internet information, using information ethically, or using a citation to search for more information on a topic.

VIII. CONCLUSION

The summary of the results for the specific information literacy knowledge areas surveyed in this study is presented in Table 1.

TABLE 1: INFORMATION LITERACY SKILLS ADDRESSED BY THE QUESTIONNAIRE AND THE RESULTS OBTAINED.

Broad Specific knowledge of the %of information literacy skill skills addressed in this area No correct

by the questionnaire answers area

Identifying the main concepts in a topic by using key 4 18.4

Concept words.

Identification Differentiating between significant and non- 8 22.4 significant terms. Using words with similar 13 36.7


meaning to describe a concept. Using Boolean operator

16 26.5 "AND" function. Using Boolean operator

9 16.3 "OR " function.

Search Strategy Using the correct search II 12.2

indexes. Using a thesaurus to get the preferred vocabulary for a 12 6.3 particular database.

Knowing when to refer to an 3 26.5

encyclopedia

Document types Knowing when to refer to a 15 24.5

journal.

Knowing the criteria of a 20 0

scholarly journal.

Knowing when to use a 1 8.2

database. Knowing when to use a search engine such as 6 51

Search tools Google. Knowing when to use a

17 10.2 library catalogue. Knowing when to use a meta search engine such as 14 16.3 Copernic. Knowing what a bibliography

10 18.4 is. Recognizing the type of document that corresponds to 5 14.3

Use of Results a bibliographic reference. Knowing the criteria used in evaluating the quality of a 18 6.1 web site. Knowing when to include a

19 4.1 reference to avoid plagiarism.

Even though the sample size of respondents is small and does not represent all engineering students, the pilot study results reveal a serious lack of knowledge in all the main areas of information literacy. This is evident in the percentage of correct answers being less than 50% in all but one specific skill of information literacy. The percentage of correct answers is especially low (less than 10%) with respect to:

i) Knowing that a thesaurus should be used to get the most appropriate terminology in a particular database.

ii) Knowing the criteria of a scholarly journal.

iii) Knowing the criteria used in evaluating the quality of a web site.

iv) Knowing when to include a reference to avoid plagiarism.

Most respondents seem to be most familiar with using a search engine such as Google to fmd information. This is not surprising as they are part of today's digital natives. However, they seem to lack the search skills required for academic research, and seriously need to be aware of the ethics in using information. An implication of this study is the importance of performing an assessment of students' information literacy

147

skills, which may have been taken for granted. It also calls for a more effective academic programme to inculcate information literacy among engineering students. This may require a more integrated curriculum and collaboration between academicians and information literacy specialists. A well-designed information literacy academic programme is important to increase students' ability to conduct quality researches in their studies and future profession.

REFERENCES

[I] D. Mittermeyer and D. Quirion, "Information literacy of incoming first-year undergraduates in Quebec ". Paper presented at the Conference of Rectors and Principals of Quebec Universities, 2003.

[2] American Library Association, Association of College and Research Libraries. (2004). Information Literacy Competency Standards for Higher Education. Chicago: ALA. Retrieved on Sept I, 2009 from http://www.a1a.orgialalmgrps/divs/acrIlstandards/standards.pdf

[3] International Federation of Library Associations. Guidelines on Information Literacy for Lifelong Learning. Retrieved on Sept 1, 2009 from http://www.ifla.org/files/information-literacvlpublicationS/iflaguidelines-en.pdf

[4] UNESCO, "Towards Information Literacy Indicator," Paris: UNESCO, 2008, Retrieved on Sept I, 2009 from

http://www.ifla.orglfiles/information-Iiteracvlpublications/towardsinformation-literacy 2008-en.pdf

[5] S. Virkus, "Information literacy in europe: a literature review. Information Research," 2003, vol. 8 no. 4, July 2003.

[6] Ministry of Higher Education Malaysia , National Higher Education Action Plan 2007-2010. Retrieved on Sept 1,2009 from

http://www.mohe.gov.mvltransformasi/images/I bi.pdf

[7] S. Webber and B. Johnston, "Conceptions of information literacy: new perspectives and implications," Journal of Information Science, 2000, vol. 26 no.6, pp. 381 -397.

[8] T. Rafste, "From bud to blossom - how to become an information-literate person and learn a subject/topic," 2002. Retrieved on Aug 20, 2009 from

http://www· tfla.queenslibrary. orgl/Vltfla68lpapersl027-097e·pdf [9] J. S. Town, "Information literacy: definition, measurement and

impact," in A. Martin & H. Rader (Eds.) Information and IT literacy: Enabling Learning in the 21st Century, London: Facet Publishing, 2002, pp.53-65 .

[10] S. Taylor and M. Krog, "Problem-based Learning at Two Medical Libraries in Norway," 2000, DF Revy, vol. 23 no. 9, pp. 257 -259.

[11] A-M. Haraldstad, "Information Literacy -Curriculum Integration with Medical School's Syllabus," Liber Quarterly: The Journal of European Research Libraries, 2002, vol. 12 no. 2/3,192-198.

[12] D. Laurillard, Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies, 2nd ed., RoutJedge]a1mer, London, 2002.

[13] I. Fourie and D. Van Niekerk,. "Follow-up on the use of portfolio assessment for a module in research information skills: an analysis of its value," Education for Information, 2001, vol. 19 no. 2, pp 107 -126.

[14] E. Grassian and J. Kaplowitz, Information Literacy Instruction: Theory and Practice. Neal-Schuman, New York, NY, 2001.

[15] R. Catts, Information Skills Survey, Technical Manual, Canberra, CAUL, 2005.

[16] ETS . iSkills Assessment, 2008. Retrieved on Aug 30, 2009 from http://www.ets.org/portaIlsite/ets/menuitem.1488512ecfd5b8849a77bl3 bc3921509l?vgnextoid=159tUe3c27a85110VgnVCMI0000022f95190R CRD&vgnextchannel=e5b2a79898a851I OV gn VCMI0000022f95190RC RD

[ieee 2009 international conference on engineering education (iceed) - kuala lumpur, malaysia...

Documents