Assessing Information Literacy OnlineInternet LibrarianOctober 2008

Kevin Roddy and Margot HansonUniversity of Hawaii

Purpose & Description

LILO (Learning Information Literacy Online) is a project of the UH Libraries Information Literacy Committee (UHLILC) for the UH System. LILO is a productivity application that guides students’ research process using library and Web resources. Students create and add to an online Research Journal, documenting their research on a topic over a period of weeks.

The origin of LILO

The UH library system obtained the code to LOBO, the Library Online Basic Orientation at NC State, through an End-User License Agreement, in Spring 2004. The tutorial has been completely redesigned and is regularly updated by UH Libraries Information Literacy committee (LILC) members. Instructional faculty, librarian, and student input is very important.

LILO today

• Tutorial• Research journal• IM chat help• Email access to librarians and instructors• Videos• Links to interactive elements

Unique needs of Hawaii

• High level of demand for local topics research materials

• Low level of Hawaii-topic indexing in national/international databases

Changing times and capturing the interest of students

• Embedded and linked interactive material from various sources

Tutorial

• Six modules• General to specific

My LILO Journal

• Research journal• Students record research process

Chat assistance

• 24-hour chat reference service

Email access to librarians and instructors

Videos

• YouTube• Shared content by other libraries• Create our own

Links to interactive elements

Student Feedback

Students in Leeward CC instructor Donna Matsumoto’s classes said:

“LILO improved my research dramatically. It has everything you need and journals to keep you updated on where you are on your research. Without LILO, I would've procrastinated on this assignment.

Written Survey Results

95% of students (36 of 38) found that LILO is helpful in their research, and would recommend LILO to their friends.

The Research Journal, Citation Builder, and Assignment Calculator were identified as the most useful LILO tools.

Future enhancements

1. Rate usefulness of video, Web, and text content

2. Course syllabus integration

3. Pre-test evaluation and push

WASC – Western Association of Schools and Colleges (4-year+)

ACCJC – Accrediting Commission for Community and Junior Colleges

Assessment

Assessment!

Assessment Challenges

1. Meeting accrediting body (WASC, ACCJC)*, individual campus/system, ALA expectations

2. Faculty Buy-In

3. Data Analysis

WASC – Western Association of Schools and Colleges (four year/graduate)

ACCJC – Accrediting Commission for Community and Junior Colleges

1. Accreditation Directives (WASC)

Western Association of Schools and Colleges (WASC) Standard 2:

“Baccalaureate programs engage students an integrated course of study of sufficient breadth and depth to prepare them for work, citizenship, and a fulfilling life. These programs also ensure the development of core learning abilities and competencies including, but not limited to, college-level written and oral communication; college-level written and oral communication; college-level quantitative skills; information literacy; and the habit of critical analysis of data and argument.”

WASC 2001 Accreditation Handbook, Standard 2 (Achieving Educational Objectives Through Core Functions): Criteria for Review.

1. Accreditation Directives - ACCJC

• Standard II. A. 3. (Instruction - General Education requirements)

A.3.b. A capability to be a productive individual and life long learner: skills include oral and written communication, information competency, computer literacy, scientific and quantitative reasoning, critical analysis/logical thinking, and the ability to acquire knowledge through a variety of means.

• Standard II.C.1.b. (Library and Learning Support Services)

The institution provides ongoing instruction for users of library and other learning support services so that students are able to develop skills in information competency.

1. UH General Education “Hallmark” Directives

1. ALA IL Competency Standards

Standards, Performance Indicators, Outcomes (“SPIOs”)Standard One The information literate student determines the nature and extent of the information needed.

Performance Indicator 2 The information literate student identifies a variety of types and formats of potential sources for information

Outcome d Identifies the purpose and audience of potential resources (e.g., popular vs. scholarly, current vs. historical)

1. Educational Testing Service (ETS) and Information Literacy

ETS investment in IL:Information and Communication Technology

(ICT) Literacy Assessment

iSkills assessment

2. Faculty Buy-in• Librarian/Instructor Collaborative Model

– IL competency is now an institutional requirement – Collaboration facilitates “total student learning”

– Good evidence for tenure/promotion and self-study documents

• Promotion– Print format (e.g., club cards)– Department, writing intensive, new faculty meeting

– Campus Involvement– Developing student learning outcomes (SLOs) for courses – Service on the campus governance and policy committees

2. Assessing IL Online

• LILO shows Instructors:– Accreditation, UH System, ALA expectations– How to integrate LILO into course– How to obtain IL competency reports

• LILO Statistics modulehttp://www.hawaii.edu/lilo/0908/lilostats_2008_2009.php

3. Data Analysis

1) Response data in student journals

2) LILO delivers response data to instructors to evaluate separately if desired

3) LILO Librarians evaluate response data and send to Instructors

4) Librarians and Instructors use response data to improve content and questions

Evaluation Cycle

1. Accreditation, UniversityAnd ALA Directives

2. LILO Content and Questions

3. Student Responses

4. Score with Rubrics

5. Response Data to Facultyas a report

Question/Rubric/SPIO example

• http://www.hawaii.edu/lilo/• http://www.hawaii.edu/lilo/0908/eval_web_p

urpose.php

Scoring Module

• LILO librarians rate student responses online

• UH SLIS graduate students participate as service learning

LILO Software/Hardware

• PHPMyAdmin 2.8.0.2• MYSQL 4.1.20 - 29 tables• Dreamweaver 8/CS3 • UH System-Administered Server • Programming team of 1

– No previous programming background– Learned mostly from books (and lynda.com)

IL Reports show…

• Evidence of IL instruction and Assessment at UH

• Evidence of Librarian/Instructor Collaborative Effort

Invaluable data for Institutional Self-Studies!

Thanks from Margot and Kevin!.

mdhanson@hawaii.edu kroddy@hawaii.edu